---
name: tenant-agent-management
description: Manage multi-tenant Hermes WebUI, 9router, and subdomains.
tags: [hermes, multi-tenant, 9router, nginx, subdomain, mrhermes]
---

# Tenant Agent Management (mrhermes.ir)

This skill provides operational procedures for managing multi-tenant Hermes WebUI instances mapped to subdomains (e.g., `username.mrhermes.ir`), isolating models behind 9Router (`MrHermes` combo), and tracking token usage per user.

## Architecture Overview
- **Domain:** `mrhermes.ir` (wildcard `*.mrhermes.ir` routed to server IP `77.237.90.133`).
- **Web Server:** Nginx acting as reverse proxy with WebSocket (`Upgrade`, `Connection`) support.
- **AI Gateway:** 9Router listening only on `127.0.0.1:20128` (external access blocked).
- **Model Exposure:** All tenants are locked to model `MrHermes` via `custom:9router` with `discover_models: false` to hide internal upstreams (Gemini, Claude, etc.).
- **Usage Tracking:** Every tenant has a distinct API key in 9Router SQLite database (`/root/.9router/db/data.sqlite`).

## CLI Commands

### 1. Provision New Tenant / Subdomain
To provision a new tenant whenever a user purchases an agent:
```bash
create-agent-tenant <subdomain_name> [optional_password]
```
Example:
```bash
create-agent-tenant ali AliHermes2026
```
This automatically:
1. Creates an isolated Linux user `/home/<username>`.
2. Allocates the next available internal port (8788, 8789, ...).
3. Generates a dedicated API key in 9Router (`key-<username>`).
4. Configures `~/.hermes/config.yaml` locked to `MrHermes`.
5. Creates and enables the systemd service `hermes-webui-<username>.service`.
6. Configures Nginx reverse proxy for `<username>.mrhermes.ir`.

### 2. View Token Usage Per User
To see token consumption (Prompt, Completion, Total tokens) grouped by each tenant:
```bash
check-user-usage
```

### 3. Password & Service Rules (Pitfalls & Lessons)
- **Systemd Password Variable:** Never use raw unescaped `$` inside `Environment="HERMES_WEBUI_PASSWORD=..."` in systemd service files, as systemd expands `$var` to empty strings. Use alphanumeric + symbols without unescaped `$`.
- **File Permissions:** Always ensure `~<user>/.hermes/webui/settings.json` is owned by `<user>:<user>`, not `root`.
- **9Router Security:** Keep `Environment=HOSTNAME=127.0.0.1` in `/etc/systemd/system/9router.service`. Never expose port 20128 to `0.0.0.0`. Access dashboard via SSH tunnel:
  ```bash
  ssh -L 20128:127.0.0.1:20128 root@77.237.90.133
  ```