llm-router CLI — Command Reference#

Package: llm-router Entry points:

• llm-router — main CLI tool (auth, anonymizer, config, util, server, completion)

Quick Start#

pip install llm-router[api]
llm-router --help
llm-router --version

Top-Level Commands#

llm-router auth — API Key & Authentication Management#

Command Tree#

llm-router auth key <command>          # API key lifecycle
llm-router auth policy <command>       # Policy management
llm-router auth rate-limit <command>   # Per-key rate limit overrides

Shared Flags (store-backed subcommands)#

Note: These flags are shared by all store-backed subcommands — key generate, key list, key delete, key disable, key enable, key rotate, policy create, rate-limit apply, and rate-limit remove. They do not apply to the read-only policy list and rate-limit list subcommands.

Note: Each --auth-redis-* flag falls back to a matching LLM_ROUTER_AUTH_REDIS_<HOST|PORT|DB|PASSWORD|PROTOCOL> environment variable before the built-in default is used. These auth-specific Redis flags are separate from the general LLM_ROUTER_REDIS_* env vars.

Key Management: llm-router auth key <command>#

generate — Create a new API key#

llm-router auth key generate \
  --policy developer \
  --expires 1750000000 \
  --store memory

Output: sk-llmr-live-<base62> key (plaintext shown once at creation) plus the generated Key ID: (e.g. key-fe8fc388) — use the ID with list/delete/ disable/enable/rotate.

list — List all API keys#

llm-router auth key list --store memory [--json]

delete <key-id> — Delete a key permanently#

llm-router auth key delete <key-id> --store memory

disable <key-id> — Deactivate without deleting#

llm-router auth key disable <key-id> [--store memory]

enable <key-id> — Re-activate a disabled key#

llm-router auth key enable <key-id> [--store memory]

rotate <key-id> — Generate a replacement key#

llm-router auth key rotate <key-id> --grace 3600 [--store memory]

Policy Management: llm-router auth policy <command>#

list — List builtin policies#

llm-router auth policy list

Builtin policies:

create <name> [<json-policy>] — Create a custom policy#

# inline JSON
llm-router auth policy create my-team '{
  "can_access": true,
  "rate_limit": 120,
  "model_whitelist": ["gpt-4", "llama-3"]
}' --store memory

# from a file, or from stdin (-) — avoids leaking policy JSON into shell history
llm-router auth policy create my-team --file my-team.json
cat my-team.json | llm-router auth policy create my-team --file -

Persistence: custom policies are saved to $LLM_ROUTER_AUTH_CUSTOM_POLICIES_FILE (default: ~/.llm-router/configs/auth/custom-policies.json) and are resolved by the server in new processes — no restart required. policy list marks them with (custom).

Rate Limit Overrides: llm-router auth rate-limit <command>#

Note: Rate limiting is always active when authentication is enabled. These commands manage per-key overrides on top of the default policy rate limit (60 rpm).

list — Show available presets#

llm-router auth rate-limit list

apply <key-id> --preset <name> — Set a per-key rate limit via preset#

llm-router auth rate-limit apply <key-id> --preset pro --store memory

Available presets:

remove <key-id> — Revert to default policy rate limit#

llm-router auth rate-limit remove <key-id> [--store memory]

llm-router config — Provider Discovery & Config Merging#

Command Tree#

llm-router config discover <host...> [-o FILE] [--all-ports] [--no-active]  # Auto-discover providers
llm-router config merge <configs...> [-o FILE]                              # Merge multiple configs

discover — Scan hosts for local LLM servers#

llm-router config discover localhost -o models-config.json
llm-router config discover localhost 192.168.1.50 --all-ports
llm-router config discover "10.0.0.1:8080" "ollama.local:11434"

Note on port scanning: When you pass host:port (e.g. "192.168.100.66:9090"), the scanner checks that exact port first for each provider type (Ollama, vLLM, LM Studio, llama.cpp, KoboldCpp, TabbyAPI). If no models are found on the explicit port, it continues scanning the default ports for every discovered provider. To restrict scanning to only the explicit port (without fallback), use --all-ports in combination with specifying all known ports explicitly.

Auto-discovered providers:

merge — Merge multiple models-config.json files#

llm-router config merge base.json override.json -o merged-config.json

Merges provider entries recursively (overlay wins on conflict), unions active_models, and deduplicates providers by api_host.

llm-router anonymizer run — Text Anonymization#

llm-router anonymizer run --algorithm fast_masker [input_file] -o output_file \
  --disable-phone --disable-url --disable-ip --disable-pesel --disable-email

llm-router util — Utility Apps (translate / genai-classifier / genai-data-augmentation)#

Light, dependency-free ports of the llm-router-utils applications. They read local JSON/JSONL files only (no HuggingFace datasets, no pandas / openpyxl / XLSX, no tenacity) and talk to the router through LLMRouterClient.

Command Tree#

llm-router util translate --llm-router-url URL --model M --dataset-path d.jsonl [--accept-field f] [-o out.jsonl]
llm-router util genai-classifier --dataset-dir DIR --prompts-dir P --output-dir O [--model-name M]
llm-router util genai-data-augmentation --dataset-path d.jsonl --prompt-file P --labels a,b [--output-dir DIR]

translate — Translate texts in JSON/JSONL datasets#

llm-router util translate \
  --llm-router-url http://localhost:8080 \
  --model speakleash/Bielik-11B-v2.3-Instruct \
  --dataset-path data.jsonl --dataset-path more.json \
  --accept-field text --accept-field title

Output: without -o, each input <stem> writes <stem>.translated.jsonl next to it; with -o, all records go to that one file. Input files are never overwritten.

Default behavior: without --accept-field, all string-valued fields of each record are sent for translation; non-string values and other record fields pass through unchanged.

genai-classifier — Classify translated datasets (JSONL only)#

llm-router util genai-classifier \
  --dataset-dir ./data --prompts-dir ./prompts --output-dir ./out \
  --model-name gpt-oss:120b --num-workers 2 --n-sample 50

Produces <name>.jsonl and <name>_clean_labels.jsonl (no XLSX).

genai-data-augmentation — Augment a local JSONL dataset#

llm-router util genai-data-augmentation \
  --dataset-path dataset.jsonl --prompt-file prompt.txt --labels cat,dog

Produces <stem>_augmented.jsonl and <stem>_augmented-train.jsonl (no XLSX).

llm-router server — REST API Server Lifecycle#

Start / stop / reload the REST API server (a managed daemon with a PID file), inspect its status, or follow its log. Unless your environment already sets them, start applies the same LLM_ROUTER_* defaults as run-rest-api-gunicorn.sh.

start accepts the usual tuning flags (--foreground, --host, --port, --server {gunicorn,waitress,flask}, --models-config, --lb-strategy, --default-lang, --debug, --log-file, --pid-file, --auth, --redis-host, --redis-port, --redis-db, --redis-password, --auth-redis-host, --auth-redis-port, --auth-redis-db, --auth-redis-password). The daemon log (--log-file) follows LLM_ROUTER_LOG_FILENAME when it is set in the shell — a bare file name is resolved against the launch CWD — and defaults to ~/.llm-router/server.log only when the variable is unset. Every LLM_ROUTER_* variable in effect at launch — defaults + shell env + CLI overrides — is snapshotted into the run record (<pid-file>.run) so status can show exactly how the server was started.

status — colored status card#

status renders a compact, color-coded card instead of a flat list (the Environment section shown below is hidden by default — add --show-env to reveal it):

  ✓ All good
  ●  Server is running (pid 7400, gunicorn, port 8080)

  Details
    PID            7400
    Log            /home/user/project/llm-router.log
    Started        2026-09-07T23:55:00+0200
    Server         gunicorn
    Host           0.0.0.0
    Port           8080
    Models config  /home/user/project/resources/configs/models-config.json
    Command        /usr/bin/python3 -m llm_router_api.rest_api

  Environment (5)
    LLM_ROUTER_BALANCE_STRATEGY  weighted
    LLM_ROUTER_REDIS_PASSWORD    ****
    LLM_ROUTER_SERVER_PORT       8080

The Log row shows the application's own rotating log (LLM_ROUTER_LOG_FILENAME, default llm-router.log). When that variable is a bare file name (no directory part) the server writes it to the CWD from which it was launched, so the run record stores the absolute path (<launch-dir>/llm-router.log) and status displays it as-is. In daemon mode the daemon's captured stdout/stderr are appended to the same file, so a single log row is enough (without the variable the daemon capture falls back to ~/.llm-router/server.log). When the server is not running the header/dot turn red (✗ Not running), and status exits with code 1.

The Models config row behaves the same way as Log: when the server was started with a relative path (e.g. resources/configs/models-config.json), the run record stores the absolute path anchored to the launch CWD, so status shows the real location of the file.

Security: environment values whose key names a credential (*PASSWORD*, *SECRET*, *TOKEN*, *API_KEY*, *CREDENTIAL*) are masked as **** so secrets are never echoed to the terminal.

log — follow the server log#

log follows the application's own log by default — the file set by LLM_ROUTER_LOG_FILENAME at start time (from the run record; a bare file name is resolved against the launch CWD). This works for both daemon and --foreground servers, since the run record is written in both cases:

llm-router server log                              # app log: last 20 lines, then follow
llm-router server log --lines 200                  # more history
llm-router server log --no-follow                  # one-shot tail, then exit
llm-router server log --log-file ~/.llm-router/server.log   # daemon's console log (daemon mode only)

stop, reload and status additionally accept --pid-file (default ~/.llm-router/server.pid), and stop accepts --force to skip the SIGTERM grace period and send SIGKILL.

llm-router completion — Shell Tab-Completion (bash / zsh)#

Generates a tab-completion script from the live CLI tree — commands, sub-commands at every nesting level (e.g. auth key generate, anonymizer run, config discover) and all long options.

Manual installation:

eval "$(llm-router completion bash)"        # once per shell
# or permanently:
source <(llm-router completion zsh)

--install writes the script straight into the default rc file for the shell (~/.bashrc for bash, ~/.zshrc for zsh; created if missing):

llm-router completion bash --install
llm-router completion zsh --install

Re-running --install is idempotent: the script is wrapped in # >>> llm-router completion (<shell>) >>> markers and any previous block is replaced in place, so no duplicates accumulate in your rc file.

Seed File (Memory Store)#

When --store memory, keys are persisted to a seed file:

~/.llm-router/configs/auth/memory-keys.json

After every generate, delete, disable, enable, or rotate operation the CLI automatically updates the seed file. The router reads this file on startup and after each request, so changes are visible without restart.

Seed File Format (ApiKeyRecord fields)#

Key Format#

All generated keys follow the sk-llmr-live-<base62> format:

sk-llmr-live-aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789abcdefABCDEF123456789

See Also#

• Authentication docs — full auth architecture, seed files, deployment options
• Rate Limiting docs — sliding-window algorithm, monitoring, presets