llm-router CLI — Command Reference#

Package: llm-router Entry points:

• llm-router — main CLI tool (auth, anonymizer, config, util)

Quick Start#

pip install llm-router[api]
llm-router --help
llm-router --version

Top-Level Commands#

llm-router auth — API Key & Authentication Management#

Command Tree#

llm-router auth key <command>          # API key lifecycle
llm-router auth policy <command>       # Policy management
llm-router auth rate-limit <command>   # Per-key rate limit overrides

Shared Flags (store-backed subcommands)#

Note: These flags are shared by all store-backed subcommands — key generate, key list, key delete, key disable, key enable, key rotate, policy create, rate-limit apply, and rate-limit remove. They do not apply to the read-only policy list and rate-limit list subcommands.

Note: Each --auth-redis-* flag falls back to a matching LLM_ROUTER_AUTH_REDIS_<HOST|PORT|DB|PASSWORD|PROTOCOL> environment variable before the built-in default is used. These auth-specific Redis flags are separate from the general LLM_ROUTER_REDIS_* env vars.

Key Management: llm-router auth key <command>#

generate — Create a new API key#

llm-router auth key generate \
  --policy developer \
  --expires 1750000000 \
  --store memory

Output: sk-llmr-live-<base62> key (plaintext shown once at creation) plus the generated Key ID: (e.g. key-fe8fc388) — use the ID with list/delete/ disable/enable/rotate.

list — List all API keys#

llm-router auth key list --store memory [--json]

delete <key-id> — Delete a key permanently#

llm-router auth key delete <key-id> --store memory

disable <key-id> — Deactivate without deleting#

llm-router auth key disable <key-id> [--store memory]

enable <key-id> — Re-activate a disabled key#

llm-router auth key enable <key-id> [--store memory]

rotate <key-id> — Generate a replacement key#

llm-router auth key rotate <key-id> --grace 3600 [--store memory]

Policy Management: llm-router auth policy <command>#

list — List builtin policies#

llm-router auth policy list

Builtin policies:

create <name> [<json-policy>] — Create a custom policy#

# inline JSON
llm-router auth policy create my-team '{
  "can_access": true,
  "rate_limit": 120,
  "model_whitelist": ["gpt-4", "llama-3"]
}' --store memory

# from a file, or from stdin (-) — avoids leaking policy JSON into shell history
llm-router auth policy create my-team --file my-team.json
cat my-team.json | llm-router auth policy create my-team --file -

Persistence: custom policies are saved to $LLM_ROUTER_AUTH_CUSTOM_POLICIES_FILE (default: ~/.llm-router/configs/auth/custom-policies.json) and are resolved by the server in new processes — no restart required. policy list marks them with (custom).

Rate Limit Overrides: llm-router auth rate-limit <command>#

Note: Rate limiting is always active when authentication is enabled. These commands manage per-key overrides on top of the default policy rate limit (60 rpm).

list — Show available presets#

llm-router auth rate-limit list

apply <key-id> --preset <name> — Set a per-key rate limit via preset#

llm-router auth rate-limit apply <key-id> --preset pro --store memory

Available presets:

remove <key-id> — Revert to default policy rate limit#

llm-router auth rate-limit remove <key-id> [--store memory]

llm-router config — Provider Discovery & Config Merging#

Command Tree#

llm-router config discover <host...> [-o FILE] [--all-ports] [--no-active]  # Auto-discover providers
llm-router config merge <configs...> [-o FILE]                              # Merge multiple configs

discover — Scan hosts for local LLM servers#

llm-router config discover localhost -o models-config.json
llm-router config discover localhost 192.168.1.50 --all-ports
llm-router config discover "10.0.0.1:8080" "ollama.local:11434"

Note on port scanning: When you pass host:port (e.g. "192.168.100.66:9090"), the scanner checks that exact port first for each provider type (Ollama, vLLM, LM Studio, llama.cpp, KoboldCpp, TabbyAPI). If no models are found on the explicit port, it continues scanning the default ports for every discovered provider. To restrict scanning to only the explicit port (without fallback), use --all-ports in combination with specifying all known ports explicitly.

Auto-discovered providers:

merge — Merge multiple models-config.json files#

llm-router config merge base.json override.json -o merged-config.json

Merges provider entries recursively (overlay wins on conflict), unions active_models, and deduplicates providers by api_host.

llm-router anonymizer run — Text Anonymization#

llm-router anonymizer run --algorithm fast_masker [input_file] -o output_file \
  --disable-phone --disable-url --disable-ip --disable-pesel --disable-email

llm-router util — Utility Apps (translate / genai-classifier / genai-data-augmentation)#

Light, dependency-free ports of the llm-router-utils applications. They read local JSON/JSONL files only (no HuggingFace datasets, no pandas / openpyxl / XLSX, no tenacity) and talk to the router through LLMRouterClient.

Command Tree#

llm-router util translate --llm-router-host URL --model M --dataset-path d.jsonl [--accept-field f] [-o out.jsonl]
llm-router util genai-classifier --dataset-dir DIR --prompts-dir P --output-dir O [--model-name M]
llm-router util genai-data-augmentation --dataset-path d.jsonl --prompt-file P --labels a,b [--output-dir DIR]

translate — Translate texts in JSON/JSONL datasets#

llm-router util translate \
  --llm-router-host http://localhost:8080 \
  --model speakleash/Bielik-11B-v2.3-Instruct \
  --dataset-path data.jsonl --dataset-path more.json \
  --accept-field text --accept-field title

Output: without -o, each input <stem> writes <stem>.translated.jsonl next to it; with -o, all records go to that one file. Input files are never overwritten.

Default behavior: without --accept-field, all string-valued fields of each record are sent for translation; non-string values and other record fields pass through unchanged.

genai-classifier — Classify translated datasets (JSONL only)#

llm-router util genai-classifier \
  --dataset-dir ./data --prompts-dir ./prompts --output-dir ./out \
  --model-name gpt-oss:120b --num-workers 2 --n-sample 50

Produces <name>.jsonl and <name>_clean_labels.jsonl (no XLSX).

genai-data-augmentation — Augment a local JSONL dataset#

llm-router util genai-data-augmentation \
  --dataset-path dataset.jsonl --prompt-file prompt.txt --labels cat,dog

Produces <stem>_augmented.jsonl and <stem>_augmented-train.jsonl (no XLSX).

Seed File (Memory Store)#

When --store memory, keys are persisted to a seed file:

~/.llm-router/configs/auth/memory-keys.json

After every generate, delete, disable, enable, or rotate operation the CLI automatically updates the seed file. The router reads this file on startup and after each request, so changes are visible without restart.

Seed File Format (ApiKeyRecord fields)#

Key Format#

All generated keys follow the sk-llmr-live-<base62> format:

sk-llmr-live-aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789abcdefABCDEF123456789

See Also#

• Authentication docs — full auth architecture, seed files, deployment options
• Rate Limiting docs — sliding-window algorithm, monitoring, presets