Authentication & Authorization#

API-key-based authentication with per-endpoint policies, rate limiting, audit trail, and Prometheus metrics.

Enabled by default: LLM_ROUTER_AUTH_ENABLED=false — set to "true" to enforce authentication.

Architecture#

Client Request → AuthMiddleware → Key Store Lookup → Permission Engine → Rate Limiter → Endpoint
               → Audit Bridge → AnyRequestAuditor
               → AuthMetrics → Prometheus

Components#

Environment Variables#

All environment variables are documented in ENV_DEFINITIONS.md. Auth-specific vars start with LLM_ROUTER_AUTH_*.

CLI Commands#

Key Management#

# Generate a new key (persists to seed file when --store memory)
llm-router auth key generate --policy developer --store memory

# List all keys
llm-router auth key list --store memory

# List with plaintext keys visible
llm-router auth key list --store memory --reveal

# Delete a key
llm-router auth key delete key-id

# Disable a key (revokes access immediately)
llm-router auth key disable key-id [--store memory]

# Enable a previously disabled key
llm-router auth key enable key-id [--store memory]

# Rotate a key (old key stays valid for grace_period)
llm-router auth key rotate key-id --grace 3600

# Reveal plaintext key (only in memory store)
llm-router auth key reveal key-id

Policy Management#

# List builtin policies
llm-router auth policy list

# Create a new policy
llm-router auth policy create my-team '{"can_access": true, "rate_limit": 120}'

Rate Limit#

# List available rate-limit presets
llm-router auth rate-limit list

# Apply a rate-limit preset to an existing key
llm-router auth rate-limit apply key-id --preset pro

# Remove rate-limit override from a key (revert to global default)
llm-router auth rate-limit remove key-id

Seed File (Memory Store)#

When using --store memory, keys are stored in process memory — they are lost on restart. To persist keys across restarts (and between the CLI and router processes), use a seed file.

Seed File Format#

The seed file is a JSON array. Each record must have key_plain and optionally the following fields (matching the ApiKeyRecord dataclass):

[
  {
    "key_id": "manual-key-001",
    "key_plain": "sk-litm-my-key-1234567890abcdefghij",
    "policy_name": "developer",
    "is_active": true,
    "expires_at": null,
    "created_at": 1718000000,
    "last_used_at": null,
    "rotate_at": null,
    "grace_until": null,
    "metadata": {},
    "policy_override": {
      "rate_limit": 300
    }
  },
  {
    "key_id": "manual-key-002",
    "key_plain": "sk-litm-my-readonly-key-0987654321zyxwvutsrq",
    "policy_name": "chat",
    "is_active": true,
    "expires_at": 1750000000,
    "created_at": 1718000000,
    "last_used_at": null,
    "rotate_at": null,
    "grace_until": null,
    "metadata": {
      "team": "backend"
    }
  }
]

How It Works#

• On router startup: MemoryKeyStore reads the seed file and loads all keys into memory.
• After CLI key operations: generate, delete, and rotate automatically write back to the seed file.
• No restart needed: Changes made via CLI are visible to the router on the next request (the router reloads the seed file on each request via the middleware).

Default Location#

~/.llm-router/configs/auth/memory-keys.json

Permission Engine#

The permission engine resolves key → policy → endpoint permissions:

1. Public endpoints — always bypass auth (health checks, version, etc.)
2. Key authentication — bcrypt hash lookup in key store
3. Key validity — check expiry, rotation, grace period
4. Policy resolution — named policy or inline override
5. Endpoint permission — per-endpoint + per-model check
6. Rate limit — sliding window check per key+IP

Builtin Policies#

Prometheus Metrics#

Auth metrics are registered when LLM_ROUTER_USE_PROMETHEUS=true:

Key Format#

Generated keys follow the sk-litm-<base62> format:

sk-litm-aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789abcdefABCDEF123456789

• Prefix: sk-litm- (configurable via LLM_ROUTER_AUTH_KEY_PREFIX)
• Entropy: 48 bytes cryptographically random (configurable via LLM_ROUTER_AUTH_KEY_LENGTH)
• Charset: base62 (a-zA-Z0-9)

Deployment Options#

Choose a key store backend based on your deployment environment. Each option is documented below with full environment configuration, key management commands, and operational notes.

1️⃣ In-Memory Store — Development / Quick Start#

Use when: You want to try auth quickly, run locally, or have a single-process deployment.

Pros: Zero external dependencies, instant setup. Seed file provides persistence across restarts. Cons: Still per-process — not multi-process safe even with seed file.

# 1. Create seed file (keys persist to disk)
mkdir -p ~/.llm-router/configs/auth
cat > ~/.llm-router/configs/auth/memory-keys.json << 'EOF'
[
  { "key_plain": "sk-litm-my-dev-key", "policy_name": "developer" }
]
EOF

# 2. Set environment variables
export LLM_ROUTER_AUTH_ENABLED=true
export LLM_ROUTER_AUTH_KEY_STORE=memory

# 3. Verify keys are loaded
llm-router auth key list --store memory --reveal

# 4. Generate more keys (auto-persisted to seed file)
llm-router auth key generate --policy readonly --store memory

# 5. Run router (keys loaded automatically from seed file)
python -m llm_router_api.rest_api

# 6. Use the key
curl -H "x-api-key: sk-litm-my-dev-key" https://host/api/chat/completions

Operational notes:

• Seed file auto-updates after generate, delete, and rotate.
• Keys are visible to both CLI and router via the same seed file.
• Suitable for development, CI, and single-node deployments.
• No Redis or Vault required.

2️⃣ Redis Store — Multi-Process / Stateful Single-Node#

Use when: You run multiple workers/processes and need persistent key storage without Vault.

Pros: Persistent keys across restarts (if Redis is durable), multi-process safe. Cons: Requires Redis instance. No encryption-at-rest (use TLS for production).

# 1. Start Redis
redis-server

# 2. Set environment variables
export LLM_ROUTER_AUTH_ENABLED=true
export LLM_ROUTER_AUTH_KEY_STORE=redis
export LLM_ROUTER_AUTH_REDIS_HOST=127.0.0.1
export LLM_ROUTER_AUTH_REDIS_PORT=6379
export LLM_ROUTER_AUTH_REDIS_DB=0
# export LLM_ROUTER_AUTH_REDIS_PASSWORD=secret  # if Redis requires auth

# 3. Generate keys (written to Redis)
llm-router auth key generate --policy developer --store redis
llm-router auth key list --store redis

# 4. Run router (reads keys from same Redis instance)
python -m llm_router_api.rest_api

# 5. Use the key
curl -H "x-api-key: sk-litm-..." https://host/api/chat/completions

Operational notes:

• Keys stored as JSON strings in Redis under secret:llm-router:api-keys:<key_id>.
• Works across multiple Flask workers — all workers see the same key set.
• Redis is already required for load balancing (same instance works).
• For production, use Redis Sentinel or a managed Redis service with TLS.

3️⃣ HashiCorp Vault — Production / Multi-Cluster / Enterprise#

Use when: You need enterprise-grade key management, secret rotation, or multi-cluster consistency.

Pros: Centralized key management, encryption, RBAC, audit logging. Cons: Requires Vault infrastructure and auth method configuration.

3a. Kubernetes Auth (recommended for K8s deployments)#

# 1. Set environment variables
export LLM_ROUTER_AUTH_ENABLED=true
export LLM_ROUTER_AUTH_KEY_STORE=vault
export LLM_ROUTER_AUTH_VAULT_ADDR=https://vault.example.com
export LLM_ROUTER_AUTH_VAULT_PATH=secret/data/llm-router/api-keys
export LLM_ROUTER_AUTH_VAULT_AUTH_METHOD=kubernetes

# The K8s service account JWT is read from:
# /var/run/secrets/kubernetes.io/serviceaccount/token (hardcoded)

# 2. Generate and use keys
llm-router auth key generate --policy developer --store vault
llm-router auth key list --store vault
curl -H "x-api-key: sk-litm-..." https://host/api/chat/completions

Operational notes:

• Vault K8s auth reads the service account JWT automatically.
• Keys stored under {vault_path}/{key_id}/data in KV v2 format.
• On rotation the old key is deactivated (is_active=False) and a new key is written.
• Ensure the Vault Role has read and create policies for the mount path.

3b. AppRole Auth (non-K8s / VM deployments)#

export LLM_ROUTER_AUTH_ENABLED=true
export LLM_ROUTER_AUTH_KEY_STORE=vault
export LLM_ROUTER_AUTH_VAULT_ADDR=https://vault.example.com
export LLM_ROUTER_AUTH_VAULT_AUTH_METHOD=approle
export LLM_ROUTER_AUTH_VAULT_ROLE_ID=your-approle-role-id
export LLM_ROUTER_AUTH_VAULT_SECRET_ID=your-approle-secret-id

3c. Token Auth (manual / CI pipelines)#

export LLM_ROUTER_AUTH_ENABLED=true
export LLM_ROUTER_AUTH_KEY_STORE=vault
export LLM_ROUTER_AUTH_VAULT_ADDR=https://vault.example.com
export LLM_ROUTER_AUTH_VAULT_AUTH_METHOD=token
export LLM_ROUTER_AUTH_VAULT_TOKEN=s.your-vault-token-here

Operational notes:

• For production, use AppRole or K8s auth — token auth is less secure.
• Vault path is configurable per environment (e.g., secret/data/llm-router-staging/api-keys).
• Supports key rotation: llm-router auth key rotate key-id --grace 3600.

Comparison Matrix#

Recommendation:

• Dev / testing: Use memory with seed file (zero setup).
• Staging / small production: Use redis (already required for load balancing).

• Multi-cluster / enterprise: Use vault with K8s or AppRole auth.

• Keys are stored as bcrypt hashes — plaintext is never persisted in the store

• The plaintext key is returned only once at creation time
• Rate limiting prevents brute-force attacks even if key hashes leak
• Audit bridge records all auth events (success, failure, rate limit) for compliance

See Also#

• Rate Limiting — sliding-window rate limiting, configuration, and monitoring
• Auditing subsystem — tamper-evident audit logging