Getting started
Install the gateway and get a first completion through the router.
archived documentation
llm-router docs
Self-hosted AI gateway -- operator documentation
v0.9.4
llm-router
Configuration
Environment variables and the models.json routing configuration.
Routing & resilience
Load balancing strategies, failover and connection keep-alive.
Security & auditing
Authentication, authorization, rate limiting and the GPG audit trail.
Observability
Prometheus metrics exposed by the router and how to scrape them.
REST API
Endpoint catalogue and the guide for adding your own endpoints.
CLI
llm-router command line reference: server lifecycle, models, utilities.
Python library
Embed the router in your own code with llm_router_lib.
Deployment
Helm chart, container image and the runtime knobs that matter in production.
Examples
Runnable stacks: vLLM, Ollama, LangChain, LlamaIndex and client SDKs.
- Integration Examples with LLM Router
- Full Stack with Local Models
- Using LlamaIndex with Local Models via the LLM‑Router
- vLLM + google/gemma‑3‑12b‑it – Quick‑Start Guide (Ubuntu)
- vLLM + speakleash/Bielik-11B-v2.3-Instruct – Przewodnik Szybkiego Startu (Ubuntu)
Release notes
What changed in every released version.