Getting started
Install the gateway and get a first completion through the router.
archived documentation
llm-router docs
Self-hosted AI gateway -- operator documentation
v0.6.4
llm-router
Security & auditing
Authentication, authorization, rate limiting and the GPG audit trail.
REST API
Endpoint catalogue and the guide for adding your own endpoints.
- REST API reference
- Endpoints Overview
- Jak tworzyć endpointy w llm-proxy-api – przewodnik
- Keep‑Alive Utility Overview
- Load Balancing Strategies
CLI
llm-router command line reference: server lifecycle, models, utilities.
Python library
Embed the router in your own code with llm_router_lib.
Deployment
Helm chart, container image and the runtime knobs that matter in production.
Examples
Runnable stacks: vLLM, Ollama, LangChain, LlamaIndex and client SDKs.
- Integration Examples with LLM Router
- Full Stack with Local Models
- Using LlamaIndex with Local Models via the LLM‑Router
- vLLM + google/gemma‑3‑12b‑it – Quick‑Start Guide (Ubuntu)
- vLLM + speakleash/Bielik-11B-v2.3-Instruct – Przewodnik Szybkiego Startu (Ubuntu)
Release notes
What changed in every released version.