{"version":"0.0.5","pages":[{"k":"index-2.html","t":"Plugins overview","s":"Plugins overview","x":"## Overview The **LLM‑Router** project ships with a modular plugin system that lets you plug‑in **anonymizers** (also called *maskers*) and **guardrails** into request‑processing pipelines. Each plugin implements a tiny, well‑defined inter…","h":["Overview","1. Anonymizers (Maskers)","1.1 What they do","1.2 Built‑in anonymizer plugins","1.3 How a masker is used","2. Guardrails","2.1 What they do","2.2 Built‑in guardrail plugins","2.3 How a guardrail is used","2.5 ML-Based PII Classification","How it complements regex maskers","Quick start","2.6 Polish Identification Regex Patterns","Available Polish rules","How they work","Adding rules to the pipeline","2.7 Semantic Routing (Model Selection)","3. Pipelines","3.1 Registration","3.2 Configuration","4. Adding a New Plugin","5. Retrieval‑Augmented Generation (RAG) Support","5.1 What the plugin does","5.2 Environment variables"],"b":"Overview # The LLM‑Router project ships with a modular plugin system that lets you plug‑in anonymizers (also called maskers ) and guardrails into request‑processing pipelines. Each plugin implements a tiny, well‑defined interface ( apply ) and can be composed in an ordered list to form a * pipeline *. Pipelines are instantiated by the MaskerPipeline and GuardrailPipeline classes and are driven automatically by the endpoint logic in endpoint_i.py . 1. Anonymizers (Maskers) # 1.1 What they do # Goal – Remove or replace personally‑identifiable information (PII) from a payload before it reaches the LLM or an external service. Typical strategy – Run a pipeline of maskers that locate spans corresponding to IDs, emails, IPs, etc., and replace each span with a placeholder such as {{MASKED_ITEM}} . 1.2 Built‑in anonymizer plugins # Plugin Description Technical notes FastMaskerPlugin ( fast_masker_plugin.py ) Thin wrapper around the FastMasker utility class. Receives a JSON‑compatible payload and returns the same payload with all detected PII masked. Implements PluginInterface . The heavy lifting is delegated to FastMasker.mask_payload(payload) . No extra I/O; the FastMasker instance is created once in __init__ . 1.3 How a masker is used # The endpoint (e.g. EndpointI._do_masking_if_needed ) checks the global flag FORCE_MASKING . If enabled, it creates a MaskerPipeline with the list of masker plugin identifiers (e.g. [\"fast_masker\"] ). The pipeline calls each plugin’s apply method sequentially, feeding the output of one as the input of the next. The final payload – now stripped of PII – proceeds to the rest of the request flow (guardrails, model dispatch, etc.). 2. Guardrails # 2.1 What they do # Goal – Verify that a request (or its response) complies with policy rules (e.g. no hateful, illegal, or unsafe content). Typical strategy – Split the payload into manageable text chunks, run a pipeline of guardrails, aggregate per‑chunk scores, and decide whether the overall request is safe. 2.2 Built‑in guardrail plugins # Plugin Description Technical notes NASKGuardPlugin ( nask_guard_plugin.py ) HTTP‑based guardrail that forwards the payload to the external NASK guardrail service ( /nask_guard endpoint) and returns a boolean safe flag together with the raw response. Inherits from HttpPluginInterface . The apply method calls _request(payload) (provided by the base class) and extracts results[\"safe\"] . Errors are caught and logged; on failure the plugin returns (False, {}) . SojkaGuardPlugin ( sojka_guard_plugin.py ) HTTP‑based guardrail that forwards the payload to the Sójka guardrail service ( /sojka_guard endpoint) and returns a safety flag. Mirrors the design of NASKGuardPlugin . The endpoint_url is built from the LLM_ROUTER_GUARDRAIL_SOJKA_GUARD_HOST environment variable. On success it returns (True, response) , otherwise (False, {}) . (Implicit) GuardrailProcessor ( processor.py ) Core logic used by the internal NASK guardrail Flask route ( nask_guardrail ). Tokenises the payload, creates overlapping chunks, runs a Hugging‑Face text‑classification pipeline, and produces a detailed safety report. Handles model loading ( AutoTokenizer , pipeline(\"text‑classification\") ), chunking ( _chunk_text ), and scoring thresholds ( MIN_SCORE_FOR_SAFE , MIN_SCORE_FOR_NOT_SAFE ). Returns a dict: {\"safe\": <bool>, \"detailed\": [...]} . 2.3 How a guardrail is used # The endpoint calls _is_request_guardrail_safe(payload) (or the analogous response guardrail). If FORCE_GUARDRAIL_REQUEST is true, a GuardrailPipeline is built from the configured plugin IDs (e.g. [\"nask_guard\", \"sojka_guard\"] ). The pipeline iterates over each guardrail plugin; each apply returns (is_safe, message) . The first plugin that reports is_safe=False short‑circuits the pipeline and the request is rejected with a 400/500 error payload. 2.5 ML-Based PII Classification # For cases where regex patterns alone are insufficient (e.g. context-dependent PII detection), the project integ…","r":"llm-router-plugins"},{"k":"llm-router-plugins/maskers/fast-masker/index.html","t":"Overview","s":"Masker plugins","x":"## Overview The **fast_masker** plugin provides a simple, rule‑based engine that scans a piece of text and replaces sensitive data ( e‑mail addresses, IPs, URLs, phone numbers, Polish PESEL identifiers, etc.) with clearly marked placeholde…","h":["Overview","Masking Rules","1. Highest Certainty — Checksum Validated Identifiers","2. High Certainty — Strict Format Validation","3. Medium-High Certainty — International Phone Numbers","4. Medium Certainty — Well-Structured Formats","5. Medium-Low Certainty — Business Identifiers","6. Lower Certainty — Pattern-Based with Context","7. Format-Based — Specific Patterns","8. Lowest Certainty — Generic Patterns","9. Beta Features","Disabled Rules (Too Noisy)","Utility Validators","Polish Identification Numbers","Financial Numbers","Vehicle and Transport","International Identification","Network and Security","Business Identifiers"],"b":"Overview # The fast_masker plugin provides a simple, rule‑based engine that scans a piece of text and replaces sensitive data ( e‑mail addresses, IPs, URLs, phone numbers, Polish PESEL identifiers, etc.) with clearly marked placeholders. The core component is the :class: ~llm_router_plugins.plugins.fast_masker.core.masker.FastMasker , which receives an ordered list of rule objects and applies each rule sequentially to the input text. Because the rules are applied in the order they are supplied, you can control precedence (e.g., replace URLs before e‑mails if needed). Masking Rules # Rules are applied in order from highest certainty (checksum-validated) to lowest certainty (pattern-based). This ordering minimizes false positives and ensures the most reliable identifiers are masked first. 1. Highest Certainty — Checksum Validated Identifiers # Rule Placeholder What it Detects Notes CreditCardRule {{CREDIT_CARD}} Credit card numbers (13-19 digits with optional spaces/dashes, e.g., 4532 1234 5678 9010 ). Validates using Luhn algorithm checksum. VinRule {{VIN}} Vehicle Identification Numbers (17 characters, e.g., 1HGBH41JXMN109186 ). Validates using ISO 3779 checksum (position 9). PeselTaggedRule {{PESEL_TAGGED}} Polish PESEL with label (e.g., PESEL: 44051401359 ). Validates checksum, preserves label prefix. PeselRule {{PESEL}} Polish PESEL numbers (11-digit personal identifiers). Validates checksum via is_valid_pesel . NipRule {{NIP}} Polish NIP numbers (plain, hyphen-separated, or wrapped in markdown). Validates checksum with weights [6,5,7,2,3,4,5,6,7] . KrsRule {{KRS}} Polish KRS numbers (10 digits, plain or hyphen-separated). Validates format only (10 digits). RegonRule {{REGON}} Polish REGON numbers (9 or 14 digits, optionally split by spaces). Validates checksum; handles both 9-digit and 14-digit forms. 2. High Certainty — Strict Format Validation # Rule Placeholder What it Detects Notes NrbRule {{NRB}} Polish NRB (bank account) numbers (26 digits, optional spaces). Supports formats: 26 digits or 2-4-4-4-4-4-4 grouping. MacAddressRule {{MAC_ADDRESS}} MAC addresses (6 octets, e.g., 00:1A:2B:3C:4D:5E ). Supports : , - , or no separators. PassportRule {{PASSPORT}} Passport numbers (2 letters + 7 digits, e.g., AB1234567 ). Case-insensitive matching. IdCardRule {{ID_CARD}} Polish ID card numbers (3 letters + 6 digits, e.g., ABC123456 ). Case-insensitive matching. SsnRule {{SSN}} US Social Security Numbers (format AAA-GG-SSSS , e.g., 123-45-6789 ). Format validation only, no checksum. 3. Medium-High Certainty — International Phone Numbers # Rule Placeholder What it Detects Notes PhoneInternationalRule {{PHONE_INTERNATIONAL}} Phone numbers with leading + and country code (e.g., +48 123 456 789 ). Matches 1-3 digit country codes with subscriber number groups. 4. Medium Certainty — Well-Structured Formats # Rule Placeholder What it Detects Notes EmailRule {{EMAIL}} E-mail addresses (e.g., user@example.com ). Permissive regex; matches local-part, @ , domain with TLD. Applied before URLs. UrlRule {{URL}} HTTP/HTTPS URLs and standalone domains (e.g., https://example.com , www.wp.pl ). Avoids code patterns like requests.post , response.json . IpRule {{IP}} , {{PORT}} IPv4, IPv6 addresses and hostname localhost . Masks ports as {{IP}}:{{PORT}} . Light octet validation; port captured separately. BankAccountRule {{BANK_ACCOUNT}} Polish IBAN (28 characters) and partially masked accounts (groups may contain X ). Exact length match; supports masked formats. 5. Medium-Low Certainty — Business Identifiers # Rule Placeholder What it Detects Notes JwtRule {{JWT}} JSON Web Tokens (3 Base64URL parts separated by dots). Validates structure; each part must be substantial (≥20 chars for header/payload). InvoiceNumberRule {{INVOICE_NUMBER}} Invoice identifiers (e.g., FV/2023/00123 , INV-2023-456 ). Case-insensitive; matches FV , INV , or INVOICE prefixes. OrderNumberRule {{ORDER_NUMBER}} E-commerce order identifiers (e.g., ORD123456 , ORDER-2023-001 ).…","r":"llm-router-plugins"},{"k":"llm-router-plugins/utils/routing/semantic/index.html","t":"Semantic Routing Plugin — simplesemanticrouting","s":"Routing plugins","x":"# Semantic Routing Plugin — `simple_semantic_routing` Heuristic-based model selection plugin for the **LLM‑Router** system. When `payload[\"model\"] == \"auto\"`, the plugin analyses the user's input and selects the most appropriate model from…","h":["1. Overview","1.1 What it does","1.2 Intent Categories","2. Architecture","2.1 Data Flow","2.2 Intent Classification Algorithm","2.3 Model Selection Algorithm","3. Configuration","3.1 JSON Config (simple.json)","3.2 Configurable Values","3.3 Environment Variable Overrides","4. Usage","4.1 Basic Usage","4.2 Payload Text Sources","4.3 Logging","5. Adding New Intents","5.1 Weight Tuning Guidelines","5.2 Phrase Format Guidelines","6. Complexity Levels","7. Built-in Patterns Reference","7.1 Code Patterns","7.2 Math Patterns","7.3 General Question Patterns","8. Running Tests","9. File Locations"],"b":"Semantic Routing Plugin — simple_semantic_routing # Heuristic-based model selection plugin for the LLM‑Router system. When payload[\"model\"] == \"auto\" , the plugin analyses the user's input and selects the most appropriate model from a configured pool. 1. Overview # 1.1 What it does # The plugin performs two-stage heuristic routing : Stage Description Intent Classification Scans the text for keywords, multi-word phrases, and regex patterns to determine the user's intent ( code , math , creative , general , or none ). Complexity Analysis Estimates token count (word count × 1.25) and maps it to a complexity level ( simple , medium , or complex ). Model Selection Combines the intent target with the complexity level to pick the best model from the pool. 1.2 Intent Categories # Intent Description Examples code Programming, debugging, implementation \"napisz funkcję\", \"fix bug\", \"debug code\", \"git commit\" math Mathematics, calculations, statistics \"oblicz prawdopodobieństwo\", \"solve equation\", \"calculate mean\" creative Creative writing, editing, brainstorming \"napisz opowiadanie\", \"write a story\", \"brainstorm\" general General questions, explanations, comparisons \"wyjaśnij jak\", \"what is the difference\", \"compare\" none No specific intent (greetings, etc.) \"cześć\", \"hello\", \"dziękuję\" 2. Architecture # 2.1 Data Flow # User Input │ ▼ classify_intent(text) ← keywords + phrases + regex patterns │ ▼ estimate_tokens(text) ← word_count × 1.25 │ ▼ complexity_level(tokens) ← simple / medium / complex │ ▼ select_model(intent, complexity) ← weighted score → model index │ ▼ Updated payload[\"model\"] 2.2 Intent Classification Algorithm # Each intent is defined in simple.json with four complementary signal types : a) Keywords — single words # Each keyword has an optional weight. If no weight is specified, the default is 1.0 . \"keywords\" : [ \"code\" , \"debug\" , \"funkcja\" ], \"weights\" : { \"debug\" : 5 , \"kod\" : 4 , \"code\" : 2 } Matching \"debug\" adds 5 to the score; matching \"code\" adds 2 . b) Phrases — multi-word expressions # Phrases use the format \"text:weight\" . The default weight is 2.0 when omitted. \"phrases\" : [ \"write code:5\" , \"fix bug:4\" , \"napisz funkcję:4\" ] Matching \"write code\" adds 5 to the score. c) Regex Patterns — structural detection # Patterns add a flat +3.0 per match, useful for detecting code structures, math formulas, and question patterns. \"patterns\" : [ \"def\\\\s+\\\\w+\\\\s*\\\\(\" , \"class\\\\s+\\\\w+\\\\s*[(:]\" , \"\\\\b\\\\d+\\\\s*[+\\\\-*/^]\\\\s*\\\\d+\\\\b\" , \"\\\\bwhat\\\\s+(is|are|does)\\\\b\" ] d) Weights — keyword importance # The weights object maps individual keywords to boost factors, allowing fine-grained control: \"weights\" : { \"debug\" : 5 , // Very strong signal for code intent \"błąd\" : 5 , // Very strong signal for code intent \"kod\" : 2 , // Moderate signal \"python\" : 2 // Moderate signal } 2.3 Model Selection Algorithm # 1. complexity_map = {\"simple\": 0, \"medium\": n//2, \"complex\": n-1} 2. idx = complexity_map[complexity] 3. if intent has a target in intent_adjustment: intent_idx = model_index_for(intent.target) idx = max(idx, intent_idx) // boost toward intent's target else: idx -= 1 // demote (no intent) 4. idx = clamp(idx, 0, n-1) 5. return models[idx] Boost / Demote Examples: Intent Complexity Base Index Intent Adjustment Result code → medium simple (idx=0) 0 intent targets index 1 max(0, 1) = 1 none complex (idx=2) 2 no target → demote 2-1 = 1 math → medium complex (idx=3) 3 intent targets index 1 max(3, 1) = 3 creative → simple medium (idx=1) 1 intent targets index 0 max(1, 0) = 1 3. Configuration # 3.1 JSON Config ( simple.json ) # All configuration lives in llm_router_plugins/resources/routing/semantic/simple.json . { \"settings\" : { \"len_thresholds_max\" : { \"simple\" : 25 , \"medium\" : 150 }, \"default_models\" : { \"simple\" : \"gpt-oss:120b\" , \"medium\" : \"qwen3.6:35b\" }, \"intent_adjustment\" : { \"code\" : \"medium\" , \"math\" : \"medium\" , \"creative\" : \"simple\" , \"general\" : \"simple\" , \"none\" : \"\" } }, \"intents\" : { \"code\" : { \"keywords\" : [ ... ], \"p…","r":"llm-router-plugins"},{"k":"llm-router-plugins/utils/rag/langchain-rag.html","t":"LangChain RAG Plugin for LLM‑Router","s":"Utility plugins","x":"# LangChain RAG Plugin for LLM‑Router A lightweight **Retrieval‑Augmented Generation (RAG)** plugin that lets you attach any local knowledge base to the LLM‑Router without touching the original application code. It builds the knowledge bas…","h":["Table of Contents","Features","Installation","CLI – Indexing & Searching","Searching","Example workflow","Error handling","License & Credits","Happy augmenting! 🚀"],"b":"LangChain RAG Plugin for LLM‑Router # A lightweight Retrieval‑Augmented Generation (RAG) plugin that lets you attach any local knowledge base to the LLM‑Router without touching the original application code. It builds the knowledge base with LangChain , stores vectors in FAISS (fallback to Milvus is possible), and injects the retrieved context into every LLM request. Table of Contents # Features Installation Configuration (environment variables) CLI – Indexing & Searching Integrating the plugin into LLM‑Router Example workflow Error handling License & Credits Features # Feature Description Indexing Recursively reads a directory of .md , .txt , .html (or any extensions you pass) → splits documents into overlapping token windows → embeds chunks with a transformer model → stores them in a persistent FAISS index. Searching Given a user query, retrieves the top‑N most similar chunks (default 10) using cosine similarity. Context injection Retrieved chunks are appended (with a clear prefix) to the user’s last message, so the downstream LLM can use the extra knowledge automatically. Full configurability All hyper‑parameters (collection name, embedder model, device, chunk size, overlap, persistence directory) are set via environment variables. Fail‑safe If the RAG subsystem is disabled or mis‑configured, the plugin raises a clear exception while the rest of the router keeps working. CLI helpers llm-router-rag-langchain command provides convenient index and search sub‑commands, plus an interactive REPL mode. Installation # Only the optional RAG dependencies are required; they are not installed with the core LLM‑Router. ```shell script Activate your virtualenv / conda env first # pip install faiss-cpu tqdm langchain langchain-community transformers torch > **Note** – If you prefer GPU‑accelerated FAISS, replace `faiss-cpu` with `faiss-gpu`. --- ## Configuration (environment variables) | Variable | Default | Description | |------------------------------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `LLM_ROUTER_LANGCHAIN_RAG_COLLECTION` | *must be set* | Name of the FAISS collection (e.g. `sample_collection`). | | `LLM_ROUTER_LANGCHAIN_RAG_EMBEDDER` | *must be set* | Hugging Face model identifier for the embedder (e.g. `sentence-transformers/all-MiniLM-L6-v2`). | | `LLM_ROUTER_LANGCHAIN_RAG_DEVICE` | `cpu` | Torch device (`cpu`, `cuda:0`, `cuda:1`, …). | | `LLM_ROUTER_LANGCHAIN_RAG_CHUNK_SIZE` | `1024` | Number of tokens per chunk. | | `LLM_ROUTER_LANGCHAIN_RAG_CHUNK_OVERLAP` | `100` | Overlap in tokens between consecutive chunks. | | `LLM_ROUTER_LANGCHAIN_RAG_PERSIST_DIR` | *none* | Directory where the FAISS index and docstore are persisted. If omitted, a folder named after the collection is created under `./workdir/plugins/utils/rag/langchain/`. | ### Example export block (add to your shell profile or a `.env` file) ```shell script export LLM_ROUTER_LANGCHAIN_RAG_COLLECTION=\"${LLM_ROUTER_LANGCHAIN_RAG_COLLECTION:-sample_collection}\" export LLM_ROUTER_LANGCHAIN_RAG_EMBEDDER=\"${LLM_ROUTER_LANGCHAIN_RAG_EMBEDDER:-sentence-transformers/all-MiniLM-L6-v2}\" export LLM_ROUTER_LANGCHAIN_RAG_DEVICE=\"${LLM_ROUTER_LANGCHAIN_RAG_DEVICE:-cpu}\" export LLM_ROUTER_LANGCHAIN_RAG_CHUNK_SIZE=\"${LLM_ROUTER_LANGCHAIN_RAG_CHUNK_SIZE:-200}\" export LLM_ROUTER_LANGCHAIN_RAG_CHUNK_OVERLAP=\"${LLM_ROUTER_LANGCHAIN_RAG_CHUNK_OVERLAP:-50}\" export LLM_ROUTER_LANGCHAIN_RAG_PERSIST_DIR=\"${LLM_ROUTER_LANGCHAIN_RAG_PERSIST_DIR:-./workdir/plugins/utils/rag/langchain/${LLM_ROUTER_LANGCHAIN_RAG_COLLECTION}}\" CLI – Indexing & Searching # The plugin ships with a convenient entry‑point: ```shell script llm-router-rag-langchain [options] ### Indexing ```shell script # Index Markdown, plain‑text and HTML files from a repository llm-router-rag-langchain index --path \"../llm-router\" --ext .md .txt .html You can run …","r":"llm-router-plugins"}]}