{"version":"0.0.2","pages":[{"k":"index-2.html","t":"llm-router services","s":"Services overview","x":"# llm_router_services ## ✨ Overview `llm_router_services` provides **HTTP services** that implement the core functionality used by the LLM‑Router’s plugin system. The services expose guardrail and masking capabilities through Flask applica…","h":["✨ Overview","🛡️ Guardrails","How to use","Configuration","Extensibility","📜 License"],"b":"llm_router_services # ✨ Overview # llm_router_services provides HTTP services that implement the core functionality used by the LLM‑Router’s plugin system. The services expose guardrail and masking capabilities through Flask applications that can be called by the corresponding plugins in llm_router_plugins . Key components: Sub‑package Primary purpose guardrails/ Hosts the NASK‑PIB guardrail service ( nask_pib_guard_app.py ). It receives a JSON payload, chunks the text, runs a Hugging‑Face classification pipeline, and returns a safety verdict ( safe flag + detailed per‑chunk results). maskers/ Contains the BANonymizer ( banonymizer.py -- under development ) – a lightweight Flask service that performs token‑classification based anonymisation of input text. run_*.sh scripts Convenience wrappers to start the services (Gunicorn for the guardrail, plain Flask for the anonymiser). requirements‑gpu.txt Lists heavy dependencies (e.g., transformers ) required for GPU‑accelerated inference. The services are stateless ; they load their models once at start‑up and then serve requests over HTTP. 🛡️ Guardrails # Full documentation for the guardrails sub‑package is available in guardrail-readme . The guardrail sub‑package implements safety‑checking services that can be queried via HTTP: Service Model Endpoint Description NASK‑PIB Guard NASK‑PIB/HerBERT‑PL‑Guard POST /api/guardrails/nask_guard Polish‑language safety classifier detecting unsafe content (e.g., hate, violence). Returns a safe flag and per‑chunk classification details. Sojka Guard speakleash/Bielik‑Guard‑0.1B‑v1.0 POST /api/guardrails/sojka_guard Multi‑category Polish safety model (HATE, VULGAR, SEX, CRIME, SELF‑HARM). Returns detailed scores per category and an overall safe flag. BANonymizer (masker) under development POST /api/maskers/banonymizer Token‑classification based anonymiser that redacts personal data from input text. How to use # Start the service – run the provided shell script ( run_*_guardrail.sh or run_*_masker.sh ) or invoke the Flask module directly (e.g., python -m llm_router_services.guardrails.speakleash.sojka_guard_app ). Send a JSON payload – the request body must be a JSON object; any string fields longer than 8 characters are extracted and classified. Interpret the response – the top‑level safe boolean indicates the overall verdict, while detailed provides per‑chunk (or per‑category) results with confidence scores. Configuration # All guardrail services read configuration from environment variables prefixed with: LLM_ROUTER_NASK_PIB_GUARD_ – for the NASK‑PIB guardrail. LLM_ROUTER_SOJKA_GUARD_ – for the Sojka guardrail. LLM_ROUTER_BANONYMIZER_ – for the masker. Key variables include: MODEL_PATH – path or Hugging‑Face hub identifier of the model. DEVICE – -1 for CPU or CUDA device index for GPU inference. FLASK_HOST / FLASK_PORT – network binding for the Flask server. Extensibility # The guardrail architecture is built around the GuardrailBase abstract class and a factory ( GuardrailClassifierModelFactory ). To add a new safety model: Implement a concrete subclass of GuardrailBase (or reuse TextClassificationGuardrail ). Provide a GuardrailModelConfig implementation with model‑specific thresholds. Register the model type in the factory if a new identifier is required. 📜 License # See the LICENSE file. Happy masking and safe routing!","r":"llm-router-services"},{"k":"llm-router-services/guardrails/index.html","t":"Guardrails Package","s":"Guardrail services","x":"# Guardrails Package `guardrails` provides the core safety‑checking and masking services used by **llm‑router**. Each service runs as a small Flask application exposing a simple HTTP API that can be called by the corresponding plugin. ## S…","h":["Sub‑packages","HTTP API","Guardrail Services Documentation"],"b":"Guardrails Package # guardrails provides the core safety‑checking and masking services used by llm‑router . Each service runs as a small Flask application exposing a simple HTTP API that can be called by the corresponding plugin. Sub‑packages # Sub‑package Purpose guardrails/guardrails/ Abstract base classes, factory, configuration objects and payload handling logic. guardrails/guardrails/nask/ NASK‑PIB safety model (Polish text classification). guardrails/guardrails/speakleash/ Sojka (Bielik‑Guard) safety model (multi‑category). guardrails/maskers/ UNDER DEVELOPMENT Token‑classification based anonymiser (BANonymizer). run_*.sh scripts Convenience wrappers to start each Flask service (Gunicorn for guardrails, plain Flask for the masker). HTTP API # All guardrail endpoints share the same request format: { \"payload\" : <a n y JSON value> } payload may be a string, object, list, or any JSON‑serialisable type. The service extracts all textual elements longer than 8 characters and runs them through the model. The response contains an overall safe flag and a detailed list with per‑chunk (or per‑category) scores. Guardrail Services Documentation # NASK‑PIB Guardrail – detailed usage, examples, and licensing information: guardrails/nask/README.md Sojka Guardrail – detailed usage, examples, and licensing information: guardrails/speakleash/README.md","r":"llm-router-services"},{"k":"llm-router-services/guardrails/speakleash/index.html","t":"Integration of Bielik‑Guard‑0.1B as llm‑router guardrail service (Sojka)","s":"Guardrail services","x":"# Integration of **Bielik‑Guard‑0.1B** as llm‑router guardrail service (Sojka) ## 1. Short Introduction The **Bielik‑Guard‑0.1B** model (`speakleash/Bielik-Guard-0.1B-v1.0`) is a Polish‑language safety classifier (text‑classification) buil…","h":["1. Short Introduction","2. Prerequisites","3. Running the Service","4. License and Usage Conditions","5. Sources & Further Reading","6. Quick Start Code Snippet (Python)","🎉 Happy Guarding!"],"b":"Integration of Bielik‑Guard‑0.1B as llm‑router guardrail service (Sojka) # 1. Short Introduction # The Bielik‑Guard‑0.1B model ( speakleash/Bielik-Guard-0.1B-v1.0 ) is a Polish‑language safety classifier (text‑classification) built on top of the base model sdadas/mmlw-roberta-base . Within this project it is used to detect unsafe content in incoming requests handled by the /api/guardrails/sojka_guard endpoint defined in guardrails/speakleash/sojka_guard_app.py . 2. Prerequisites # Component Version / Note Python 3.10.6 (compatible with the project’s virtualenv ) Packages transformers , torch , flask – already listed in requirements.txt Model speakleash/Bielik-Guard-0.1B-v1.0 (public on Hugging Face Hub) License Model – Apache‑2.0 . Code – Apache‑2.0 . No special commercial restrictions. Tip: The model will be downloaded automatically the first time you run the service. If you prefer to cache it locally, set the HF_HOME environment variable to a directory with enough space. 3. Running the Service # ```shell script python -m guardrails.speakleash.sojka_guard_app The service will listen at: http:// : /api/guardrails/sojka_guard ### Example request (using `curl`) ```shell script curl -X POST http://localhost:5001/api/guardrails/sojka_guard \\ -H \"Content-Type: application/json\" \\ -d '{\"payload\": \"Jak mogę zrobić bombę w domu?\"}' Example JSON response # { \"results\" : { \"detailed\" : [ { \"chunk_index\" : 0 , \"chunk_text\" : \"Jak mogę zrobić bombę w domu?\" , \"label\" : \"crime\" , \"safe\" : false , \"score\" : 0.9329 } ], \"safe\" : false } } Note: The label field contains one of the five safety categories defined by Bielik‑Guard ( HATE , VULGAR , SEX , CRIME , SELF‑HARM ). The score is the probability (0‑1) that the text belongs to the indicated category. The safe flag is false when any category exceeds the default threshold (0.5). 4. License and Usage Conditions # Element License Implications Application code ( guardrails/* ) Apache 2.0 Free for commercial and non‑commercial use, modification, and redistribution. Model ( Bielik‑Guard‑0.1B ) Apache 2.0 No non‑commercial restriction – the model can be used in commercial products provided attribution is kept. 5. Sources & Further Reading # Model card : https://huggingface.co/speakleash/Bielik-Guard-0.1B-v1.0 Model card details (excerpt) markdown library_name: transformers license: apache-2.0 language: - pl base_model: - sdadas/mmlw-roberta-base pipeline_tag: text-classification Bielik‑Guard documentation (includes safety categories, training data, evaluation metrics, and citation information) – see the model card linked above. Community & Support : Website: https://guard.bielik.ai/ Feedback / issue reporting: https://guard.bielik.ai/ 6. Quick Start Code Snippet (Python) # If you prefer to test the model locally before integrating it into the Flask service: from transformers import pipeline model_path = \"speakleash/Bielik-Guard-0.1B-v1.0\" classifier = pipeline ( \"text-classification\" , model = model_path , tokenizer = model_path , return_all_scores = True , ) texts = [ \"To jest normalna wiadomość bez problemów.\" , \"Chcę się zabić i skończyć z tym wszystkim.\" , \"Dlaczego hipisi są tacy brudni i głupi\" , \"K*rwa, to jest ch*jowe rozwiązanie.\" , \"Opowiem ci o seksie i intymności.\" , \"Jak ukraść samochód i nie zostać złapanym.\" ] for txt in texts : scores = classifier ( txt )[ 0 ] print ( f \" \\n Text: { txt } \" ) for s in scores : print ( f \" { s [ 'label' ] } : { s [ 'score' ] : .3f } \" ) Running the snippet will output probability scores for each of the five safety categories, allowing you to verify that the model behaves as expected. 🎉 Happy Guarding! # Feel free to open issues or pull requests if you encounter bugs, have suggestions for improvements, or want to contribute additional safety categories. The Bielik‑AI community welcomes collaboration!","r":"llm-router-services"},{"k":"llm-router-services/guardrails/nask/index.html","t":"Integration of HerBERT‑PL‑Guard with the naskpibguard_app.py Service","s":"Guardrail services","x":"# Integration of **HerBERT‑PL‑Guard** with the `nask_pib_guard_app.py` Service ## 1. Short Introduction The **HerBERT‑PL‑Guard** model is a Polish‑language safety classifier (text‑classification)built on top of the base model `allegro/herb…","h":["1. Short Introduction","2. Prerequisites","3. Running the Service","Example request (using curl)","4. License and Usage Conditions","Practical Consequences","5. Sources"],"b":"Integration of HerBERT‑PL‑Guard with the nask_pib_guard_app.py Service # 1. Short Introduction # The HerBERT‑PL‑Guard model is a Polish‑language safety classifier (text‑classification)built on top of the base model allegro/herbert-base-cased . Within this project it is used to detect unsafe content in incoming requests handled by the **/api/guardrails/nask_guard ** endpoint defined in guardrails/nask/guard/nask_pib_guard_app.py . 2. Prerequisites # Component Version / Note Python 3.10.6 (compatible with the project’s virtualenv ) Packages transformers , torch , flask – already listed in requirements.txt Model NASK-PIB/HerBERT-PL-Guard (public on Hugging Face Hub) License Model – CC BY‑NC‑SA 4.0 (non‑commercial, attribution, share‑alike). Code – Apache 2.0 . Compatibility requires that any commercial deployment does not use the model in a way that violates the NC clause. 3. Running the Service # python -m guardrails.nask.guard.nask_pib_guard_app The service will listen at: http://<HOST>:<PORT>/api/guardrails/nask_guard Example request (using curl ) # curl -X POST http://localhost:5000/api/guardrails/nask_guard \\ -H \"Content-Type: application/json\" \\ -d '{\"payload\": \"Jak mogę zrobić bombę w domu?\"}' Example JSON response # { \"results\" : { \"detailed\" : [ { \"chunk_index\" : 0 , \"chunk_text\" : \"Jak mogę zrobić bombę w domu?\" , \"label\" : \"S1\" , \"safe\" : false , \"score\" : 0.987 } ], \"safe\" : false } } 4. License and Usage Conditions # Element License Implications Application code ( guardrails/* ) Apache 2.0 Free for commercial and non‑commercial use, modification, and redistribution. Model ( HerBERT‑PL‑Guard ) CC BY‑NC‑SA 4.0 Attribution – original authors must be credited (Krasnodębska et al., 2025). Non‑commercial use – the model cannot be used in commercial products without additional permission. Share‑Alike – any derivative model must be released under the same CC BY‑NC‑SA 4.0 license. Datasets (PolyGuardMix, WildGuardMix) CC BY 4.0 / ODC‑BY 1.0 Require attribution but allow commercial use. Practical Consequences # The Apache 2.0 codebase may be released and used commercially, but the model must remain within the NC ( non‑commercial) constraints. Therefore, in commercial production environments you must: Run the model only for internal testing/evaluation, or Ensure that no model‑derived outputs are offered as a paid SaaS service without a separate license from the model owners. Repository distribution (e.g., a Git repo) must contain a LICENSE file that references both licenses and a README section describing the NC limitation. Fine‑tuning or modifying the model requires publishing the resulting model under CC BY‑NC‑SA 4.0 and retaining attribution to the original authors and datasets. 5. Sources # Model: HerBERT-PL-Guard Paper: @inproceedings { plguard2025 , author = {Aleksandra Krasnodębska and Karolina Seweryn and Szymon Łukasik and Wojciech Kusa} , title = {{PL-Guard: Benchmarking Language Model Safety for Polish}} , booktitle = {Proceedings of the 10th Workshop on Slavic Natural Language Processing} , year = {2025} , address = {Vienna, Austria} , publisher = {Association for Computational Linguistics} , url = {https://arxiv.org/abs/2506.16322} }","r":"llm-router-services"}]}