)
How do you add guardrails to LiteLLM?
LiteLLM proxy supports guardrails via a guardrails section in your config.yaml. Native options include Presidio for PII masking, Lakera for prompt injection detection, and Aporia for policy enforcement.
Each guardrail runs in pre_call, during_call, or post_call mode and can be toggled per-request, per-API-key, or per-team. For enterprise deployments, native guardrails cover the basics but leave gaps in intent-based policy enforcement, jurisdiction-aware routing, inference-level audit trails, and agent security.
NeuralTrust TrustGate and TrustGuard address those gaps as a security layer on top of LiteLLM.
TL;DR - Key Takeaways
- LiteLLM is a recognized AI gateway for routing, cost management, and load balancing across 100+ LLM providers. It was not designed as a security product.
- LiteLLM's native guardrails support: prompt injection detection (via Lakera), PII masking (via Presidio), and secret detection, all requiring external services.
- Native guardrails run in three modes:
pre_call(before the LLM call),during_call(in parallel), andpost_call(after the response). - The gaps: no intent-based policy enforcement, no jurisdiction-aware routing, no inference-level audit trails, no agent security, no adversarial pre-deployment testing.
- NeuralTrust's TrustGate sits in front of LiteLLM as a security enforcement layer, adding the enterprise controls that native guardrails do not cover.
- The architecture is additive: LiteLLM continues to handle routing and cost tracking; NeuralTrust handles security enforcement, compliance logging, and anomaly detection.
LiteLLM routes your AI requests but is weak at security, and it was not trying to be good at it. You configure native guardrails with a few lines in config.yaml: Presidio for PII, Lakera for prompt injection, Aporia for policy. That handles the basics.
For anything enterprise (compliance audit trails, jurisdiction routing, multi-agent monitoring, red teaming) you need a security layer on top. NeuralTrust TrustGate and TrustGuard are built for exactly that.
)
LiteLLM Is Great at Routing. That Is Not the Same as Security.
Picture this. You're three months into a production AI deployment. LiteLLM is working perfectly. It's routing customer queries to GPT-4o, falling back to Claude when OpenAI is slow, tracking spend per team, enforcing rate limits. Your engineers love it.
Then your security lead asks a question: "What's inspecting what goes into those models before it gets there?"
Silence.
LiteLLM was built to solve the multi-provider routing problem. It solved it brilliantly. The GitHub repository has tens of thousands of stars for good reason. But routing AI traffic and securing AI traffic are two different jobs.
This article shows you both: how to configure LiteLLM's native guardrails correctly, and where they stop being enough.
What LiteLLM's Native Guardrails Actually Give You
LiteLLM proxy has a guardrails section in its config.yaml. It supports four core capabilities out of the box:
- Prompt injection detection via Lakera Guard or LLMGuard: scans user inputs for attempts to hijack the model's instructions.
- PII masking via Microsoft Presidio: detects and masks personal data (credit cards, email addresses, SSNs, phone numbers) before it reaches the LLM.
- Secret detection via LiteLLM's built-in
hide_secretscallback: catches API keys and credentials in inputs. - Policy enforcement via Aporia: lets you define conversation-level policies that flag or block non-compliant inputs and outputs.
Each guardrail can run in three modes:
pre_call: runs before the LLM call, on input onlyduring_call: runs in parallel with the LLM call, on inputpost_call: runs after the LLM call, on input and output
Here is what a working config.yaml looks like with native guardrails configured:
Start the proxy:
Test it -- send a request that includes a phone number:
With pii-masking on, +1 415-555-0123 gets replaced before it ever reaches OpenAI.
You can also toggle guardrails per-request:
And enforce them at the key level: useful for giving internal tools more access than customer-facing endpoints:
This is solid for prototyping. For most internal tools and early production deployments, this gets you reasonably far.
The problem shows up when "reasonably far" is not far enough.
)
What Native LiteLLM Guardrails Cannot Do
Here is the gap list. These are not feature requests, they are the things that come up in real enterprise deployments.
1. No intent-based policy enforcement
Presidio detects PII patterns. Lakera detects known prompt injection patterns. Neither understands what a conversation is trying to accomplish at the semantic level. If a user is slowly probing for system prompt contents across ten innocent-looking messages, native guardrails will not catch it.
2. No jurisdiction-aware routing
LiteLLM routes by model performance, cost, and load. It does not know that EU personal data should not be routed to a US provider without explicit legal cover. It will happily send a French customer's medical data to an endpoint that exposes you to CLOUD Act jurisdiction.
3. No inference-level audit trail
LiteLLM logs via callbacks -- you can send data to Langfuse, Helicone, S3. That is application-level logging. An enterprise audit trail records the exact input, output, model version, and timestamp at the inference layer in an immutable format a regulator can verify. These are different things.
4. No multi-agent or MCP security
As your LiteLLM deployment evolves from single prompts to agentic workflows -- chains of tool calls, memory access, orchestrator-to-subagent communication -- native guardrails only see the outermost API call. What happens inside the chain is invisible to them.
6. No adversarial pre-deployment testing
You can configure guardrails for patterns you know about. You cannot know what you do not know. OWASP LLM Top 10 lists ten attack categories. LiteLLM native guardrails partially cover LLM01 (prompt injection) and LLM02 (insecure output handling). The other eight you are on your own for.
7. No compliance documentation
DORA, HIPAA, EU AI Act -- all require documented evidence that your AI system has controls in place. LiteLLM generates logs. It does not generate the audit artifacts a compliance team can present to a regulator.
Adding NeuralTrust as the Enterprise Security Layer
NeuralTrust TrustGate and TrustGuard are designed to sit in front of LiteLLM, adding the security controls that LiteLLM deliberately does not try to provide.
The architecture is simple: your applications point to TrustGate. TrustGate enforces policies, masks PII, detects prompt injection with intent-level analysis, and routes to your LiteLLM proxy. LiteLLM continues to do what it does best -- manage model routing, handle fallbacks, track spend.
If you prefer to keep LiteLLM as your application-facing API, NeuralTrust can also integrate as a guardrail provider via LiteLLM's generic_guardrail_api interface:
In this mode, LiteLLM calls TrustGate as a guardrail on every request. TrustGuard's runtime analysis runs on input before the LLM call and on output before it reaches the user.
Your application code does not change at all:
)
LiteLLM Native vs LiteLLM + NeuralTrust
| Capability | LiteLLM Native Guardrails | LiteLLM + NeuralTrust |
|---|---|---|
| Prompt injection detection | Via Lakera (external API dependency) | Intent-based analysis, no external dependency |
| PII masking | Via Presidio (requires separate Presidio server) | Built-in detection, no extra infrastructure |
| Secret detection | Built-in, rule-based | Built-in plus semantic context analysis |
| Output content filtering | Via Aporia or Google Text Moderation | Built-in anomaly detection and toxicity scoring |
| Jurisdiction-aware routing | Not supported | Routes by data classification and legal jurisdiction |
| Inference-level audit trail | Callback-based application logs | Immutable inference records (input, output, model version, timestamp) |
| Multi-agent and MCP security | Outermost API call only | Full chain visibility via TrustLens |
| Pre-deployment adversarial testing | Not supported | TrustTest covers OWASP LLM Top 10 |
| Compliance documentation | Not included | DORA, EU AI Act, HIPAA audit artifacts |
| Guardrail per team, per key | Yes | Yes, with additional data classification controls |
What This Means in Practice
You do not have to choose between LiteLLM and enterprise security. The two are complementary.
LiteLLM handles the routing problem. It has solved it well. Switching away from LiteLLM to get better security would mean giving up years of routing, cost tracking, and provider management functionality.
NeuralTrust handles the security problem. TrustGuard adds runtime enforcement. TrustGate adds the API gateway layer with jurisdiction awareness. TrustLens maps every AI touchpoint in your environment. TrustTest runs the adversarial tests before you go live.
The combination covers both halves of what enterprise AI infrastructure requires: routing that works and security you can prove.
FAQs about adding NeuralTrust's Guardrails on top of LiteLLM
1. How do I add guardrails to LiteLLM?
Add a guardrails section to your LiteLLM config.yaml. Each guardrail entry needs a guardrail_name, a litellm_params block with the guardrail provider (e.g., lakera, presidio, aporia, generic_guardrail_api), and a mode setting (pre_call, during_call, or post_call). Set default_on: true to apply the guardrail to all requests, or manage per-key and per-request using the /key/generate endpoint and the metadata.guardrails field. Full documentation is at docs.litellm.ai/docs/proxy/guardrails/quick_start.
2. What prompt injection detection does LiteLLM support natively?
LiteLLM supports prompt injection detection via Lakera Guard, LLMGuard, LlamaGuard, and Google Text Moderation. Each is configured as an external API call in pre_call or during_call mode. Lakera is the most commonly used integration. Native detection is pattern-based and relies on the external service's models. For intent-level analysis and detection of multi-turn prompt injection attempts, an additional layer like NeuralTrust TrustGuard is needed.
3. Does LiteLLM support PII masking out of the box?
Yes, via Microsoft Presidio. You configure a presidio guardrail in config.yaml and specify which entity types to mask (CREDIT_CARD, EMAIL_ADDRESS, US_SSN, etc.) and at what confidence threshold. Presidio requires running its own server separately from LiteLLM. LiteLLM's Presidio integration also supports a logging_only mode, where PII is masked in logs but not in the actual LLM request.
4. Can I use NeuralTrust with LiteLLM?
Yes. NeuralTrust TrustGate integrates with LiteLLM in two ways: (1) as a guardrail provider via LiteLLM's generic_guardrail_api interface, which lets LiteLLM call TrustGate on every request without changing your application code; (2) as a proxy layer in front of LiteLLM, where your application points to TrustGate and TrustGate routes to your LiteLLM instance. Both approaches are additive -- LiteLLM continues to handle provider routing, cost tracking, and fallbacks.
5. What security gaps does LiteLLM leave for enterprise deployments?
LiteLLM's native guardrails do not support intent-based policy enforcement (only pattern matching), jurisdiction-aware routing, inference-level audit trails in compliance-ready formats, multi-agent workflow security, or pre-deployment adversarial testing. For organisations subject to DORA, HIPAA, the EU AI Act, or FedRAMP, these gaps need to be closed by an additional security layer. LiteLLM's documentation acknowledges that guardrails are in beta and designed to integrate with specialised security providers rather than replace them.
About the Author
Roger Howroyd is Head of Global SEO and AI at NeuralTrust, where he leads the company's search strategy across SEO, AEO, GEO, and LLM optimization. He specializes in AI-powered search, content strategy, backlink development, and SEM. Connect on LinkedIn.
NeuralTrust is an AI agent security platform, recognized in the Gartner Hype Cycle for Application Security 2026, the Gartner Hype Cycle for Infrastructure Security 2026, the Gartner 2025 Market Guide for AI Gateways and Guardian Agents, and the KuppingerCole 2025 Leadership Compass for Generative AI Defense. ISO 27001 certified. Headquartered in Barcelona.