NeuralTrust AI Security Model Performance Report 2026

What is NeuralTrust's AI security model performance?

NeuralTrust's guardrail models achieve ROC-AUC scores of 0.991 to 0.9997 across four tasks: jailbreak detection (92% detection rate, 2.3% false-positive rate), toxicity detection (92.1% detection rate, 2.4% false-positive rate), indirect prompt injection detection (99.8% detection rate, 0.6% false-positive rate), and prompt moderation across 13 topics (average ROC-AUC 0.992).

All models are benchmarked across 9 languages and 5 industry verticals. On jailbreak detection, NeuralTrust leads AWS Bedrock Guardrails by 18 percentage points and Azure AI Content Safety by 59 percentage points at matched false-positive rates.

---

TL;DR - Key Takeaways

• NeuralTrust's jailbreak detection model achieves 92% detection at a 2.3% false-positive rate, benchmarked on 21,600 prompts across 9 languages, 8 attack families, and 5 industry verticals.
• The indirect prompt injection model reaches 99.8% detection with a 0.6% false-positive rate and zero false positives on realistic benign agent traffic, making it purpose-built for agentic and RAG deployments.
• At matched false-positive rates, NeuralTrust leads AWS Bedrock Guardrails by 18 percentage points on jailbreak detection and by 4-6 points on toxicity detection.
• At matched false-positive rates, NeuralTrust leads Azure AI Content Safety (Prompt Shields) by 59 percentage points on jailbreak detection and by 5-10 points on toxicity detection.
• Each model is stress-tested under 10 adversarial text perturbations, totalling approximately 129,600 distinct scored inputs per model.
• A closed-loop evaluation platform continuously mines production false positives and negatives and feeds them into versioned, benchmarked retrainingm, so every release must beat its predecessor before shipping.

Every enterprise security team evaluating AI guardrails needs the same three numbers: detection rate, false-positive rate, and how those numbers hold under adversarial pressure.

This report provides all three for NeuralTrust's four production guardrail models, measured on held-out benchmarks totalling over 22,000 rows per model, stress-tested under 10 perturbation types, and compared against AWS Bedrock Guardrails and Azure AI Content Safety on the same labelled suites.

The headline: NeuralTrust leads both providers on both comparable tasks at matched operating points. The detail is below.

---

The Question Every CISO Asks

Most AI security evaluations start with a single question: does this actually work?

It is a reasonable question. The vendor demo shows the model blocking a jailbreak attempt. What it does not show is performance on the tenth variant of that attack, phrased in Turkish, wrapped in base64, in a financial services context. Or the false-positive rate on legitimate traffic from a banking customer who just phrased their support request in an unusual way.

This report answers that question directly. It presents the performance evidence for NeuralTrust's four production guardrail models: jailbreak detection, toxicity detection, indirect prompt injection detection, and prompt moderation. Every number is backed by labelled benchmark data. The limitations are stated alongside the strengths.

---

The Four Models

NeuralTrust builds lightweight, multilingual guardrail models that screen traffic to and from large language models. The four models covered in this report are:

1. Jailbreak detection

(NeuralTrust/prompt-jailbreak) screens incoming prompts for attempts to subvert an LLM's guardrails: system-prompt overrides, role-play exploits, and encoded instructions. It runs as a pre-processing layer before the generation model receives the request.

2. Toxicity detection

(NeuralTrust/toxicity-detector) flags toxic text across six categories: insults, profanity, identity-based hate, harassment, threats, and sexual harassment. It screens both user inputs and model outputs across nine languages.

3. Indirect prompt injection detection

(NeuralTrust/indirect-prompt-injections) flags malicious instructions hidden in external content that an LLM ingests: retrieved web pages, documents, emails, API responses, and tool returns.

This is the primary attack surface for agentic and RAG systems. The model complements the jailbreak detector rather than replacing it: jailbreak screens the user's own prompt, injection screening covers everything the agent reads.

4. Prompt moderation

(NeuralTrust/prompt-moderator) is a suite of 13 independent binary classifiers sharing a single multilingual encoder. It routes sensitive content across topics including violence, hate speech, medical and financial advice, personal information, and misinformation to stricter policies or human review.

---

Evaluation Methodology

The jailbreak and toxicity models are evaluated on an internal industry-by-language benchmark suite. The benchmark holds a roughly 50/50 split of attack and benign prompts spread across five industry verticals (airlines, banking, energy, telco, generic) and nine languages (English, Spanish, French, Portuguese, German, Turkish, Italian, Catalan, Galician).

Per-category precision is computed against a shared benign pool rather than being isolated to the attack slice, so scores reflect genuine separability from real benign traffic rather than inflated per-category figures.

Three production operating points are reported:

Beyond the labelled benchmark, each model is stress-tested under 10 single-factor text perturbations: typos, character substitution, leetspeak, homoglyphs (Cyrillic look-alikes), Unicode obfuscation, inline encoding tricks (base64/hex/ROT13), whitespace tricks, accent stripping, code-mixing, and industry-phrasing wrappers. This expands the true evaluation volume to approximately 129,600 distinct scored inputs per model.

The indirect injection and prompt moderation models are evaluated on held-out test splits of their own training corpora rather than the industry benchmark suite, which the full report notes explicitly.

---

Jailbreak Detection: Full Results

Benchmark: 21,600 rows (10,800 jailbreak / 10,800 benign) across 9 languages, 8 attack families, 5 industries. Run date: July 2026.

Overall Performance

Threshold-free ROC-AUC: 0.991

At the primary operating point, the model detects 92% of jailbreak attempts while wrongly flagging 2.3% of legitimate prompts.

Performance by Industry

Performance holds across all five verticals. Energy is the most adversarially difficult slice and remains strong. Banking detection (0.943) is the highest across regulated industries.

Performance by Attack Family

The most common attack families (instruction overrides and system-prompt extraction) are detected at 99%. Structured-payload (fragmentary, template-style payloads) is the hardest slice and the one gap the report states plainly.

Performance by Language

Detection is strong and consistent across all nine languages. Portuguese (0.95) and Italian (0.95) are the top performers outside English.

Robustness Under Adversarial Perturbation

Under the 10-perturbation stress sweep, the jailbreak model shows no fragility: no perturbation produces a meaningful detection drop at the primary operating point. For most transforms, the detection drop is negative, meaning the model becomes more likely to flag the obfuscated form, because the evasion itself is a strong adversarial signal. Unicode obfuscation produces a detection drop of -0.078 and homoglyph substitution -0.068, both improving detection. The largest positive drop is accent stripping at +0.004, which is negligible.

---

Toxicity Detection: Full Results

Benchmark: 21,553 rows (10,753 toxic / 10,800 benign) across 9 languages, 6 toxicity types, 5 industries. Run date: July 2026.

Overall Performance

Threshold-free ROC-AUC: 0.992

At the balanced operating point, the model detects 92.1% of toxic content while wrongly flagging 2.4% of safe text.

Performance by Toxicity Type

Overt categories (insult, profanity) are strongest. Implicit or context-dependent categories (threats, sexual harassment) are harder and benefit from the lenient operating point.

Performance by Language

Robustness

Inline-encoded payloads (base64/hex/ROT13 wrapping) produce the only material robustness gap: a detection drop of +0.104. Two mitigating factors: the mean recovery rate at L3 is +0.039 (signal attenuated, not lost), and encoding tricks inflate token count by 3.2x, making them conspicuous to an upstream length/entropy pre-filter. All other perturbations produce detection drops of +0.037 or lower, with several negative (leetspeak -0.038, character substitution -0.026).

---

Indirect Prompt Injection Detection: Full Results

Evaluation basis: Held-out test split of the training corpus (in-distribution), not the industry benchmark suite. 22,298 rows (11,149 injection / 11,149 benign) across 12 upstream sources. English only. Run date: July 2026.

Overall Performance

Threshold-free ROC-AUC: 0.9997

The model's scores are sharply bimodal (concentrated near 0 or 1), so L3 and L5 produce identical decisions. Only the strict L1 point differs, and only marginally.

Detection by Source

Across approximately 3,200 rows of benign agent and tool-trace traffic from a range of contemporary agentic models, the model produced zero false positives. The overall false-positive rate is 0.6%. Browser-agent page injections are the weakest slice (0.811) and the gap the report notes explicitly.

This model is the critical guardrail for any agentic or RAG deployment where the attack surface is not the user's prompt but the content the agent retrieves and acts on.

---

Prompt Moderation: Full Results

Evaluation basis: Per-topic held-out validation sets from production release records. 13 topic classifiers; figures are macro-averages. Trained and evaluated across 9 languages. Not directly comparable to the jailbreak and toxicity benchmark-suite figures.

Headline Performance

Per-Topic Breakdown

Every topic classifier achieves ROC-AUC above 0.985. The shared-encoder architecture keeps the model compact and fast while each topic head is versioned and retrained independently as data matures.

---

Competitive Benchmark: NeuralTrust vs. AWS Bedrock vs. Azure AI Content Safety

Both providers were scored on the same full labelled suites as the NeuralTrust headline numbers: 21,600 jailbreak rows and 21,553 toxicity rows across 9 languages. The fair comparison basis is matched false-positive rate: at equal over-blocking, whose detection is higher?

Important caveats stated up front: Bedrock returns categorical confidence buckets (NONE/LOW/MEDIUM/HIGH) rather than a continuous probability; Azure Prompt Shields returns a single boolean on jailbreak.

The benchmark suite was constructed to NeuralTrust's own taxonomies and industry cover-stories, a home-field advantage acknowledged explicitly in the full report. Provider results are therefore indicative rather than a definitive vendor ranking.

Jailbreak Detection

At its primary operating point, NeuralTrust detects 92.0% of jailbreaks versus Bedrock's 74.1% at the most permissive cut: a +18 percentage-point advantage at comparable over-blocking. Versus Azure Prompt Shields (33.0%): a +59 percentage-point advantage.

The gap is concentrated in harder attack families. At matched operating points:

Azure Prompt Shields collapses on indirect and obfuscated attacks: indirect detection is effectively zero (0.00) and structured-payload is 0.09. Bedrock handles overt attacks comparably but falls off significantly on the harder families.

Jailbreak by language at matched operating points:

NeuralTrust leads across all nine languages. The multilingual advantage is most pronounced on Catalan, where NeuralTrust (0.92) leads Bedrock (0.65) by 27 points.

Toxicity Detection

Toxicity is a closer contest. Bedrock at its most permissive cut (88.1%) is essentially level with NeuralTrust's strict L1 (88.6%). NeuralTrust's more sensitive L3 and L5 operating points extend a +4 to +6 percentage-point lead over Bedrock and a +5 to +10 percentage-point lead over Azure at comparable over-blocking. NeuralTrust's edge is largest on implicit categories:

Sexual harassment detection: NeuralTrust 0.87 vs Bedrock 0.72 vs Azure 0.72. Harassment detection: NeuralTrust 0.92 vs Azure 0.73. The specialised model matters most in the hard, implicit categories.

---

How NeuralTrust Keeps Improving

These numbers are not a static snapshot. Every model runs inside a closed-loop evaluation platform that continuously mines real deployment traffic for the model's own mistakes.

The loop operates in four stages:

1. First, production predictions are re-scored with LLM-assisted judgment over a configurable window, yielding per-run counts of correct, false-positive, and false-negative decisions plus a rolling 90-day accuracy trend.
2. Second, false positives and negatives are clustered into human-readable categories and de-duplicated across runs.
3. Third, selected categories become a new versioned dataset revision — not by logging user content, but by regenerating examples that cover the identified failure patterns.
4. Fourth, a new dataset revision triggers an automated retrain, benchmark run, and interactive test. The retrained model must beat its predecessor on the benchmark before it ships. If it regresses, it rolls back.

This process is how the prompt moderation model's translation head reduced its false-positive rate on short transactional messages from roughly 5% to 0% over successive revisions, while holding recall constant. Every release is benchmarked, versioned, and rollback-safe.

---

Summary: All Four Models

Jailbreak and toxicity figures are at the primary operating point (L3, threshold 0.85), measured on the industry-by-language benchmark suite. Indirect injection figures are at the default threshold (0.5) on the held-out test split (in-distribution, English-only).

Prompt moderation figures are macro-averages over 13 per-topic held-out validation sets. Different evaluation bases: see the full methodology sections above.

---

Frequently Asked Questions

1. What is NeuralTrust's jailbreak detection accuracy?

NeuralTrust's jailbreak detection model (prompt-jailbreak) achieves a 92.0% detection rate with a 2.3% false-positive rate at its primary balanced operating point, on a benchmark of 21,600 prompts across 9 languages, 8 attack families, and 5 industries. The threshold-free ROC-AUC is 0.991. At a stricter operating point (threshold 0.95), detection is 91.2% with a 2.1% false-positive rate. At the most permissive point (threshold 0.65), detection is 92.5% with a 2.7% false-positive rate.

2. How does NeuralTrust compare to AWS Bedrock Guardrails for jailbreak detection?

On the same 21,600-row benchmark suite at matched false-positive rates, NeuralTrust detects 92.0% of jailbreak attempts versus AWS Bedrock Guardrails' 74.1% at its most permissive operating point — an 18 percentage-point advantage. The gap is largest on obfuscated and fragmented attack families: payload-splitting (NeuralTrust 0.92 vs Bedrock 0.62), indirect attacks (NeuralTrust 0.90 vs Bedrock 0.51), and structured-payload (NeuralTrust 0.75 vs Bedrock 0.40).

3. How does NeuralTrust compare to Azure AI Content Safety for jailbreak detection?

On the same benchmark, NeuralTrust detects 92.0% of jailbreak attempts versus Azure Prompt Shields' 33.0%, a 59 percentage-point advantage. Azure Prompt Shields returns a single boolean and collapses on indirect attacks (detection effectively 0.00) and structured-payload attacks (0.09). NeuralTrust leads across all 9 languages and all 8 attack families tested.

4. What is NeuralTrust's indirect prompt injection detection accuracy?

NeuralTrust's indirect prompt injection model (indirect-prompt-injections) achieves 99.8% detection with a 0.6% false-positive rate at its default operating threshold, on a held-out test set of 22,298 rows across 12 upstream sources. The ROC-AUC is 0.9997. It produced zero false positives across approximately 3,200 rows of realistic benign agent and tool-trace traffic. The weakest slice is browser-agent page injections at 81.1% detection. The model is English-only in its current production release.

5. What is the false-positive rate for NeuralTrust's AI guardrails?

At the primary balanced operating point, the false-positive rate is 2.3% for jailbreak detection, 2.4% for toxicity detection, and 0.6% for indirect prompt injection detection. False-positive rate is the fraction of legitimate traffic incorrectly flagged, the key over-blocking metric for any guardrail deployed in production. At the stricter operating point (threshold 0.95), jailbreak false-positive rate drops to 2.1% and toxicity to 1.5%.

6. Does NeuralTrust's jailbreak detection work in multiple languages?

Yes. The jailbreak and toxicity models are each evaluated across nine languages: English, Spanish, French, Portuguese, German, Turkish, Italian, Catalan, and Galician. Jailbreak detection at L3 ranges from 0.92 (English, Spanish) to 0.85 (Galician) across all nine. Toxicity detection at L3 ranges from 0.97 (Spanish, French) to 0.87 (German). Both models are trained and benchmarked multilingually, not English-first with coverage bolted on.

7. How does NeuralTrust keep its models up to date?

NeuralTrust runs a closed-loop continuous-evaluation platform that mines production false positives and false negatives from real deployment traffic, clusters them into failure categories, regenerates training examples to cover those patterns, and retrains the model. Every new model release must beat its predecessor on the benchmark suite before it ships; any regression triggers an automatic rollback. No user content is added directly to the training set.

---

About This Research

This report presents performance evidence for NeuralTrust's four production guardrail models as of July 2026. All benchmark runs are pinned to exact model versions and UTC timestamps with per-shard hashing for reproducibility. Contamination tracking (automated de-duplication and train/test overlap analysis) guards against benchmark leakage.

The full model report (external release) is available for download. Comparative provider benchmarking was conducted on July 23, 2026 against AWS Bedrock Guardrails (eu-north-1, STANDARD tier) and Azure AI Content Safety (West Europe, FourSeverityLevels). Limitations of the head-to-head comparison are described in the methodology section of the full report.

Download the full NeuralTrust AI Security Model Performance Report (PDF)