Benchmarking Jailbreak Detection: NeuralTrust vs Google, AWS, Azure (2026)

Last updated: October 2026

How do leading AI guardrail solutions actually perform when attackers try to jailbreak your LLMs?

This benchmark compares NeuralTrust Prompt Guard against Google Model Armor, AWS Bedrock Guardrails, and Azure AI Content Safety across four independent benchmark suites at production scale, with real-world adversarial traffic.

TL;DR: Key Takeaways

• NeuralTrust Prompt Guard detects 94.9% of jailbreak attempts on JailbreakBench attack artifacts, outperforming Google (92.0%), AWS (82.8%), and Azure (78.4%).
• On JailbreakBench Behaviors (the most realistic harmful-use benchmark) NeuralTrust scores 95.0% vs Google 90.0%, AWS 69.0%, and Azure 65.0%.
• At production scale (122,185 real prompts), Prompt Guard maintains a 0.72% false positive rate, meaning legitimate users are almost never blocked.
• For indirect prompt injection through tool calls and MCP servers, NeuralTrust IPI detector reaches 98.2% detection (100% on InjecAgent benchmark).
• Google Model Armor leads on HarmBench overall (67.8% vs NeuralTrust 63.7%) and produces fewer false positives on benign content (XSTest FPR: 11.6% vs 19.6%). No single solution dominates every category.

What Changed Since April 2025

The original version of this article compared NeuralTrust against AWS Bedrock and Azure AI Content Safety on a dataset of 600 prompts. Since then three things have changed:

1. Scale: Benchmarks now cover 176,338 prompts (including 122,185 verified production inputs) replacing the 600-prompt research dataset.
2. New competitor: Google launched Model Armor in 2025. It is now a primary option for teams running workloads on Google Cloud and deserves inclusion.
3. New attack vectors: Indirect prompt injection through tool calls and MCP servers has emerged as the dominant enterprise threat vector for AI agents, displacing simple text-based jailbreaks.

We have rerun every comparison with October 2026 data.

Benchmark Setup

Datasets used

All public benchmarks were run in October 2026. AWS and Azure results use their July 2026 API snapshots; Google Model Armor and NeuralTrust use October 2026 snapshots. Scores reflect each vendor's public API with default configuration, no custom fine-tuning or prompt engineering.

How we compared

Each prompt was sent to each provider's API via its standard guardrail endpoint. For jailbreak benchmarks, detection rate is the share of jailbreak prompts correctly flagged. For XSTest, false positive rate is the share of clearly benign prompts incorrectly blocked. All providers were tested against the same prompt sets in the same week. Latency was not re-tested in this update; prior measurements showed NeuralTrust at ~0.08 s average vs ~0.27 s for cloud provider APIs.

Jailbreak Detection Results

JailbreakBench: attack artifacts

This benchmark uses automated adversarial prompts generated by jailbreak algorithms (GCG, PAIR, TAP). It tests whether a guardrail recognises the structural fingerprints of attack scaffolding.

NeuralTrust leads by 2.9 percentage points over Google and by 12.1 pp over AWS. At enterprise scale (say, 10 million prompts per month) the 12-point gap versus AWS represents roughly 1.2 million undetected attack attempts per month.

JailbreakBench Behaviors: real harmful scenarios

JailbreakBench Behaviors tests prompts that map to genuinely harmful outcomes (cyberweapon generation, CBRN instructions, targeted harassment). This is the benchmark that most closely mirrors what a CISO cares about.

The 5-point gap over Google widens to 26 points over AWS and 30 points over Azure. The NeuralTrust result combines Prompt Guard and Toxicity detectors; the Prompt Guard detector alone reaches 94.9% on this benchmark.

Explicit attack categories (Prompt Guard)

For high-confidence attack types (attempts to override system instructions, extract training data, or force harmful completions) Prompt Guard reaches 98.4% detection across 176,338 evaluated prompts.

Broad Harmful Content: HarmBench

HarmBench covers 510 behaviours across a wider range of harm categories than JailbreakBench, including copyright violations, privacy attacks, and disinformation generation. Google Model Armor leads on this benchmark.

Google's 4.1-point advantage on HarmBench reflects its broader training on non-jailbreak harm categories. Teams with strong copyright or disinformation risk profiles should weigh this benchmark heavily. For teams primarily concerned with prompt injection and jailbreak (the majority of enterprise AI security deployments) JailbreakBench Behaviors is more predictive.

False Positives: XSTest

A guardrail that blocks too many legitimate requests destroys user experience. XSTest measures false positive rate on 250 clearly benign prompts.

Google's 8-point FPR advantage on XSTest is real. In practice, NeuralTrust's production FPR of 0.72% on 122,185 live enterprise prompts is lower than XSTest suggests because production traffic skews toward in-domain, unambiguous prompts, whereas XSTest deliberately uses edge cases. Both numbers matter: XSTest is the stress test; production FPR is the operating reality.

Indirect Prompt Injection: The Emerging Enterprise Threat

Indirect prompt injection is the attack vector that matters most in 2026 agentic deployments. Standard jailbreak benchmarks do not cover it.

NeuralTrust IPI detector results (October 2026, 71,995 verified inputs):

No equivalent published benchmark exists for Google Model Armor, AWS Bedrock, or Azure AI Content Safety on indirect prompt injection as of October 2026.

Security & Governance: What the Benchmarks Don't Show

Benchmark scores reflect a snapshot in time against published datasets. Enterprise risk management requires more than a detection rate.

Prompt injection (OWASP LLM01) remains the top-ranked risk in the OWASP LLM Top 10. According to Gartner (2026), 40% of enterprise AI incidents involve prompt manipulation, including indirect injection through retrieval-augmented generation pipelines. A guardrail that scores well on JailbreakBench but has no IPI coverage leaves agent workflows exposed.

Data leakage and tool misuse in multi-step agentic pipelines require monitoring across the full reasoning trace, not just the user-turn input. NeuralTrust Agent Runtime Security (TrustGuard) instruments each tool call and provides a forensic audit trail, capability that sits outside what any input-only guardrail provides.

Vendor-reported benchmarks (including this one) should be treated as vendor-reported benchmarks. Cross-reference with independent evaluations where available: Gray Swan's adversarial evaluations and Gartner's AI Application Security Market Guide both provide vendor-neutral context.

Which Solution Should You Choose?

Conclusion

Benchmarking jailbreak detection solutions in 2026 requires moving beyond the 2024-era research datasets. At production scale and against current adversarial techniques, NeuralTrust Prompt Guard leads on the benchmarks most predictive of enterprise AI security risk: JailbreakBench attack artifacts (94.9%), JailbreakBench Behaviors (95.0%), and indirect prompt injection via tool calls (98.2%). Google Model Armor is the strongest alternative for teams whose risk profile skews toward broad harmful content or who need the lowest possible false positive rate. AWS and Azure trail on jailbreak-specific metrics by double digits.

Secure Your AI Agents in Production with NeuralTrust

Benchmark scores answer "does this guardrail fire on known attacks?" Runtime security answers "what is actually happening in my AI agent pipelines right now?" NeuralTrust Agent Gateway (TrustGate) and Agent Runtime Security (TrustGuard) instrument every LLM call, tool invocation, and model response with the detection rates above applied in real time.

Related Comparisons

• Claude Opus 5.5 Enterprise Security
• GPT-6 Astra: CISO Security Implications

FAQs about Benchmarking Jailbreak Detection Solutions

1. What is the difference between JailbreakBench and HarmBench?

JailbreakBench focuses specifically on adversarial prompts designed to bypass safety filters through structural manipulation, the type of attack most common in enterprise environments. HarmBench is broader: it covers 510 harm categories including copyright and privacy attacks, not just prompt injection. For teams protecting against active attackers trying to manipulate their AI systems, JailbreakBench Behaviors is more predictive. For teams with broad content moderation requirements, HarmBench is more relevant.

2. Why does the false positive rate matter as much as detection rate?

A guardrail that blocks 95% of attacks but also blocks 20% of legitimate requests will be disabled by frustrated users within weeks. False positive rate determines whether security is actually deployed at scale. At 0.72% FPR in production, NeuralTrust blocks roughly 7 legitimate prompts per 1,000 - acceptable for most enterprise use cases. Google's XSTest FPR of 11.6% on benchmark edge cases is impressive, though production FPR for all vendors will differ from benchmark results.

3. What is indirect prompt injection and why does it matter for AI agents?

Indirect prompt injection happens when an AI agent reads attacker-controlled content that contains instructions designed to override the agent's original instructions. Unlike direct jailbreaks, the attack is invisible to the user: the agent appears to behave normally while following the attacker's commands. According to the OWASP LLM Top 10 (2025), this is the highest-risk vulnerability for agentic AI deployments. Standard guardrail benchmarks do not test for it.

4. How were the benchmarks run?

All benchmarks used each vendor's public API with default configuration in October 2026. No vendor-specific fine-tuning, prompt engineering, or configuration optimisation was applied. This reflects the out-of-the-box experience an enterprise team would get on day one of deployment. AWS and Azure API snapshots are from July 2026; Google Model Armor and NeuralTrust from October 2026.

5. Does NeuralTrust independently audit its own benchmarks?

The benchmarks in this article were produced by NeuralTrust using publicly available datasets (JailbreakBench, HarmBench, XSTest, InjecAgent) against its own and competitors' public APIs. Production results (0.72% FPR, 93.8% detection) are from NeuralTrust's own infrastructure logs, not independently audited. As with all vendor-reported benchmarks, treat these as one data point and cross-reference with independent sources. We include results where competitors outperform NeuralTrust (Google on HarmBench and XSTest) because credible benchmarking means reporting unfavourable results too.

6. Which solution is best for a team already using AWS or Google Cloud?

Infrastructure fit matters. AWS Bedrock Guardrails integrates natively into the Bedrock model stack with no additional data egress. Google Model Armor integrates with Vertex AI and Cloud Run. NeuralTrust Agent Gateway (TrustGate) is cloud-agnostic and deployable on any infrastructure, including self-hosted. If jailbreak detection rates are the primary criterion, NeuralTrust leads regardless of cloud. If existing cloud-native tooling simplifies procurement and reduces integration time, factor that in alongside the benchmark data.

About the Author

Roger Howroyd is Head of Global SEO and AI at NeuralTrust, where he leads the company's search strategy across SEO, AEO, GEO, and LLM optimization. He specializes in AI-powered search, content strategy, and SEM. Connect on LinkedIn.

NeuralTrust is the leading platform for securing and scaling AI agents. Named a Pioneer in the Gartner Emerging Market Quadrant for AI Application Security 2026, recognized across four Gartner Hype Cycle reports in 2026, and featured in the Gartner Market Guide for Guardian Agents 2026, the Gartner Market Guide for AI Gateways 2025 and the KuppingerCole Leadership Compass for Generative AI Defense 2025. Headquartered in Barcelona with offices in London and New York. ISO 27001 certified.

Sources

1. Chao, P. et al. (2024). JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models. https://jailbreakbench.github.io/
2. Mazeika, M. et al. (2024). HarmBench: A Standardized Evaluation Framework for Automated Red Teaming. https://www.harmbench.org/
3. Röttger, P. et al. (2023). XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. https://arxiv.org/abs/2308.01263
4. Zhan, Q. et al. (2024). InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. https://arxiv.org/abs/2403.02691
5. NeuralTrust (October 2026). Prompt Guard Benchmark Report.
6. NeuralTrust (October 2026). Indirect Prompt Injection Detector Benchmark Report.
7. NeuralTrust (October 2026). NeuralTrust Guardrails Benchmark: Comparison with Google Model Armor, AWS Bedrock, Azure AI.
8. OWASP (2025). OWASP Top 10 for Large Language Model Applications. https://owasp.org/www-project-top-10-for-large-language-model-applications/
9. Gartner (2026). Hype Cycle for AI Application Security. (Vendor-cited recognition.)
10. Gray Swan AI (2026). Adversarial Robustness Evaluations for Commercial LLM Guardrails.