Last updated: October 22, 2026
How do you deploy a self-hosted AI gateway? You run the gateway's data plane inside your own Kubernetes cluster, point your applications at it, and keep prompts, provider keys and logs in infrastructure you control. This guide walks through that setup for Agent Gateway (TrustGate), step by step, using NeuralTrust's public deployment documentation as of October 2026.
Every command comes from the documentation.
TL;DR: Key Takeaways
- According to NeuralTrust's deployment documentation (2026), there are 4 deployment models: SaaS, Private (Hybrid), External and a Central control plane for several clusters.
- A standalone Private data plane needs Redis and PostgreSQL, plus 2 entry points: port 8081 for LLM traffic and port 8082 for MCP (requirements).
- The install takes 2 Kubernetes Secrets and 1 Helm command (Deploy TrustGate).
- The proxy and MCP workloads run 2 replicas by default, and autoscaling is off by default (high availability).
- OWASP ranks prompt injection first among 10 LLM risks, so a private network is not enough (OWASP, 2025).
| Model | Who runs what | Best for |
|---|---|---|
| SaaS | NeuralTrust runs both planes | Fast start, free tier, no infrastructure |
| Private (hybrid) | You run the data plane, NeuralTrust runs the control plane | Prompts must stay in your network |
| External (self-hosted) | You run everything | Air-gapped or strict residency |
| Central control plane | You run one control plane and several data planes in your own clusters | Business units, jurisdictions or environments that need one console |
Which Self-Hosted AI Gateway Model Fits You?
A self-hosted AI gateway splits into a data plane, which carries requests, and a control plane, which holds configuration and the console. The models differ by where each plane runs.
In the Private model, the data plane runs in your cluster and pulls its configuration from the NeuralTrust control plane. Your cluster opens the connection, so nothing is pushed in. Raw prompts and responses stay in your PostgreSQL, and only metadata is exported, through a collector inside your cluster, according to the Hybrid guide (2026).
In the External model, you run both planes, the console and the analytics stack. The documentation positions it for air-gapped or strict data-residency environments, and it needs at least 4 nodes of 8 vCPU and 32 GiB.
In the Central control plane model, you run one control plane and place data planes in other clusters that you operate. Raw payloads stay in the cluster that produced them, and metadata goes to your central ClickHouse, according to the deployment overview (2026). The documentation positions it for separate business units, jurisdictions or environments that still need one console, one alerting setup and one place for reporting.
This guide covers the Private model, the fastest way to keep prompts inside your network. We built the steps from the public documentation in October 2026, and placeholders such as <VERSION> stay as placeholders.
Prerequisites and Sizing
| Requirement | Detail |
|---|---|
| Kubernetes cluster | At least 2 nodes, 8 vCPU and 16 GiB each |
| Redis | Required. Used for rate limiting and semantic caching. Redis OSS is sufficient |
| PostgreSQL | Required. Production floor of 2 vCPU, 4 GiB RAM and 20 GiB storage |
| Ingress controller and DNS | Public entry on 443, with hostnames you control |
| Registry key | Needed to pull the image. Issued with an enterprise offer (support@neuraltrust.ai) |
| Outbound access | HTTPS on port 443 to three NeuralTrust hosts: config sync, telemetry and DataBridge (databridge.neuraltrust.ai). Also to your LLM providers and your registry |
| Inbound access | HTTPS to your published entry points from the console's source IP |
Kafka, a vector database, a service mesh and GPU nodes are not required, according to the requirements page.
The registry key matters if you planned a free path. The free AI Gateway tier runs as SaaS, while a Private data plane needs the key, so contact the team first.
Step-by-Step: Deploy the Gateway on Kubernetes
This walkthrough follows the Deploy TrustGate page, from an empty namespace to a routed request.
Step 1: Prepare the namespace and pull secret
Create the namespace and the image pull Secret with the registry key you received.
Pull the chart sources as well, so the example values file is on disk: helm pull <chart-ref> --version <VERSION> --untar.
Step 2: Create the gateway in the console
In the console, open TrustGate, choose New Gateway, name it, then choose Private and Kubernetes. Copy CONFIG_SYNC_TOKEN and the DataAgent enrollment JWT from the wizard, and keep the generated values file secret.
Step 3: Create the two Secrets
In a standard install, do not create CONFIG_SYNC_LKG_KEY. The chart generates it. GitOps installs are the exception, covered in the Security and Governance section below.
Step 4: Set the cluster values
Write a small values file with your platform and domain.
For production, set global.postgresql.deploy: false and global.redis.deploy: false, then give each its host. The documentation calls the in-cluster stores a proof-of-concept default (configuration).
Step 5: Install with Helm
The example file turns on TrustGate and wires in both Secrets. Pin <VERSION> for a reproducible install.
Step 6: Expose the LLM and MCP entry points with TLS
Use separate hostnames. The LLM URL maps to port 8081 and the MCP URL to port 8082.
| Service | Hostname | Port |
|---|---|---|
| LLM traffic | gateway.<domain> | 8081 |
| MCP traffic | mcp.<domain> | 8082 |
Point DNS at your ingress, then enter both full URLs, including https://, under Settings, Agent Gateway, General in the console. Do not bootstrap both from one URL. The Hybrid guide warns that this breaks MCP silently: tool calls never resolve and no error appears.
Terminate TLS at the ingress. A certificate manager such as cert-manager can issue certificates through an Ingress annotation. If a TLS-intercepting proxy sits on your egress path, supply its CA in three places: configSync.tlsCa for config sync, dataagent.databridge.tlsCa for DataBridge, and global.clickstack.egress.tlsCaSecretName for telemetry. The last one takes the name of a Secret, not a file path. Each setting replaces the system roots for its own connection, so each bundle must include every CA that connection needs, and setting only one leaves the other two connections untrusted. Never use configSync.tlsInsecure, which deployed environments refuse.
Step 7: Create an application and send the first request
In the console, add a model provider and test the connection, then create an application and issue its key. The key is shown once, so copy it. The provider credential stays in the gateway, and callers hold only their own key. Send a request using the Connect snippet:
Clients that accept only a base URL and an API key can send the key as an Authorization: Bearer header, per the quickstart. For tool traffic, add an MCP server in the registry and attach it to an application. To use identity-provider tokens instead of static keys, open the application's Auth settings. Our guide to MCP authentication covers the options.
Verify, Observe and Troubleshoot the Gateway
Start with kubectl get pods -n neuraltrust and kubectl get ingress -n neuraltrust. Expect the agentgateway-proxy, agentgateway-mcp and dataagent workloads to be ready.
A pod that is Running but never Ready usually means config sync is failing. The Hybrid guide lists three causes, in order: a wrong token or one with trailing whitespace, blocked egress to *.neuraltrust.ai on port 443, or a TLS-intercepting proxy whose CA is not trusted. A data plane that cannot reach those hosts can start normally and still stop receiving updates, so test reachability from inside the cluster.
Then test behavior. Send one allowed and one blocked request, and confirm both decisions appear in the console. The Activity view shows status, model, latency, tokens and a trace ID per request.
Check the telemetry split too. In the Private model, raw prompts and responses go to your PostgreSQL through TELEMETRY_EXPORTERS_RAW=postgres. Metadata does not go straight from the gateway to NeuralTrust. The gateway sends it over OTLP, through TELEMETRY_EXPORTERS_METADATA=otlp, to a collector running inside your cluster, and that collector is what exports it to NeuralTrust. The install needs the DataAgent enrollment Secret for this path, so a missing piece surfaces at install time. Still confirm that both flows reach their destinations.
Scale and Keep the Gateway Available
Scale the request path by adding replicas. The documented node counts are scheduling minimums, not throughput figures, so measure under your own load.
| Setting | Default | Guidance |
|---|---|---|
agentgateway.dataPlane.replicas | 2 | Raise for traffic |
agentgateway.mcp.replicas | 2 | Raise for tool traffic |
dataagent.replicas | 1 | Keep at 1 |
autoscaling.enabled | false | Leave off unless you have a capacity reason |
Duplicate DataAgents register duplicate streams, which is why it stays at 1. Enable pod disruption budgets with global.highAvailability.podDisruptionBudget.enabled: true so node drains do not remove every replica at once. The high availability guide defines four tiers, and the first needs node pools across three zones with managed PostgreSQL and Redis.
Plan for a control-plane outage as well. According to the config sync documentation (2026), a running pod keeps serving its last-known-good configuration and logs a staleness warning after 24 hours. The cache lives on an emptyDir, so a pod that restarts without a control-plane connection cannot load its snapshot and does not become Ready.
Security and Governance for a Self-Hosted AI Gateway
Self-hosting moves control of data to you. It does not remove the risks that reach a gateway through prompts, tools and keys. According to OWASP (2025), the top LLM risks include prompt injection (LLM01), sensitive information disclosure (LLM02), excessive agency (LLM06) and unbounded consumption (LLM10). A private network addresses none of them on its own.
Harden four places. Secrets: the Kubernetes documentation recommends encryption at rest and least-privilege RBAC (Kubernetes docs). For GitOps, pre-create every Secret and set global.autoGenerateSecrets: false with global.preserveExistingSecrets: true. That includes CONFIG_SYNC_LKG_KEY. In this mode the chart generates nothing, so add the key to the config sync Secret as a base64 value that decodes to exactly 32 bytes, or the data planes refuse to start. Network: allow only the three documented NeuralTrust hosts outbound, and limit who can reach the entry points, remembering that the console's source IP must be among them. Keys: keep provider credentials in the gateway and issue each application its own key. Access: use an identity provider where you can.
Then add policy. Agent Gateway (TrustGate) applies rate limits, budgets and guardrails, and Enforce mode refuses calls that break a policy. Budgets and rate limits answer unbounded consumption directly. In a full Hybrid install, the gateway also calls Agent Runtime Security (TrustGuard), which calls an ML firewall, as part of NeuralTrust's Runtime Security Mesh. For background, read what an AI gateway is and how it differs from an MCP gateway.
Self-Hosted or Managed: Which Should You Choose?
Choose by data rules and operating capacity. The table maps common situations to a model.
| Your situation | Choose |
|---|---|
| You want to test the gateway this week | SaaS, using the free tier |
| Prompts must not leave your network, but a managed console is fine | Private data plane |
| Air-gapped, or no dependency on a vendor service allowed | External (fully self-hosted) |
| One console across several clusters, with data kept where it is produced | Central control plane |
| Data must stay in the EU or US, with no infrastructure team | Managed service, hosted in the EU or US |
| UK-only residency, including metadata | External, run in your own UK environment |
The managed service can be hosted in the EU or US. For UK-only residency, run the platform in your own UK environment. In the Private model, metadata still reaches NeuralTrust telemetry hosts in the EU or US, through the collector in your cluster. The External model has no NeuralTrust network requirement, and in the Central control plane model metadata goes to your own ClickHouse.
Self-hosting costs you operations: upgrades, certificates, backups and on-call. The documentation does not describe upgrade steps for a standalone TrustGate install, so confirm that process with NeuralTrust support.
Conclusion
A self-hosted AI gateway comes down to three decisions: which model fits your data rules, whether your cluster meets the prerequisites, and who owns operations. For teams that need prompts inside their network, a Private data plane delivers that in seven steps and one Helm command. Add managed datastores, policies and a tested recovery plan before production traffic arrives.
Secure Your AI Gateway in Production with NeuralTrust
Start on the free SaaS tier to see the gateway working, then move the data plane into your cluster.
Related Guides
FAQs about Self-Hosted AI Gateway
1. What is a self-hosted AI gateway?
A self-hosted AI gateway routes and governs LLM and MCP traffic from infrastructure you run. Applications call it instead of calling model providers directly, and provider keys, prompts and logs stay in your environment.
2. Do I need Kubernetes to self-host an AI gateway?
For NeuralTrust's documented install, yes. The Private data plane and the External install deploy with a Helm chart on Kubernetes. The quickstart mentions Docker and manual options, but this guide covers only the Kubernetes path.
3. What infrastructure does a self-hosted AI gateway need?
A standalone data plane needs at least 2 nodes with 8 vCPU and 16 GiB each, Redis, PostgreSQL, an ingress controller and DNS. Kafka, a vector database and GPUs are not required.
4. Is a self-hosted AI gateway free?
The free tier of NeuralTrust's AI Gateway runs as SaaS. A Private data plane needs a registry key that NeuralTrust issues with an enterprise offer. Budget for nodes, managed datastores and operating time too.
5. Does self-hosting keep my prompts private?
In the Private model, raw prompts and responses stay in your PostgreSQL, and only metadata goes to NeuralTrust, through a collector inside your cluster. In the External model, nothing leaves your environment. In the Central control plane model, raw payloads stay in the cluster that produced them.
6. What happens if the control plane goes down?
A running data plane keeps serving its last-known-good configuration and logs a warning after 24 hours. A pod that restarts without control-plane access does not become Ready, so avoid restarts during an outage.
About the Author
Roger Howroyd is Head of Global SEO and AI at NeuralTrust, where he leads the company's search strategy across SEO, AEO, GEO, and LLM optimization. He specializes in AI-powered search, content strategy, and SEM. Connect on LinkedIn.
NeuralTrust is the leading platform for securing and scaling AI agents. Named a Pioneer in the Gartner Emerging Market Quadrant for AI Application Security 2026, recognized across four Gartner Hype Cycle reports in 2026, and featured in the Gartner Market Guide for Guardian Agents 2026, the Gartner Market Guide for AI Gateways 2025 and the KuppingerCole Leadership Compass for Generative AI Defense 2025. Headquartered in Barcelona with offices in London and New York. ISO 27001 certified.
Sources
- NeuralTrust Docs: Deployment overview (accessed October 2026)
- NeuralTrust Docs: Hybrid deployment (accessed October 2026)
- NeuralTrust Docs: Deploy TrustGate (accessed October 2026)
- NeuralTrust Docs: Requirements and dependencies (accessed October 2026)
- NeuralTrust Docs: Config sync (accessed October 2026)
- NeuralTrust Docs: High availability (accessed October 2026)
- NeuralTrust Docs: Configuration (accessed October 2026)
- NeuralTrust Docs: TrustGate quickstart (accessed October 2026)
- OWASP Top 10 for LLM Applications 2025 (accessed October 2026)
- Kubernetes Docs: Good practices for Kubernetes Secrets (accessed October 2026)
)
)
)
)
)
)
)