NeuralTrust has been recognized by Gartner → Read more
Back

Self-Hosted AI Gateway: Deploy in 7 Steps

Roger Howroyd October 21, 2026
Share
Self-Hosted AI Gateway: Deploy in 7 Steps

Last updated: October 22, 2026

How do you deploy a self-hosted AI gateway? You run the gateway's data plane inside your own Kubernetes cluster, point your applications at it, and keep prompts, provider keys and logs in infrastructure you control. This guide walks through that setup for Agent Gateway (TrustGate), step by step, using NeuralTrust's public deployment documentation as of October 2026.

Every command comes from the documentation.

TL;DR: Key Takeaways

  • According to NeuralTrust's deployment documentation (2026), there are 4 deployment models: SaaS, Private (Hybrid), External and a Central control plane for several clusters.
  • A standalone Private data plane needs Redis and PostgreSQL, plus 2 entry points: port 8081 for LLM traffic and port 8082 for MCP (requirements).
  • The install takes 2 Kubernetes Secrets and 1 Helm command (Deploy TrustGate).
  • The proxy and MCP workloads run 2 replicas by default, and autoscaling is off by default (high availability).
  • OWASP ranks prompt injection first among 10 LLM risks, so a private network is not enough (OWASP, 2025).
ModelWho runs whatBest for
SaaSNeuralTrust runs both planesFast start, free tier, no infrastructure
Private (hybrid)You run the data plane, NeuralTrust runs the control planePrompts must stay in your network
External (self-hosted)You run everythingAir-gapped or strict residency
Central control planeYou run one control plane and several data planes in your own clustersBusiness units, jurisdictions or environments that need one console

Which Self-Hosted AI Gateway Model Fits You?

A self-hosted AI gateway splits into a data plane, which carries requests, and a control plane, which holds configuration and the console. The models differ by where each plane runs.

In the Private model, the data plane runs in your cluster and pulls its configuration from the NeuralTrust control plane. Your cluster opens the connection, so nothing is pushed in. Raw prompts and responses stay in your PostgreSQL, and only metadata is exported, through a collector inside your cluster, according to the Hybrid guide (2026).

In the External model, you run both planes, the console and the analytics stack. The documentation positions it for air-gapped or strict data-residency environments, and it needs at least 4 nodes of 8 vCPU and 32 GiB.

In the Central control plane model, you run one control plane and place data planes in other clusters that you operate. Raw payloads stay in the cluster that produced them, and metadata goes to your central ClickHouse, according to the deployment overview (2026). The documentation positions it for separate business units, jurisdictions or environments that still need one console, one alerting setup and one place for reporting.

This guide covers the Private model, the fastest way to keep prompts inside your network. We built the steps from the public documentation in October 2026, and placeholders such as <VERSION> stay as placeholders.

Prerequisites and Sizing

RequirementDetail
Kubernetes clusterAt least 2 nodes, 8 vCPU and 16 GiB each
RedisRequired. Used for rate limiting and semantic caching. Redis OSS is sufficient
PostgreSQLRequired. Production floor of 2 vCPU, 4 GiB RAM and 20 GiB storage
Ingress controller and DNSPublic entry on 443, with hostnames you control
Registry keyNeeded to pull the image. Issued with an enterprise offer (support@neuraltrust.ai)
Outbound accessHTTPS on port 443 to three NeuralTrust hosts: config sync, telemetry and DataBridge (databridge.neuraltrust.ai). Also to your LLM providers and your registry
Inbound accessHTTPS to your published entry points from the console's source IP

Kafka, a vector database, a service mesh and GPU nodes are not required, according to the requirements page.

The registry key matters if you planned a free path. The free AI Gateway tier runs as SaaS, while a Private data plane needs the key, so contact the team first.

Try our AI Gateway today for free

Step-by-Step: Deploy the Gateway on Kubernetes

This walkthrough follows the Deploy TrustGate page, from an empty namespace to a routed request.

Step 1: Prepare the namespace and pull secret

Create the namespace and the image pull Secret with the registry key you received.

Copied!

Pull the chart sources as well, so the example values file is on disk: helm pull <chart-ref> --version <VERSION> --untar.

Step 2: Create the gateway in the console

In the console, open TrustGate, choose New Gateway, name it, then choose Private and Kubernetes. Copy CONFIG_SYNC_TOKEN and the DataAgent enrollment JWT from the wizard, and keep the generated values file secret.

Step 3: Create the two Secrets

Copied!

In a standard install, do not create CONFIG_SYNC_LKG_KEY. The chart generates it. GitOps installs are the exception, covered in the Security and Governance section below.

Step 4: Set the cluster values

Write a small values file with your platform and domain.

Copied!

For production, set global.postgresql.deploy: false and global.redis.deploy: false, then give each its host. The documentation calls the in-cluster stores a proof-of-concept default (configuration).

Step 5: Install with Helm

Copied!

The example file turns on TrustGate and wires in both Secrets. Pin <VERSION> for a reproducible install.

Step 6: Expose the LLM and MCP entry points with TLS

Use separate hostnames. The LLM URL maps to port 8081 and the MCP URL to port 8082.

ServiceHostnamePort
LLM trafficgateway.<domain>8081
MCP trafficmcp.<domain>8082

Point DNS at your ingress, then enter both full URLs, including https://, under Settings, Agent Gateway, General in the console. Do not bootstrap both from one URL. The Hybrid guide warns that this breaks MCP silently: tool calls never resolve and no error appears.

Terminate TLS at the ingress. A certificate manager such as cert-manager can issue certificates through an Ingress annotation. If a TLS-intercepting proxy sits on your egress path, supply its CA in three places: configSync.tlsCa for config sync, dataagent.databridge.tlsCa for DataBridge, and global.clickstack.egress.tlsCaSecretName for telemetry. The last one takes the name of a Secret, not a file path. Each setting replaces the system roots for its own connection, so each bundle must include every CA that connection needs, and setting only one leaves the other two connections untrusted. Never use configSync.tlsInsecure, which deployed environments refuse.

Step 7: Create an application and send the first request

In the console, add a model provider and test the connection, then create an application and issue its key. The key is shown once, so copy it. The provider credential stays in the gateway, and callers hold only their own key. Send a request using the Connect snippet:

Copied!

Clients that accept only a base URL and an API key can send the key as an Authorization: Bearer header, per the quickstart. For tool traffic, add an MCP server in the registry and attach it to an application. To use identity-provider tokens instead of static keys, open the application's Auth settings. Our guide to MCP authentication covers the options.

Verify, Observe and Troubleshoot the Gateway

Start with kubectl get pods -n neuraltrust and kubectl get ingress -n neuraltrust. Expect the agentgateway-proxy, agentgateway-mcp and dataagent workloads to be ready.

A pod that is Running but never Ready usually means config sync is failing. The Hybrid guide lists three causes, in order: a wrong token or one with trailing whitespace, blocked egress to *.neuraltrust.ai on port 443, or a TLS-intercepting proxy whose CA is not trusted. A data plane that cannot reach those hosts can start normally and still stop receiving updates, so test reachability from inside the cluster.

Then test behavior. Send one allowed and one blocked request, and confirm both decisions appear in the console. The Activity view shows status, model, latency, tokens and a trace ID per request.

Check the telemetry split too. In the Private model, raw prompts and responses go to your PostgreSQL through TELEMETRY_EXPORTERS_RAW=postgres. Metadata does not go straight from the gateway to NeuralTrust. The gateway sends it over OTLP, through TELEMETRY_EXPORTERS_METADATA=otlp, to a collector running inside your cluster, and that collector is what exports it to NeuralTrust. The install needs the DataAgent enrollment Secret for this path, so a missing piece surfaces at install time. Still confirm that both flows reach their destinations.

Scale and Keep the Gateway Available

Scale the request path by adding replicas. The documented node counts are scheduling minimums, not throughput figures, so measure under your own load.

SettingDefaultGuidance
agentgateway.dataPlane.replicas2Raise for traffic
agentgateway.mcp.replicas2Raise for tool traffic
dataagent.replicas1Keep at 1
autoscaling.enabledfalseLeave off unless you have a capacity reason

Duplicate DataAgents register duplicate streams, which is why it stays at 1. Enable pod disruption budgets with global.highAvailability.podDisruptionBudget.enabled: true so node drains do not remove every replica at once. The high availability guide defines four tiers, and the first needs node pools across three zones with managed PostgreSQL and Redis.

Plan for a control-plane outage as well. According to the config sync documentation (2026), a running pod keeps serving its last-known-good configuration and logs a staleness warning after 24 hours. The cache lives on an emptyDir, so a pod that restarts without a control-plane connection cannot load its snapshot and does not become Ready.

Security and Governance for a Self-Hosted AI Gateway

Self-hosting moves control of data to you. It does not remove the risks that reach a gateway through prompts, tools and keys. According to OWASP (2025), the top LLM risks include prompt injection (LLM01), sensitive information disclosure (LLM02), excessive agency (LLM06) and unbounded consumption (LLM10). A private network addresses none of them on its own.

Harden four places. Secrets: the Kubernetes documentation recommends encryption at rest and least-privilege RBAC (Kubernetes docs). For GitOps, pre-create every Secret and set global.autoGenerateSecrets: false with global.preserveExistingSecrets: true. That includes CONFIG_SYNC_LKG_KEY. In this mode the chart generates nothing, so add the key to the config sync Secret as a base64 value that decodes to exactly 32 bytes, or the data planes refuse to start. Network: allow only the three documented NeuralTrust hosts outbound, and limit who can reach the entry points, remembering that the console's source IP must be among them. Keys: keep provider credentials in the gateway and issue each application its own key. Access: use an identity provider where you can.

Then add policy. Agent Gateway (TrustGate) applies rate limits, budgets and guardrails, and Enforce mode refuses calls that break a policy. Budgets and rate limits answer unbounded consumption directly. In a full Hybrid install, the gateway also calls Agent Runtime Security (TrustGuard), which calls an ML firewall, as part of NeuralTrust's Runtime Security Mesh. For background, read what an AI gateway is and how it differs from an MCP gateway.

Self-Hosted or Managed: Which Should You Choose?

Choose by data rules and operating capacity. The table maps common situations to a model.

Your situationChoose
You want to test the gateway this weekSaaS, using the free tier
Prompts must not leave your network, but a managed console is finePrivate data plane
Air-gapped, or no dependency on a vendor service allowedExternal (fully self-hosted)
One console across several clusters, with data kept where it is producedCentral control plane
Data must stay in the EU or US, with no infrastructure teamManaged service, hosted in the EU or US
UK-only residency, including metadataExternal, run in your own UK environment

The managed service can be hosted in the EU or US. For UK-only residency, run the platform in your own UK environment. In the Private model, metadata still reaches NeuralTrust telemetry hosts in the EU or US, through the collector in your cluster. The External model has no NeuralTrust network requirement, and in the Central control plane model metadata goes to your own ClickHouse.

Self-hosting costs you operations: upgrades, certificates, backups and on-call. The documentation does not describe upgrade steps for a standalone TrustGate install, so confirm that process with NeuralTrust support.

Conclusion

A self-hosted AI gateway comes down to three decisions: which model fits your data rules, whether your cluster meets the prerequisites, and who owns operations. For teams that need prompts inside their network, a Private data plane delivers that in seven steps and one Helm command. Add managed datastores, policies and a tested recovery plan before production traffic arrives.

Secure Your AI Gateway in Production with NeuralTrust

Start on the free SaaS tier to see the gateway working, then move the data plane into your cluster.

Try our AI Gateway today for free

Related Guides

FAQs about Self-Hosted AI Gateway

1. What is a self-hosted AI gateway?

A self-hosted AI gateway routes and governs LLM and MCP traffic from infrastructure you run. Applications call it instead of calling model providers directly, and provider keys, prompts and logs stay in your environment.

2. Do I need Kubernetes to self-host an AI gateway?

For NeuralTrust's documented install, yes. The Private data plane and the External install deploy with a Helm chart on Kubernetes. The quickstart mentions Docker and manual options, but this guide covers only the Kubernetes path.

3. What infrastructure does a self-hosted AI gateway need?

A standalone data plane needs at least 2 nodes with 8 vCPU and 16 GiB each, Redis, PostgreSQL, an ingress controller and DNS. Kafka, a vector database and GPUs are not required.

4. Is a self-hosted AI gateway free?

The free tier of NeuralTrust's AI Gateway runs as SaaS. A Private data plane needs a registry key that NeuralTrust issues with an enterprise offer. Budget for nodes, managed datastores and operating time too.

5. Does self-hosting keep my prompts private?

In the Private model, raw prompts and responses stay in your PostgreSQL, and only metadata goes to NeuralTrust, through a collector inside your cluster. In the External model, nothing leaves your environment. In the Central control plane model, raw payloads stay in the cluster that produced them.

6. What happens if the control plane goes down?

A running data plane keeps serving its last-known-good configuration and logs a warning after 24 hours. A pod that restarts without control-plane access does not become Ready, so avoid restarts during an outage.

About the Author

Roger Howroyd is Head of Global SEO and AI at NeuralTrust, where he leads the company's search strategy across SEO, AEO, GEO, and LLM optimization. He specializes in AI-powered search, content strategy, and SEM. Connect on LinkedIn.

NeuralTrust is the leading platform for securing and scaling AI agents. Named a Pioneer in the Gartner Emerging Market Quadrant for AI Application Security 2026, recognized across four Gartner Hype Cycle reports in 2026, and featured in the Gartner Market Guide for Guardian Agents 2026, the Gartner Market Guide for AI Gateways 2025 and the KuppingerCole Leadership Compass for Generative AI Defense 2025. Headquartered in Barcelona with offices in London and New York. ISO 27001 certified.

Sources

  1. NeuralTrust Docs: Deployment overview (accessed October 2026)
  2. NeuralTrust Docs: Hybrid deployment (accessed October 2026)
  3. NeuralTrust Docs: Deploy TrustGate (accessed October 2026)
  4. NeuralTrust Docs: Requirements and dependencies (accessed October 2026)
  5. NeuralTrust Docs: Config sync (accessed October 2026)
  6. NeuralTrust Docs: High availability (accessed October 2026)
  7. NeuralTrust Docs: Configuration (accessed October 2026)
  8. NeuralTrust Docs: TrustGate quickstart (accessed October 2026)
  9. OWASP Top 10 for LLM Applications 2025 (accessed October 2026)
  10. Kubernetes Docs: Good practices for Kubernetes Secrets (accessed October 2026)

Subscribe to our newsletter

Share

Join the leaders securing the agent ecosystem

Get a Demo