Keep Agents Running When AI APIs Fail

If your agents call a provider directly, every request fails until that provider recovers, and your users see errors or stalled workflows. With TrustGate in front of your models, the gateway detects the failing provider and sends traffic to a backup model you have already configured, so your agents keep answering while the outage runs its course.

More often than most teams expect. Every major provider publishes incidents regularly, ranging from full outages to elevated error rates, slow responses, and quota or rate limits. Some weeks see several incidents, and a single day can bring more than one. Any agent tied to one provider inherits every one of those incidents.

LLM fallback is the practice of automatically sending a request to an alternative model or provider when the primary one fails, times out, or hits a limit. In TrustGate it is configured once at the gateway as an ordered list of backups, so your application keeps making the same call and never needs its own failover code.

Fallback happens within the same request. When the gateway sees an error, a timeout, or a rate limit response, it retries according to your rules and then moves to the next provider in the list, so the request completes without your application ever handling the failure. Users typically notice nothing beyond a slightly longer response.

Yes. Every retry and fallback is logged at the gateway with the original provider, the error that triggered the switch, the backup that served the request, and the timing. You can review this per agent, per region, or per provider to see how often outages hit you and confirm that your fallback rules worked as expected.