One gateway for all your AI Applications

AI Gateway is one endpoint between your applications and the AI models they call, built on the Azion Web Platform. It authenticates each team, picks a model by policy, falls back when a call fails, caches responses the caller allows, and writes a usage and audit log line for every call. It runs as a function on Azion's distributed infrastructure, behind Firewall.

Functions runs the gateway logic on an application, and AI Inference runs the models hosted on Azion. Firewall applies WAF, rate limits, and DDoS Protection before a request reaches the function. KV Store holds team keys and policies, the Cache API stores allowed responses, and Connectors reach your own model servers. Real-Time Events, Real-Time Metrics, and Data Stream record and export every call.

Any model on AI Inference, including Qwen3 30B A3B, GPT-OSS 20B, and Mistral 3 Small, plus embedding, reranking, and vision models. The gateway also calls third-party providers whose APIs accept the OpenAI chat completions format, and your own model servers through Connectors. For a provider with its own request format, adapt the request in the function.

Usually only the base URL and the key. The gateway accepts OpenAI chat completions requests at /v1/chat/completions, so OpenAI SDKs and compatible clients keep working. Teams send a model alias, such as general, and the platform team decides which model answers it.

Each alias has an ordered list of models. When a call throws an error, returns no choices, answers with an error status, or runs past its timeout, the gateway tries the next model. Every failed attempt is logged, and if all of them fail, the gateway answers 502.

Every call logs the team and the tokens the model used, so your analytics platform can sum spend per team. When a team reaches its budget, set its status to blocked, and the gateway answers 429 to its key. Third-party calls use your own provider keys, so providers bill your accounts at their own prices, with no markup from Azion.

Provider keys are stored as secret environment variables and read inside the function, so they never appear in code or reach a team. Each team gets its own gateway key, and KV Store keeps only its SHA-256 hash. To validate JSON Web Tokens instead, run the JWT function from Marketplace on Firewall.

Real-Time Metrics charts requests, status codes, upstream status, and function invocations. Real-Time Events keeps each request's log lines for 7 days, grouped by request ID. Data Stream sends the same lines to Splunk, Datadog, Elasticsearch, Google BigQuery, Amazon S3, Apache Kafka, or any HTTPS endpoint.

Managed AI gateways give you a fixed set of features to configure. On Azion, the gateway is code you own: routing, fallback, caching, and authentication live in one function, so you can apply any policy your teams need. The same platform runs models on AI Inference and protects the endpoint with Firewall, so there's no separate proxy to secure.

An application with Application Accelerator, which lets a rule run the gateway function, and KV Store, which is in Preview and enabled through Technical Support. The documentation covers the full setup, from the function code to the usage stream. Talk to a specialist to plan the models, routes, and budgets for your teams.