Build AI agents
Run a tool-calling agent loop in a function on Azion, with a third-party model, state between steps, human approval, and a history of every task.
A product or operations team builds agents that complete tasks instead of only answering: they call internal APIs and SaaS tools, keep state across steps, and hand off to a person for approval when needed. A model that can call tools still decides nothing on its own. Something has to run the loop, execute each tool the model asks for, stop before an action a person must approve, and remember where the task stopped. This page runs that loop as a function on Azion with a tool-calling model from a third-party provider, keeps paused tasks in KV Store, and records every task in SQL Database. The result is measured by task completion rate, steps and latency per task, and the share of actions that needed human approval.
This use case does not cover assistants that only answer from company content, or exposing tools to external agents. For those, refer to Build and run customer support AI assistants and Deploy remote MCP servers.
Prerequisites
- An application and a workload that serve your domain, with Application Accelerator turned on, which the Run Function behavior requires. To create them, refer to Applications quickstart.
- KV Store and SQL Database enabled on the account. Both are in Preview and not enabled by default, so request access through Technical Support.
- A personal token with the Edit SQL Database permission, for the database and KV Store calls. To create one, refer to Personal tokens.
- The Azion CLI, installed and authorized, to store the environment variables.
- A third-party provider whose chat endpoint accepts the OpenAI chat completions format with tool definitions, with its URL, the name of a model that supports tool calling, and an API key.
- The values of your own setup. This page gives the agent two tools over an order service at
https://api.example.com/v1:get_order, which readsGET /orders/{id}, andrefund_order, which callsPOST /orders/{id}/refundsand needs a person’s approval. It usesagent-statefor the KV Store namespace,agent-historyfor the database, andwww.example.comfor the domain. Replace each value with yours in every step.
Required products
| The agent needs | Which means | Product | Documented in |
|---|---|---|---|
| A loop that sends the task to the model, runs the tools it asks for, and stops when it answers | A function that calls the provider and the internal APIs with fetch() | Functions | Functions quickstart |
| A paused task that resumes where it stopped | The task’s messages and pending action, stored under the task’s ID until a person decides | KV Store | KV Store API |
| A record of every task, its steps, and its approvals | A task_events table written through the Azion API | SQL Database | Write rows to SQL Database from a function |
| Rules that run the function on the agent’s paths | The Run Function behavior, which requires Application Accelerator on the application | Application Accelerator | Run a function on one path, and roll it back |
The model comes from the third-party provider. A team that runs the model on AI Inference instead calls a model whose Tool calling capability is Yes in AI models, with the same tools array.
Reference architecture
This page builds the Tool-calling agent over third-party LLMs: every step of the loop calls the provider’s API, and the tools run in the function.
Read the diagram from the function outward. The loop lives in the function, and every turn of it crosses to the provider: the arrow to the provider is taken once per step, so the provider’s latency multiplies by the number of steps, and its failures can end a task at any step. The arrow to the order service leaves Azion only when the tool is an external service. KV Store and SQL Database hold what must outlive one request: the state of a task that waits for a person, and the record of what every task did.
Dataflow
- Your application sends a task to
POST /api/agent/tasks, and the application’s rule runs the agent function. The function gives the task an ID, records it as started, and sends the task and the tool definitions to the provider with the provider key. - When the model asks for
get_order, the function calls the order service with the backend token and sends the result back to the model as the next message. - When the model asks for
refund_order, the function stores the task’s messages and the pending refund in KV Store, records the pause, and answersawaiting_approval. - A person sends the decision to
POST /api/agent/approve. The function reads the task from KV Store, runs the refund only when approved, and resumes the loop. - When the model answers with text and no tool call, the function records the task as completed and returns the answer.
- A provider call that fails ends the task with
502, and the function writes that failure to its log. A design that adds a second provider moves the task to it instead, so provider keys, latency, and fallback are decided in the function. A task that reaches six steps stops withstep_limit.
Components
- Functions: runs the agent loop and the tool execution. The loop, the step limit, the approval rule, and the fallback decision are code in the
ops-agentfunction, and the provider key is an environment variable the function reads. - third-party LLM provider: the integration that reasons at each step and chooses the tools. It is the one part Azion does not operate, so its availability, its rate limits, and its billing stay with the provider.
- LangGraph: the agent framework, a design option. It runs inside the function in place of a hand-written loop, as the LangGraph AI Agent Boilerplate does.
- KV Store: holds the step state between requests, in the
agent-statenamespace, read and written by one key per task. A paused task resumes from it, and an expiration on the key drops a task nobody decides on. - SQL Database: holds the task history in the
task_eventstable, one row per event, so completion rate, steps per task, and approvals are counted with SQL. A function opens it through a read replica, so the history is written through the Azion API. - internal APIs and SaaS tools: the integrations the tools act on, here the order service at
https://api.example.com/v1. Each tool is a call from the function, with credentials the function reads from environment variables. - application: the Platform Resource that serves the agent API. Its rules run the function on the agent’s paths.
Other designs for this use case
- Tool-calling agent on platform-hosted models: for teams that keep the whole agent on Azion. The function sends the task and the tool definitions to a tool-calling model on AI Inference instead of a provider, so reasoning and tool execution stay inside Azion and only the tool calls leave it.
Configure the agent’s stores
The agent keeps two kinds of data, and each one goes where its access pattern fits:
- A paused task goes to KV Store. The function reads and writes it by one key,
task:<task-id>, and only when a task pauses or resumes, far below the one write per second that KV Store accepts on a key. Each write setsexpirationTtlto86400, so a task nobody decides on is dropped after one day. - The task history goes to SQL Database. One row per event lets you count completions, steps, and approvals with SQL. A function opens SQL Database through a read replica, so the agent writes the rows through the Azion API, with a personal token.
To create the namespace, send its name to the KV Store API:
The API answers 201 with the namespace. A namespace cannot be renamed or deleted, so check the name before you send it:
To create the database, send its name to the SQL Database API:
The API answers 202 with the database in data. Keep its id, and send GET /v4/workspace/sql/databases/<database-id> until status reads created, which takes about 15 seconds. Then create the table:
The API answers 200 with "state": "executed". A statement that fails still answers 200, with error in place of results in its entry, so read the entry before you continue.
The account holds the agent-state namespace and the agent-history database with an empty task_events table.
Configure the agent function
The agent function runs the loop on /api/agent/tasks and resumes it on /api/agent/approve. The decisions it applies:
- The loop stops at six steps. A step is one provider call plus the tools it asks for. Six steps bound the time one task holds a request, and they keep an invocation well inside the 50 outbound
fetch()calls a function may make. - A tool result goes back as a
usermessage. The message readsResult of <tool>: <JSON>, and the model’s tool request goes back as anassistantmessage naming the tools it called. Both use only thesystem,user, andassistantroles that Model invocation documents for the chat format. refund_orderpauses the task. A tool inAPPROVAL_TOOLSnever runs from the loop. The function stores the task and returns the action for a person to decide, and the approval path runs it only when the decision isapproved.- The approval path checks a secret, and a pending action runs once. The function clears the pending action in KV Store before it runs the tool, so a second approval of the same task answers
404. - Every task ID is validated. The ID is a UUID the function generated, and the approval path refuses any other value before the ID reaches a SQL statement.
- Keys stay out of the code. The provider key, the backend token, the approver secret, and the personal token are environment variables.
- Every event is one history row.
record()writes it as Write rows to SQL Database from a function describes, withSQL_DATABASE_IDandAZION_TOKEN. A failed write is logged ashistory_write_failedand does not stop the task.
To store the values the function reads, run these commands with the Azion CLI. A key that contains key, token, or secret is stored as a secret by default:
Create a function named ops-agent with this code. The provider call sends the key as Authorization: Bearer; change that header to the one your provider requires:
Run the function with these values, following Functions quickstart:
- Function instance:
ops-agent, with no Args. - Rule: a Request Phase rule created as Run a function on one path, and roll it back describes, with these values: the name
agent - tasks and approvals, one criteria group with${uri}starts with/api/agent/in place of the guide’s path and method groups, and the Run Function behavior selecting theops-agentinstance. Both agent paths takePOST, and the function answers any other method with405.
A task sent to /api/agent/tasks runs until the model answers, a refund needs approval, or six steps pass, and every start, pause, decision, and end lands in task_events. A new rule takes a few minutes to propagate.
Verify the setup
-
A read-only task completes. Send a task that needs only
get_order:ShellThe response carries
"status":"completed", ananswerthat states the order’s status, andstepsof 2 or more: one step that callsget_order, and one that answers. -
A refund waits for a person. Send
{"task":"Refund order <order-id>, the customer received a damaged item."}to the same path. The response carries"status":"awaiting_approval", thetask_id, and anactionnamingrefund_orderwith the order ID, and the order service has received no refund. -
An approval resumes the task once. Approve the refund with the
task_idfrom that response:ShellThe response carries
"status":"completed", and the order service has received one refund. The same request sent again answers404withno pending action for this task, and a request without theAuthorizationheader answers401. -
Every task is recorded. Read the events of the refund task through the SQL Database API, with the statement
SELECT event, steps FROM task_events WHERE task_id = '<task-id>' ORDER BY rowid;. Theresultsrows liststarted,awaiting_approval,approved, andcompleted, in that order.
When a request answers with your application’s own page, the rule may still be propagating. When a task answers failed, read the task_failed line under the Functions Console data source of Real-Time Events.
Measuring results
| Metric | Where to read it | What working looks like |
|---|---|---|
| Task completion rate | SELECT COUNT(DISTINCT task_id) FROM task_events WHERE event = 'completed'; over the same count for started, through the SQL Database API | Most tasks complete. A rise in step_limit or failed points at a prompt, a tool, or the provider |
| Steps per task | The steps of the completed events in task_events | Stable per kind of task. A task that keeps reaching six steps needs a better tool description or a narrower task |
| Latency per task | The Request Time of requests to /api/agent/tasks, in the HTTP Requests data source of Real-Time Events | Grows with steps, and holds steady for the same kind of task |
| Share of actions that needed human approval | The count of awaiting_approval events over the count of started events | Matches the share of tasks that ask for a refund, and rejected events stay rare |
Best practices
- Put every action that changes data behind approval until its record is clean. The model decides when to call a tool, so a tool that moves money or deletes data runs only after a person approves it. Move a tool out of
APPROVAL_TOOLSonly whentask_eventsshows its approvals are routine. - Describe each tool for the model, not for a developer. The model picks a tool from its
descriptionand fills itsparametersfrom the schema. A description that says when to use the tool, and a schema that marks each fieldrequired, cut the steps a task takes. - Keep the step limit, and read the tasks that reach it. A loop with no limit runs until the function’s 5-minute wall-clock limit or its 50 outbound calls stop it. The
step_limitevent names the tasks to look at. - Write task history through the API, and check each entry. The SQL Database API answers
200even when a statement fails, witherrorin that statement’s entry, so the function reads every entry and logs a failed write. For the error shapes, refer to SQL Database best practices.