Answer questions from a vector table

Answer a question from a function with the nearest passages of a SQL Database vector table and a chat model on AI Inference that cites them.

Last updated

View as Markdown Agent setup

You answer a question from a function: it embeds the question with a model on AI Inference, retrieves the nearest passages of a SQL Database vector table with vector_top_k, and asks a chat model to answer from them and cite them. The passages come from documents in an Object Storage bucket, which a second function on the same application ingests. For the general ingestion pattern, refer to Embed documents into a vector table with AI Inference.

Prerequisites

  • An application and a workload that serve your domain, with Application Accelerator turned on, which the Run Function behavior requires. To create them, refer to Applications quickstart.
  • SQL Database enabled on the account. The product is in Preview and is not enabled by default, so request access through Technical Support.
  • A passages table with a passages_idx vector index for 1,024-dimension vectors. To create them, refer to Create the vector table.
  • An Object Storage bucket with workloads_access set to read_only, holding the documents as UTF-8 text or Markdown files. To create the bucket and upload the files, refer to Object Storage quickstart.
  • A personal token with the Edit SQL Database permission, for the database calls. To create one, refer to Personal tokens.
  • The Azion CLI, installed and authorized, to store the environment variables.

The examples use support-docs for the bucket, support-kb for the database, returns-policy.md for one document in it, /admin/ingest and /api/ask for the two paths, and www.example.com for the domain. Replace them with your values.

Ingest the documents from a bucket

The ingestion function turns one document into rows of passages: it reads the document from the bucket, splits it into passages of up to 1,500 characters, embeds all of them in one call, and writes them through the Azion API, because the runtime connection to a database is read-only. The route writes to the database, so it refuses a request without the ingestion secret.

To store the four values the function reads, run these commands with the Azion CLI. A key that contains token or secret is stored as a secret by default:

Shell
01azion create variables --key "AZION_TOKEN" --value "[TOKEN VALUE]"
02azion create variables --key "INGEST_SECRET" --value "<ingest-secret>"
03azion create variables --key "SQL_DATABASE_ID" --value "<database-id>" --secret false
04azion create variables --key "DOCS_BUCKET" --value "support-docs" --secret false

Create the variables before the function: a function that is already running does not read a changed value until it is redeployed.

Create a function named support-ingest with this code:

JavaScript
01import Storage from "azion:storage";
02
03const EMBEDDING_MODEL = "Qwen/Qwen3-Embedding-4B";
04const DIMENSIONS = 1024;
05const MAX_PASSAGE = 1500;
06const MAX_PASSAGES = 99;
07
08function split(text) {
09 const passages = [];
10 let current = "";
11 for (const paragraph of text.split(/\n\s*\n/)) {
12 const p = paragraph.trim();
13 if (!p) continue;
14 if (current && current.length + p.length > MAX_PASSAGE) {
15 passages.push(current);
16 current = "";
17 }
18 current = current ? current + "\n\n" + p : p;
19 }
20 if (current) passages.push(current);
21 return passages;
22}
23
24const quote = (value) => "'" + value.replaceAll("'", "''") + "'";
25
26export default {
27 async fetch(request, env, ctx) {
28 if (request.method !== "POST") {
29 return new Response("Method not allowed", { status: 405 });
30 }
31 if (request.headers.get("Authorization") !== `Bearer ${Azion.env.get("INGEST_SECRET")}`) {
32 return new Response("Unauthorized", { status: 401 });
33 }
34 const key = new URL(request.url).searchParams.get("key");
35 if (!key) {
36 return Response.json({ error: "key is required" }, { status: 400 });
37 }
38
39 const object = await new Storage(Azion.env.get("DOCS_BUCKET")).get(key);
40 const passages = split(new TextDecoder().decode(await object.arrayBuffer()));
41 if (passages.length === 0 || passages.length > MAX_PASSAGES) {
42 return Response.json({ error: `${passages.length} passages; split the document` }, { status: 413 });
43 }
44
45 const embedded = await Azion.AI.run(EMBEDDING_MODEL, {
46 "input": passages,
47 "encoding_format": "float",
48 "dimensions": DIMENSIONS
49 });
50
51 const statements = [`DELETE FROM passages WHERE source = ${quote(key)};`];
52 for (const item of embedded.data) {
53 statements.push(
54 `INSERT INTO passages (source, content, embedding) VALUES (${quote(key)}, ${quote(passages[item.index])}, vector('[${item.embedding.join(",")}]'));`
55 );
56 }
57
58 const result = await fetch(
59 `https://api.azion.com/v4/workspace/sql/databases/${Azion.env.get("SQL_DATABASE_ID")}/query`,
60 {
61 method: "POST",
62 headers: {
63 "Accept": "application/json",
64 "Authorization": `Token ${Azion.env.get("AZION_TOKEN")}`,
65 "Content-Type": "application/json"
66 },
67 body: JSON.stringify({ statements })
68 }
69 );
70 const body = await result.json();
71 const failed = (body.data ?? []).find((entry) => entry.error);
72 if (!result.ok || failed) {
73 return Response.json({ error: failed?.error ?? `SQL API answered ${result.status}` }, { status: 502 });
74 }
75 return Response.json({ key, passages: passages.length });
76 },
77};

Run the function on its path with these values, following Functions quickstart:

  • Function instance: support-ingest, with no Args.
  • Rule: a Request Phase rule named support - ingest, with the criterion ${uri} starts with /admin/ingest and the Run Function behavior selecting the support-ingest instance.

To create the rule in Azion Console or with the API, refer to Add the rule that runs the function.

A POST to /admin/ingest with the secret and a document key replaces that document’s passages in passages, and answers with the number it stored. The DELETE makes a second ingestion of the same key replace its rows instead of duplicating them.

To ingest the example document:

Shell
01curl -X POST 'https://www.example.com/admin/ingest?key=returns-policy.md' \
02 -H 'Authorization: Bearer <ingest-secret>'

The function answers with the key and the number of passages it stored, such as {"key":"returns-policy.md","passages":3}. A request without the Authorization header answers 401. To count the rows of the document through the SQL Database API:

Shell
01curl --request POST \
02 --url https://api.azion.com/v4/workspace/sql/databases/<database-id>/query \
03 --header 'Accept: application/json' \
04 --header 'Authorization: Token [TOKEN VALUE]' \
05 --header 'Content-Type: application/json' \
06 --data '{"statements":["SELECT COUNT(*) FROM passages WHERE source = '\''returns-policy.md'\'' AND embedding IS NOT NULL;"]}'

The single entry in data carries results, whose one row holds the same number the ingestion returned.

The Build and run customer support AI assistants use case uses the values of this example.

These checks confirm the Build and run customer support AI assistants use case.

Store the database name

The function opens the database by name, through a read replica.

To store the database name the function opens, run:

azion create variables --key "SQL_DATABASE_NAME" --value "support-kb" --secret false

The account holds the SQL_DATABASE_NAME variable that the function reads with Azion.env.get().

The Build and run customer support AI assistants use case uses the values of this example.

Create the assistant function

The function embeds the question with the same model and width as the passages, retrieves the four nearest passages, and sends them with the last six messages of history and the question to the chat model. It writes one "event":"answer" log line per answer, recording whether the answer cites a passage.

Create a function named support-assistant with this code:

JavaScript
01const EMBEDDING_MODEL = "Qwen/Qwen3-Embedding-4B";
02const CHAT_MODEL = "Qwen/Qwen3-30B-A3B-Instruct-2507-FP8";
03const DIMENSIONS = 1024;
04const TOP_K = 4;
05
06const SYSTEM_PROMPT =
07 "You answer customer questions using only the numbered passages in the user message. " +
08 "Cite every passage you use as [n]. If the passages do not contain the answer, " +
09 "say that you do not know and suggest contacting support.";
10
11async function retrieve(question) {
12 const embedded = await Azion.AI.run(EMBEDDING_MODEL, {
13 "input": question,
14 "encoding_format": "float",
15 "dimensions": DIMENSIONS
16 });
17 const vector = embedded.data[0].embedding.join(",");
18
19 const { Database } = Azion.Sql;
20 const connection = await Database.open(Azion.env.get("SQL_DATABASE_NAME"));
21 const rows = await connection.query(
22 `SELECT passages.source, passages.content FROM vector_top_k('passages_idx', vector('[${vector}]'), ${TOP_K}) JOIN passages ON passages.rowid = id;`
23 );
24
25 const passages = [];
26 let row = await rows.next();
27 while (row) {
28 passages.push({ source: row.getString(0), content: row.getString(1) });
29 row = await rows.next();
30 }
31 return passages;
32}
33
34export default {
35 async fetch(request, env, ctx) {
36 if (request.method !== "POST") {
37 return new Response("Method not allowed", { status: 405 });
38 }
39 const { question, history = [] } = await request.json();
40 if (typeof question !== "string" || question.trim() === "") {
41 return Response.json({ error: "question is required" }, { status: 400 });
42 }
43
44 const passages = await retrieve(question);
45 const context = passages
46 .map((p, i) => `[${i + 1}] (${p.source})\n${p.content}`)
47 .join("\n\n");
48
49 const modelResponse = await Azion.AI.run(CHAT_MODEL, {
50 "stream": false,
51 "max_tokens": 800,
52 "messages": [
53 { "role": "system", "content": SYSTEM_PROMPT },
54 ...history.slice(-6),
55 { "role": "user", "content": `Passages:\n${context}\n\nQuestion: ${question}` }
56 ]
57 });
58
59 const answer = modelResponse?.choices?.[0]?.message?.content ?? "";
60 const cited = /\[\d+\]/.test(answer);
61 console.log(JSON.stringify({ event: "answer", cited, passages: passages.length }));
62
63 return Response.json({
64 answer,
65 sources: passages.map((p, i) => ({ n: i + 1, source: p.source }))
66 });
67 },
68};

To create the function and its instance, follow Functions quickstart with the name support-assistant, and name the instance support-assistant, with no Args.

The application carries a support-assistant instance that answers a question from the passages nearest to it.

The Build and run customer support AI assistants use case uses the values of this example.

Run the function on the path

Create a Request Phase rule as Add the rule that runs the function shows, with the name support - ask, the criterion ${uri} is equal /api/ask, and the Run Function behavior selecting the support-assistant instance.

A POST to /api/ask with a question answers with the model’s answer, which cites passages as [n], and with the document each cited passage came from. A new rule takes a few minutes to propagate.

Confirm the answers cite the documents

To ask a question the document answers:

Shell
01curl -X POST https://www.example.com/api/ask \
02 -H 'Content-Type: application/json' \
03 -d '{"question":"How many days do I have to return an item?"}'

The response carries answer, with at least one [n] citation, and sources, whose entries name returns-policy.md.

Ask a question no document covers, such as What is the capital of France?. The answer says that the assistant does not know, and it carries no citation.

The function answers from the passages of the table, and only from them.

These checks confirm the Build and run customer support AI assistants use case.

Next steps

Vector searchThe vector_top_k function, the distance functions, and the bounds of a vector index.Build and run customer support AI assistantsThe design this function serves: the passage size, the number retrieved, and the answer cap, with the reason for each.