Embed documents into a vector table with AI Inference

Turn the documents in a bucket into passages with vectors from an AI Inference embedding model, stored in a SQL Database vector table.

Last updated

View as Markdown Agent setup

You turn a document stored in Object Storage into passages, embed each passage with a model on AI Inference, and store the passages and their vectors in a SQL Database table, from a function and the Azion API. To query the table by meaning once it holds vectors, refer to Vector search.

A retrieval step finds passages by comparing vectors, so every passage needs one, produced by the same model and at the same width as the vector of each later question. The function reads one document per request, embeds all of its passages in one model call, and writes them in one call to the Azion API.

  1. A request names the object key of one document, and the function reads that document from the bucket.
  2. The function splits the text into passages.
  3. One call to the embedding model returns a vector for every passage.
  4. The function builds a DELETE for the document’s earlier rows and one INSERT per passage, and sends them to the query endpoint of the Azion API.
  5. The rows land in the vector table, and the vector index covers them.

Prerequisites

  • SQL Database enabled on your account. The product is in Preview and is not enabled by default, so request access through Technical Support.
  • A database, its identifier, and a personal token, stored as the SQL_DATABASE_ID and SQL_TOKEN environment variables, as Write rows to SQL Database from a function describes. A function writes a database only through the Azion API, because its own connection is read-only.
  • A bucket that holds the documents as UTF-8 text or Markdown files. To create one and upload the files, refer to Object Storage quickstart.
  • The Azion CLI installed and authorized, and a function project to add the code to. To create and deploy one, refer to Deploy a function with Azion CLI.

The examples read the bucket my-bucket, embed with Qwen/Qwen3-Embedding-4B at 1,024 dimensions, write to a table named passages, and answer on www.example.com. Replace them, <database-id>, <personal-token>, and <object-key> with your own values.

Create the vector table

The vector column declares the width of the vectors it holds, and the embedding request asks for the same width. Qwen/Qwen3-Embedding-4B returns one of five widths, 256, 512, 1024, 2048, or 4096, through the dimensions field, so a F32_BLOB(1024) column matches a request for 1024. A vector index needs a table with a ROWID or a single-column primary key, and its second argument sets the distance metric.

To create the table and its index, send both statements to the query endpoint:

Shell
01curl --request POST \
02 --url https://api.azion.com/v4/workspace/sql/databases/<database-id>/query \
03 --header 'Accept: application/json' \
04 --header 'Authorization: Token <personal-token>' \
05 --header 'Content-Type: application/json' \
06 --data {"statements":[
07 "CREATE TABLE passages (id INTEGER PRIMARY KEY AUTOINCREMENT, source TEXT NOT NULL, content TEXT NOT NULL, embedding F32_BLOB(1024));",
08 "CREATE INDEX passages_idx ON passages (libsql_vector_idx(embedding, '\''metric=cosine'\''));"
09 ]}

The API answers 200 with "state": "executed" and one entry per statement. A statement that fails still answers 200, with error in place of results in its entry, so read both entries:

{"state":"executed","data":[{"results":{"columns":[],"rows":[],...}},{"results":{"columns":[],"rows":[],...}}]}

The database holds an empty passages table and the passages_idx index. The index adds a shadow table, passages_idx_shadow, that a table listing shows beside passages.

Embed a document into the table

The function reads the document with the Storage class of the azion:storage module, which takes the bucket name and no token. It sends every passage in one input array, and the model answers with one entry per input in data, where data[].index is the position of the passage and data[].embedding its vector. Each vector goes into its statement as text inside vector('[...]').

The route writes to the database, so it refuses a request that does not carry a secret. To store that secret with the Azion CLI:

azion create variables --key INGEST_SECRET --value <ingest-secret> --secret true

The command prints the UUID of the variable it created:

Created variable with UUID 00000000-0000-0000-0000-000000000005

To embed one document per request, use this code as the function’s entrypoint:

JavaScript
01import Storage from 'azion:storage';
02
03const BUCKET = 'my-bucket';
04const EMBEDDING_MODEL = 'Qwen/Qwen3-Embedding-4B';
05const DIMENSIONS = 1024;
06const MAX_PASSAGE = 1500;
07const MAX_PASSAGES = 99;
08
09// A passage is a run of paragraphs up to MAX_PASSAGE characters.
10function split(text) {
11 const passages = [];
12 let current = '';
13 for (const paragraph of text.split(/\n\s*\n/)) {
14 const p = paragraph.trim();
15 if (!p) continue;
16 if (current && current.length + p.length > MAX_PASSAGE) {
17 passages.push(current);
18 current = '';
19 }
20 current = current ? `${current}\n\n${p}` : p;
21 }
22 if (current) passages.push(current);
23 return passages;
24}
25
26function sqlText(value) {
27 return `'${String(value).replaceAll("'", "''")}'`;
28}
29
30async function writeRows(statements) {
31 const response = await fetch(
32 `https://api.azion.com/v4/workspace/sql/databases/${Azion.env.get('SQL_DATABASE_ID')}/query`,
33 {
34 method: 'POST',
35 headers: {
36 Accept: 'application/json',
37 Authorization: `Token ${Azion.env.get('SQL_TOKEN')}`,
38 'Content-Type': 'application/json',
39 },
40 body: JSON.stringify({ statements }),
41 },
42 );
43 if (!response.ok) {
44 throw new Error(`Azion API answered ${response.status}`);
45 }
46 const failed = (await response.json()).data.find((entry) => entry.error);
47 if (failed) {
48 throw new Error(failed.error);
49 }
50}
51
52export default {
53 async fetch(request, env, ctx) {
54 if (request.method !== 'POST') {
55 return new Response('Method not allowed', { status: 405 });
56 }
57 if (request.headers.get('Authorization') !== `Bearer ${Azion.env.get('INGEST_SECRET')}`) {
58 return new Response('Unauthorized', { status: 401 });
59 }
60 const key = new URL(request.url).searchParams.get('key');
61 if (!key) {
62 return Response.json({ error: 'key is required' }, { status: 400 });
63 }
64
65 let text;
66 try {
67 const object = await new Storage(BUCKET).get(key);
68 text = new TextDecoder().decode(await object.arrayBuffer());
69 } catch (error) {
70 return Response.json({ error: String(error) }, { status: 404 });
71 }
72 const passages = split(text);
73 if (passages.length === 0 || passages.length > MAX_PASSAGES) {
74 return Response.json({ error: `${passages.length} passages; split the document` }, { status: 413 });
75 }
76
77 const embedded = await Azion.AI.run(EMBEDDING_MODEL, {
78 input: passages,
79 encoding_format: 'float',
80 dimensions: DIMENSIONS,
81 });
82
83 // The DELETE makes a second run for the same key replace its rows instead of adding to them.
84 const statements = [`DELETE FROM passages WHERE source = ${sqlText(key)};`];
85 for (const item of embedded.data) {
86 statements.push(
87 `INSERT INTO passages (source, content, embedding) VALUES (${sqlText(key)}, ${sqlText(passages[item.index])}, vector('[${item.embedding.join(',')}]'));`,
88 );
89 }
90 try {
91 await writeRows(statements);
92 } catch (error) {
93 console.log(error.message);
94 return Response.json({ error: 'The passages were not stored' }, { status: 502 });
95 }
96 return Response.json({ key, passages: passages.length });
97 },
98};

The code applies four decisions:

  • One document per request. One read, one model call, and one write bound the work of each invocation. A function may use 2 seconds of CPU time and 50 outbound fetch() calls per invocation.
  • At most 99 passages per document. The DELETE and 99 INSERT statements make 100 statements, and a call carrying 100 statements succeeds. A longer document answers 413, so split it into two files.
  • The width is one constant. DIMENSIONS sets the request, and it must equal the F32_BLOB width of the column. A question is comparable with a passage only when the same model produced both vectors at the same width.
  • A missing key answers 404. get throws StorageError: Object not found for a key that holds no object, and Bucket not found for a bucket the account does not have.

Under azion dev, Azion.AI is undefined and azion:storage reads the local disk, so deploy the function, as Deploy a function with Azion CLI shows, and test it there. Then send one document:

Shell
01curl -X POST 'https://www.example.com/?key=<object-key>' \
02 -H 'Authorization: Bearer <ingest-secret>'

The function answers with the key and the number of passages it stored, such as {"key":"<object-key>","passages":3}. A request without the Authorization header answers 401. The table holds one row per passage of the document, each with its vector.

Confirm the passages carry vectors

To count the rows of the document that hold a vector, send a SELECT to the query endpoint:

Shell
01curl --request POST \
02 --url https://api.azion.com/v4/workspace/sql/databases/<database-id>/query \
03 --header 'Accept: application/json' \
04 --header 'Authorization: Token <personal-token>' \
05 --header 'Content-Type: application/json' \
06 --data '{"statements":["SELECT COUNT(*) AS passages FROM passages WHERE source = '\''<object-key>'\'' AND embedding IS NOT NULL;"]}'

The single entry in data carries results, whose one row holds the number the function returned:

{"state":"executed","data":[{"results":{"columns":["passages"],"rows":[[3]],...}}]}

Every passage of the document carries a vector, and vector_top_k on passages_idx can return it. A query vector must come from Qwen/Qwen3-Embedding-4B at 1,024 dimensions too.

Next steps

Vector searchQuery the passages with vector_top_k, and read the distance functions and index bounds.Qwen3 Embedding 4BThe model id, the widths it returns, and its context length.Write rows to SQL Database from a functionStore the database credentials and check the result of every statement a function sends.Build and run customer support AI assistantsAn assistant that ingests a documentation bucket this way and answers questions from the passages.