Associate workloads with a stream

Choose the workloads a Data Stream stream collects logs from, in Azion Console or with the Azion API, and confirm the stream sends them.

Last updated

View as Markdown Agent setup

You can associate workloads with a Data Stream stream from Azion Console or with the Azion API. To collect from every workload with sampling instead, refer to Configure sampling on a stream.

A stream collects the logs of the workloads associated with it and sends them to its endpoint. One stream can collect from a single workload or from several. Unlike sampling, a workload filter leaves your other streams active. For every field of the filter, refer to Stream settings.

Select your interface once. The prerequisites and every task below show only that path.

Prerequisites

  • Access to Azion Console. To sign in, refer to How to access Azion Console.
  • An account with fewer than 3,000 workloads. At 3,000 workloads or more, the Console blocks the stream forms, and you manage streams through the API only.

Create a stream for your workloads

You choose the workloads in the same form that sets the stream’s data source, template, and endpoint. The example uses the Applications data source. Activity History events belong to the account rather than to a workload, so an Activity History stream uses sampling instead.

To create the stream with the API, send a POST request with a filter_workloads item in transform. Replace [TOKEN VALUE] with your personal token, <workload-id> with the ID of your workload, and the endpoint values with your own:

Shell
01curl -X POST 'https://api.azion.com/v4/workspace/stream/streams' \
02 -H 'Content-Type: application/json' \
03 -H 'Authorization: Token [TOKEN VALUE]' \
04 -d {
05 "name": "workloads-to-http",
06 "active": true,
07 "inputs": [
08 { "type": "raw_logs", "attributes": { "data_source": "workloads" } }
09 ],
10 "transform": [
11 { "type": "filter_workloads", "attributes": { "workloads": [<workload-id>] } },
12 { "type": "render_template", "attributes": { "template": 2 } }
13 ],
14 "outputs": [
15 {
16 "type": "standard",
17 "attributes": {
18 "url": "https://logs.example.com/ingest",
19 "headers": { "Authorization": "Bearer [TOKEN]" }
20 }
21 }
22 ]
23 }

The workloads data source is Applications in the Console, and template 2 is Applications Event Collector. To collect from several workloads, list their IDs in workloads, from 1 to 600. The API answers 201 with the stored stream:

JSON
01{
02 "state": "executed",
03 "data": {
04 "id": 12345,
05 "name": "workloads-to-http",
06 "last_editor": "user@example.com",
07 "created": "2026-01-01T12:00:00.000000Z",
08 "last_modified": "2026-01-01T12:00:00.000000Z",
09 "product_version": "1.0",
10 …
11 "transform": [
12 { "type": "filter_workloads", "attributes": { "workloads": [<workload-id>] } },
13 { "type": "render_template", "attributes": { "template": 2 } }
14 ],
15 …
16 }
17}

The transform array holds the workload filter you sent. An ID that is not a workload on your account is refused with 400:

{"errors":[{"code":"32003","title":"Workloads Not Belong Account","detail":"Workloads [1] do not belong to the account.","status":"400","source":{"pointer":"/data/transform/0/attributes/workloads"},"meta":{"invalid_workloads":[1]}}]}

The API has no option for every workload. A stream without filter_workloads needs a sampling item, and saving it deactivates every other stream on the account. For that path, refer to Configure sampling on a stream.

To create the stream in Azion Console:

Open Data Stream

Access Azion Console > Data Stream.

Select + Stream
Name the stream

In the General section, enter a Name. For example: workloads-to-http.

Select the data source

In the Input section, select Applications in Data Source. For the other data sources, refer to Data sources and variables.

Turn off Sampling

In the Transform section, Option starts at All Current and Future Workloads, with Sampling on. Turn off Sampling before you change Option, so the stream carries no sampling.

All Current and Future Workloads covers every workload on the account, including the ones you create later. A stream saved with Sampling on deactivates every other stream on the account.

Choose Filter Workloads

Set Option to Filter Workloads. The Workloads pick list opens. On some accounts, the values read All Current and Future Domains and Filter Domains.

Move the workloads to Chosen Workload

Use the controls of the pick list:

  • To find a workload in Available Workload, enter characters in the search box, or scroll through the list.
  • To move a workload to Chosen Workload, select it and then select the > arrow.
  • To move several at once, select them with ctrl on Windows and Linux or command on Mac, and then select >.
  • To move every workload the list shows, select >>.
  • To remove workloads from Chosen Workload, select them and then select <. To remove all of them, select <<.
Select the template

In the Render Template section, select Applications Event Collector in Template. For the other templates, refer to Templates and payload.

Set the endpoint

In the Output section, select your endpoint in the field labeled Connector, and fill in its fields. For the fields of each endpoint, refer to Endpoints.

Keep the stream active

In the Status section, keep Active turned on.

Select Save
Cancel the sampling warning

If the Attention dialog opens, Sampling is still on. Select Cancel, set Option back to All Current and Future Workloads, and turn off Sampling. Set Option to Filter Workloads again, make sure your workloads are in Chosen Workload, and select Save.

The Console shows Your data stream has been created, and the Data Stream list shows the stream with the status Active.

The stream saves without contacting the endpoint, and an activation takes effect after one to two minutes.

Confirm the delivery

Real-Time Events records every send of a stream, delivered or not, with the status code the endpoint returned. Send a few requests to a chosen workload, then wait about a minute.

To read the sends with the API, query the dataStreamedEvents dataset of the Real-Time Events GraphQL API. Replace the dates with a range that covers the activation of the stream:

Shell
01curl -X POST 'https://api.azion.com/v4/events/graphql' \
02 -H 'Content-Type: application/json' \
03 -H 'Authorization: Token [TOKEN VALUE]' \
04 -d '{"query":"query { dataStreamedEvents(limit: 20, filter: {tsRange: {begin: \"2026-10-02T19:45:00\", end: \"2026-10-02T20:30:00\"}}, orderBy: [ts_DESC]) { ts endpointType statusCode streamedLines dataStreamed } }"}'

The API answers 200 with one record for each send, the latest first:

JSON
01{
02 "data": {
03 "dataStreamedEvents": [
04 {
05 "ts": "2026-10-02T19:47:04Z",
06 "endpointType": "S3",
07 "statusCode": 200,
08 "streamedLines": 2,
09 "dataStreamed": 2797
10 }
11 ]
12 }
13}

A statusCode of 200 means the endpoint accepted the batch, and streamedLines counts the log lines it carried. An empty dataStreamedEvents list means the stream has not sent in the range.

To find the sends in Azion Console:

Open Real-Time Events

Access Azion Console > Real-Time Events.

Select the Data Stream data source
Read the latest sends

Each row is one send. Read its Endpoint Type, Status Code, and Streamed Lines.

A Status Code of 200 means the endpoint accepted the batch.

A status other than 200 is the answer of the endpoint, and 503 means Data Stream found the endpoint unavailable. For the causes, refer to Troubleshoot Data Stream.

Next steps

Stream settingsLook up the workload filter and sampling fields, with their bounds and errors.Configure sampling on a streamCollect a share of the logs of every workload, including future ones.Edit, stop, or delete a streamChange the chosen workloads of a stream, pause it, or remove it.Data Stream quickstartCreate a first stream and confirm that its logs reach the endpoint.