Stream settings

A stream is the Data Stream object that collects the logs of one data source, shapes each log line with a template, and sends the lines to one endpoint. Azion Console and the Azion API write the same object, so each table on this page names the Console label beside the API field. API fields are written as paths inside the request body, such as inputs[0].attributes.data_source.

Stream object

The JSON body below is a complete stream. It sends every Activity History event, shaped by the Activity History Collector template, to a bucket over the S3 protocol:

The three arrays carry the parts of the stream:

• inputs holds one data source.
• transform holds one render_template item and exactly one of sampling and filter_workloads. Each item has a type and attributes, so transform[sampling].attributes.rate names the attributes.rate key of the item whose type is sampling.
• outputs holds one endpoint. The API keeps the first entry and drops any other entry without an error.

The API path is /v4/workspace/stream/streams, and one stream is /v4/workspace/stream/streams/<stream-id>. Every request carries the header Authorization: Token [TOKEN VALUE], a personal token of the account. A POST answers 201 with "state": "executed" and the stored object, which adds the read-only fields id, last_editor, created, last_modified, and product_version. A PATCH changes only the keys you send: {"active": true} activates a stream and returns it with its data source, transforms, and endpoint unchanged. A PUT takes the same required keys as a create. For every operation, refer to the Azion API reference. For the token and the first request, refer to Azion API.

A saved stream has no deployment step. A change of active state takes effect after one to two minutes, and during that window the previously active stream keeps sending. An account with 3,000 workloads or more manages its streams through the API only, because the Console blocks the create and edit forms at that count.

General

The General section names the stream. The API requires name, inputs, transform, and outputs on every create.

Input

The Input section picks the data source the stream collects from. Each data source has its own variables, listed on Data sources and variables.

The Console names each data source differently from its API slug:

Activity History carries the account’s Activity History. Functions needs Functions on the account, and WAF Events needs Firewall with WAF.

Transform

The Transform section sets which workloads the stream collects from and what share of their events it sends.

The Option radio picks the scope. Each choice sends a different item:

A stream carries exactly one of the two items. Without either, the API refuses it with 32002, whatever its data source. With both, the API refuses it with 32007. The API sets no default for the fields below; the defaults are the Console’s.

Activity History events belong to the account rather than to a workload, so an Activity History stream uses sampling. On some accounts, the Console reads All Current and Future Domains and Filter Domains instead.

Saving an active stream with a sampling item deactivates every other stream on the account, and the API returns no error. A rate of 100 counts as sampling. Before saving, the Console shows After activating and saving these settings, all other Data Streams will be disabled. and adds When multiple Data Streams have different sampling rates, the system uses the lowest percentage. To run several streams at once, give each one a filter_workloads item instead of sampling.

Render Template

The Render Template section picks the template that turns each event into a log line. The Console groups the Template options under Azion’s Templates and Custom Templates, and shows the variables of the selected template in the read-only Data Set field.

Azion provides five preset templates:

Create Custom Template opens a drawer that creates a custom template on the account, listed under Custom Templates. For the variables of each preset and the custom template format, refer to Templates and payload.

Output

The Output section sets the endpoint that receives the log lines. The Console labels the field Connector, and the API carries the choice in outputs[0].type.

Status

The Status section turns the stream on or off. The list of streams shows each one as Active or Inactive.

A change of active state takes effect after one to two minutes. Activating a stream that has a sampling item deactivates every other stream on the account.

Permissions

Account permissions decide who can change a stream. For how permissions are granted, refer to Teams and permissions. View Data Stream shows the account’s streams but does not allow creating, editing, or deleting them. Creating, editing, and deleting need Edit Data Stream.

With View Data Stream only, the Console disables + Stream and every form field, and the edit page shows no Save. Six key fields are masked, with an icon that reveals the value: S3 Access Key, Secret Key, and Object Key Prefix; AWS Kinesis Data Firehose Access Key and Secret Key; and Azure Monitor Shared Key. Without the edit permission, a lock icon replaces the reveal icon.

Errors

The API refuses each request below with HTTP 400, except 10002 with 401, and creates nothing. The errors array carries the code, the title, and a source.pointer to the field; the number in a pointer is the item’s position in its array.

Two behaviors return no error. A second entry in outputs is dropped, and the stream keeps the first one. Saving an active stream with a sampling item deactivates every other stream on the account.

Related resources