Data Stream quickstart

Stream your account's Activity History to an Object Storage bucket with Data Stream, in Azion Console or the API, and confirm the first log lines arrive.

Last updated

View as Markdown Agent setup

This guide instructs you through sending your first logs with Data Stream. By the end you will have:

  • Your first stream, which sends the Activity History of your account to an Object Storage bucket.
  • An Activity History event that you produce yourself.
  • The first log lines stored as an object in the bucket, which confirms the delivery.

The guide uses Activity History because any change you make in the account produces an event. You trigger the delivery yourself, and you need no account outside Azion.

The result rests on four objects, in this order:

  1. An Object Storage bucket receives the logs, and an Object Storage credential lets Data Stream write to it. You create both before you start.
  2. The stream collects the logs of one data source, Activity History. It sends every event, with sampling at 100%. The Activity History Collector template shapes each event into one log line. The endpoint is Simple Storage Service (S3), pointed at the bucket. The Console labels the endpoint field Connector.
  3. An event: a change you make in the account, which Activity History records.
  4. An object in the bucket that holds the log lines of that event.

Each setting of the stream is described on Stream settings.

Select the interface you will use. The prerequisites and every stage below follow that choice.

Prerequisites

  • An Azion account. To create one, refer to Create an account.
  • The Edit Data Stream permission on the account, which allows creating, editing, and deleting streams. For more information, refer to Teams Permissions.
  • An Object Storage bucket. To create one, refer to Object Storage quickstart.
  • An Object Storage credential that reaches the bucket, with four capabilities: List Files, Write Files, List All Bucket Names, and List Buckets. In the API, these are listFiles, writeFiles, listAllBucketNames, and listBuckets. Without the two bucket-list capabilities, every send fails with status 503 and no object reaches the bucket. The secret key shows only once, when the credential is created. To create the credential, refer to S3 protocol compatibility credentials.

Create the stream

The stream reads the Activity History data source, sends every event, and writes the log lines to the bucket. A stream needs either sampling or a workload filter; this one keeps sampling on at 100%. To collect the logs of chosen workloads instead, refer to Stream settings.

To create the stream with the Azion API, send a POST request to the streams endpoint. Replace [TOKEN VALUE] with your personal token, <your-bucket> with the name of your bucket, and the two keys with the keys of your credential:

Shell
01curl --request POST \
02 --url https://api.azion.com/v4/workspace/stream/streams \
03 --header 'Accept: application/json' \
04 --header 'Authorization: Token [TOKEN VALUE]' \
05 --header 'Content-Type: application/json' \
06 --data {
07 "name": "activity-to-bucket",
08 "active": true,
09 "inputs": [
10 { "type": "raw_logs", "attributes": { "data_source": "activity_history" } }
11 ],
12 "transform": [
13 { "type": "sampling", "attributes": { "rate": 100 } },
14 { "type": "render_template", "attributes": { "template": 251 } }
15 ],
16 "outputs": [
17 {
18 "type": "s3",
19 "attributes": {
20 "host_url": "https://s3.us-east-005.azionstorage.net",
21 "bucket_name": "<your-bucket>",
22 "region": "us-east-005",
23 "access_key": "[ACCESS KEY]",
24 "secret_key": "[SECRET KEY]",
25 "object_key_prefix": "activity",
26 "content_type": "plain/text"
27 }
28 }
29 ]
30}

Template 251 is Activity History Collector. A 201 carries the stream:

JSON
01{
02 "state": "executed",
03 "data": {
04 "id": 12345,
05 "name": "activity-to-bucket",
06 "last_editor": "user@example.com",
07 "created": "2026-01-01T11:30:47.000000Z",
08 "last_modified": "2026-01-01T11:30:47.000000Z",
09 "product_version": "1.0",
10 "active": true,
11 "inputs": [
12 { "type": "raw_logs", "attributes": { "data_source": "activity_history" } }
13 ],
14 "transform": [
15 { "type": "sampling", "attributes": { "rate": 100 } },
16 { "type": "render_template", "attributes": { "template": 251 } }
17 ],
18 "outputs": [
19 {
20 "type": "s3",
21 "attributes": {
22 "access_key": "[ACCESS KEY]",
23 "secret_key": "[SECRET KEY]",
24 "region": "us-east-005",
25 "object_key_prefix": "activity",
26 "bucket_name": "<your-bucket>",
27 "content_type": "plain/text",
28 "host_url": "https://s3.us-east-005.azionstorage.net"
29 }
30 }
31 ]
32 }
33}

The stream exists and is active. Keep the value of id, which identifies the stream in the next stage. A request without the sampling item fails with 400 and the code 32002, Workloads Must Be Provided.

To create the stream in Azion Console:

Open the Data Stream page

Access Azion Console > Data Stream.

Select + Stream
Name the stream

In the General section, in Name, enter activity-to-bucket.

Select the data source

In the Input section, set Data Source to Activity History.

Keep every event

In the Transform section, keep Option as All Current and Future Workloads. Keep Sampling on and Sampling Rate (%) at 100, the defaults.

Select the template

In the Render Template section, set Template to Activity History Collector.

Select the endpoint

In the Output section, set Connector to Simple Storage Service (S3).

Point the endpoint at the bucket

Enter the values of your bucket and credential:

  • URL: https://s3.us-east-005.azionstorage.net
  • Bucket Name: the name of your bucket
  • Region: us-east-005
  • Access Key and Secret Key: the keys of your credential
  • Object Key Prefix: activity
  • Content Type: plain/text
Keep the stream active

In the Status section, keep Active on.

Select Save
Confirm the sampling warning

The Attention dialog warns that saving disables every other stream on the account. Select Confirm.

The Console shows Your data stream has been created, and the Data Stream list shows the stream with the status Active.

Saving the stream checks the format of each field. It does not contact the bucket, so a wrong key shows only as a failed send.

Generate an event

A stream starts to send one to two minutes after you save it as active. Wait two minutes, then make one change in the account. Each save of the stream is an Activity History event, so renaming the stream is enough.

To rename the stream with the Azion API, send a PATCH request to the stream. Replace <stream-id> with the id of the stream you created:

Shell
01curl --request PATCH \
02 --url https://api.azion.com/v4/workspace/stream/streams/<stream-id> \
03 --header 'Accept: application/json' \
04 --header 'Authorization: Token [TOKEN VALUE]' \
05 --header 'Content-Type: application/json' \
06 --data {
07 "name": "activity-to-bucket-2"
08}

A 200 carries the stream with the changed name and a later last_modified:

JSON
01{
02 "state": "executed",
03 "data": {
04 "id": 12345,
05 "name": "activity-to-bucket-2",
06 "last_editor": "user@example.com",
07 "created": "2026-01-01T11:30:47.000000Z",
08 "last_modified": "2026-01-01T12:01:39.000000Z",
09 "product_version": "1.0",
10 "active": true,
11 …
12 }
13}

Activity History records the edit as one event.

To rename the stream in Azion Console:

Open the stream

In the Data Stream list, select activity-to-bucket.

Change the name

In Name, enter activity-to-bucket-2.

Select Save
Confirm the sampling warning

If the Attention dialog opens, select Confirm.

The Console shows Your data stream has been updated. Activity History records the edit as one event.

Within about a minute of the event, Data Stream sends its log line to the bucket.

Confirm the logs arrived

Data Stream writes each send as one object. The object name is the prefix activity, a /, the date and time of the send in the YYYY/MM/DD/hh/mm/ format, and a UUID.

To list the objects in the bucket with the Azion API, send a GET request to the bucket. Replace <your-bucket> with the name of your bucket:

Shell
01curl --request GET \
02 --url https://api.azion.com/v4/workspace/storage/buckets/<your-bucket>/objects \
03 --header 'Accept: application/json' \
04 --header 'Authorization: Token [TOKEN VALUE]'

The response lists the object that the stream wrote, with its key, its time, and its size in bytes:

JSON
01{
02 "continuation_token": null,
03 "results": [
04 {
05 "key": "activity/2026/01/01/12/02/11111111-1111-1111-1111-111111111111",
06 "last_modified": "2026-01-01T12:02:03.000000Z",
07 "size": 2797,
08 "is_folder": false
09 },
10 …
11 ]
12}

The object is in the bucket, which confirms that the stream delivered the event.

To find the object in Azion Console:

Open the bucket list

Access Azion Console > Object Storage > Buckets.

Select the bucket

Select the bucket the stream sends to.

Find the object under the prefix

The object is listed under a key that starts with activity/. Each / in the key groups the object under a prefix, not a folder.

The object is in the bucket, which confirms that the stream delivered the event.

With Content Type plain/text, the object holds one Activity History record per line, as a JSON object. The keys are the ones the Activity History Collector template sets. One line, trimmed:

{"comment": "-", "user_ip": "203.0.113.10", "request_data": "…", "resource_name": "activity-to-bucket-2", …, "title": "Stream activity-to-bucket-2 was edited", "author_email": "user@example.com", …, "resource_id": "12345", …, "time": "2026-01-01T12:01:39Z", "type": "edited", …, "resource_type": "Stream"}

Your first stream delivers the Activity History of your account to the bucket. Every send, delivered or rejected, also leaves a record with its HTTP status in Real-Time Events.

Next steps

EndpointsSend the logs to your SIEM, big-data, or stream-processing platform.Data sources and variablesCollect the logs of applications, functions, or WAF, and read what each variable holds.Create a custom templateChoose the variables each log line carries.How Data Stream worksHow a stream filters, batches, and delivers each log line.