--- name: azion-send-data-stream-data-to-object-storage description: >- Point a Data Stream stream at an Object Storage bucket through the S3 endpoint, in Azion Console or with the Azion API, and confirm the delivery. --- # Send Data Stream data to Object Storage You can send the logs of a stream to an [Object Storage](/en/documentation/platform/object-storage/) bucket from Azion Console or with the Azion API. To create a stream that writes to a bucket from the start, refer to the [Data Stream quickstart](/en/documentation/platform/data-stream/quickstart/). To send the logs to an Amazon S3 bucket, refer to [Send logs to Amazon S3](/en/documentation/guides/platform/observability/endpoint-amazon-s3/). [Data Stream](/en/documentation/platform/data-stream/) reaches the bucket through the S3 endpoint and writes each batch of log lines as one object. In the stream form, the endpoint is set in the field labeled **Connector**, and its option is *Simple Storage Service (S3)*. For every field and its bounds, refer to [Endpoints](/en/documentation/platform/data-stream/endpoints/#azion-object-storage). --- Select your interface once. The prerequisites and every task below show only that path. ## Prerequisites - The **Edit Data Stream** permission. For the permissions, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#permissions). - A stream on the account. To create one, refer to the [Data Stream quickstart](/en/documentation/platform/data-stream/quickstart/). - An Object Storage bucket. To create one, refer to the [Object Storage quickstart](/en/documentation/platform/object-storage/quickstart/). - An Object Storage credential that reaches the bucket, with four capabilities: **List Files**, **Write Files**, **List All Bucket Names**, and **List Buckets**. In the API, these are `listFiles`, `writeFiles`, `listAllBucketNames`, and `listBuckets`. Without the two bucket-list capabilities, every send fails with status `503`, and no object reaches the bucket. The secret key shows only once, when the credential is created. To create the credential, refer to [S3 protocol compatibility credentials](/en/documentation/platform/object-storage/s3-compatibility/#credentials). **Console** - Access to Azion Console. To sign in, refer to [How to access Azion Console](/en/documentation/guides/platform/account-and-billing/how-to-access-azion-console/). **API** - A personal token. To create one, refer to [How to manage a personal token](/en/documentation/guides/platform/account-and-billing/personal-tokens/). - `curl`. - The ID of the stream. The list response of `GET /v4/workspace/stream/streams` carries it in the `id` field of each stream. --- ## Point the stream at the bucket The stream keeps its data source, its filters, and its template. Only the endpoint changes, with the bucket's S3 address, its region, and the key pair of the credential. **Console** To point the stream at the bucket in Azion Console: 1. **Open Data Stream** Access [Azion Console](https://console.azion.com/) > **Data Stream**. 2. **Open the stream** In the list, select the row of the stream. 3. **Select the S3 endpoint** In the **Output** section, select *Simple Storage Service (S3)* in **Connector**. 4. **Enter the bucket location** Enter the values of your bucket: - **URL**: `https://s3.us-east-005.azionstorage.net` - **Bucket Name**: the name of your bucket - **Region**: `us-east-005` 5. **Enter the credential** Enter the access key in **Access Key** and the secret key in **Secret Key**. The Console masks both fields. On an account with **View Data Stream** only, a lock icon replaces the reveal icon. On an edit, the Console does not require **Secret Key**. 6. **(Optional) Enter the object prefix** In **Object Key Prefix**, enter the start of each object name. For example: `activity`. Without a prefix, the objects go to the root of the bucket. 7. **Select the content type** In **Content Type**, select *plain/text* or *application/gzip*. With *plain/text*, each object holds one log line per line. 8. **Keep the stream active** In the **Status** section, keep **Active** turned on. 9. **Select Save** 10. **Confirm the sampling warning** If the **Attention** dialog opens, select **Confirm**. The Console shows `Your data stream has been updated`. In the **Data Stream** list, the **Connector** column shows `Amazon S3` for the stream. **API** To point the stream at the bucket with the API, send a `PATCH` request with the `outputs` key only. Replace `` with the ID of your stream, `` with the name of your bucket, and the two keys with the keys of your credential: ```bash curl -X PATCH 'https://api.azion.com/v4/workspace/stream/streams/' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{ "outputs": [ { "type": "s3", "attributes": { "host_url": "https://s3.us-east-005.azionstorage.net", "bucket_name": "", "region": "us-east-005", "access_key": "[ACCESS KEY]", "secret_key": "[SECRET KEY]", "object_key_prefix": "activity", "content_type": "plain/text" } } ] }' ``` The API answers `200` with the stored stream: ```json { "state": "executed", "data": { "id": 12345, "name": "activity-to-bucket", "last_editor": "user@example.com", "created": "2026-01-01T11:30:47.000000Z", "last_modified": "2026-01-01T11:58:54.000000Z", "product_version": "1.0", … "outputs": [ { "type": "s3", "attributes": { "access_key": "[ACCESS KEY]", "secret_key": "[SECRET KEY]", "region": "us-east-005", "object_key_prefix": "activity", "bucket_name": "", "content_type": "plain/text", "host_url": "https://s3.us-east-005.azionstorage.net" } } ] } } ``` The `outputs` entry holds the bucket, and `last_modified` moves forward. The data source and the transforms stay as they were. For every key of the body, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#stream-object). Saving checks the format of each field and does not contact the bucket. A wrong key or a missing capability shows only as a failed send. An activation takes effect after one to two minutes. If the stream uses sampling, saving it as active deactivates every other stream on the account; a workload filter leaves them active. --- ## Confirm the delivery A stream sends a batch every 60 seconds, or sooner when it reaches 2,000 log lines. Two checks confirm the delivery: the object in the bucket, and the send that Real-Time Events records. ### Find the object in the bucket Each object name is the **Object Key Prefix**, a `/`, the date and time of the send in the `YYYY/MM/DD/hh/mm/` format, and a UUID. Data Stream adds the `/` after the prefix. **Console** To find the object in Azion Console: 1. **Open the bucket list** Access [Azion Console](https://console.azion.com/) > **Object Storage** > **Buckets**. 2. **Select the bucket** Select the bucket the stream sends to. 3. **Find the object under the prefix** With the prefix `activity`, the object is listed under a key that starts with `activity/`. The object is in the bucket, which confirms that the stream delivered a batch. **API** To list the objects in the bucket with the API, send a `GET` request to the bucket. Replace `` with the name of your bucket: ```bash curl -X GET 'https://api.azion.com/v4/workspace/storage/buckets//objects' \ -H 'Accept: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' ``` The API answers `200` with one entry per object, each with its key, its time, and its size in bytes: ```json { "continuation_token": null, "results": [ { "key": "activity/2026/01/01/12/02/11111111-1111-1111-1111-111111111111", "last_modified": "2026-01-01T12:02:03.000000Z", "size": 2797, "is_folder": false }, … ] } ``` The key starts with the prefix `activity`, followed by the date and time of the send, which confirms that the stream delivered a batch. ### Read the send in Real-Time Events [Real-Time Events](/en/documentation/platform/real-time-events/data-sources/#data-stream) records every send of a stream, delivered or not, with the status code the endpoint returned. **Console** To find the sends in Azion Console: 1. **Open Real-Time Events** Access [Azion Console](https://console.azion.com/) > **Real-Time Events**. 2. **Select the Data Stream data source** 3. **Read the latest sends** Each row is one send. Find the rows with `S3` in **Endpoint Type**, and read their **Status Code**. A **Status Code** of `200` means the bucket accepted the batch. **Streamed Lines** gives the number of log lines in the batch. **API** To read the sends with the API, query the `dataStreamedEvents` dataset of the Real-Time Events GraphQL API. Replace the dates with a range that covers the change of the stream: ```bash curl -X POST 'https://api.azion.com/v4/events/graphql' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{"query":"query { dataStreamedEvents(limit: 20, filter: {tsRange: {begin: \"2026-01-01T12:00:00\", end: \"2026-01-01T12:45:00\"}}, orderBy: [ts_DESC]) { ts endpointType statusCode streamedLines dataStreamed } }"}' ``` The API answers `200` with one record for each send, the latest first: ```json { "data": { "dataStreamedEvents": [ { "ts": "2026-01-01T12:02:04Z", "endpointType": "S3", "statusCode": 200, "streamedLines": 2, "dataStreamed": 2797 } ] } } ``` A `statusCode` of `200` means the bucket accepted the batch, and `streamedLines` and `dataStreamed` give its size in log lines and bytes. An empty `dataStreamedEvents` list means the stream has not sent in the range. For every field, refer to [Real-Time Events GraphQL fields](/en/documentation/devtools/graphql/gql-real-time-events-fields/#datastreamedevents-data-stream). An object under the prefix and a send with status `200` together confirm the delivery. A status of `503` means Data Stream found the endpoint unavailable. With Object Storage, check first that the credential carries `listAllBucketNames` and `listBuckets`. For the causes, refer to [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting/#real-time-events-shows-status-503-and-nothing-reaches-the-endpoint). --- ## Next steps - [Data Stream quickstart](/en/documentation/platform/data-stream/quickstart.md): Create a stream that writes to a bucket and confirm its first logs. - [Endpoints](/en/documentation/platform/data-stream/endpoints.md#azion-object-storage): Every field of the S3 endpoint, with its type, bounds, and API name. - [Object Storage](/en/documentation/platform/object-storage.md): Manage the bucket, its objects, and the credentials that reach it. - [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting.md): Find what to change when a send returns a status other than 200.