--- name: azion-send-logs-to-amazon-s3 description: >- Create a stream that writes the logs of your applications as objects in an Amazon S3 bucket, in Azion Console or with the Azion API, and confirm the delivery. --- # Send logs to Amazon S3 You can send the logs of a stream to an [Amazon S3](https://aws.amazon.com/s3/) bucket from Azion Console or with the Azion API. To send them to an Azion Object Storage bucket, whose credential needs other capabilities, refer to [Send Data Stream data to Object Storage](/en/documentation/guides/platform/observability/connector-azion-object-storage/). [Data Stream](/en/documentation/platform/data-stream/) writes each batch of log lines as one object in the bucket. In the stream form, the endpoint is set in the field labeled **Connector**, and Amazon S3 is its *Simple Storage Service (S3)* option. For every field and its bounds, refer to [Endpoints](/en/documentation/platform/data-stream/endpoints/#simple-storage-service-s3). The example collects the requests of one workload with the *Applications* data source. --- Select your interface once. The prerequisites and every task below show only that path. ## Prerequisites - An Azion account with the **Edit Data Stream** permission. For the permissions, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#permissions). - A [workload](/en/documentation/platform/workloads/) on the account that receives requests. - An Amazon S3 bucket. The bucket must exist before the stream sends to it. It can use server-side encryption with Amazon S3 managed keys (SSE-S3). - The region code of the bucket, such as `us-east-1`. - An AWS Identity and Access Management (IAM) credential with an access key ID and a secret access key. The credential needs the `s3:ListBucket` permission, to list the objects of the bucket, and the `s3:PutObject` permission, to write objects in it. **Console** - Access to Azion Console. To sign in, refer to [How to access Azion Console](/en/documentation/guides/platform/account-and-billing/how-to-access-azion-console/). **API** - A personal token. To create one, refer to [How to manage a personal token](/en/documentation/guides/platform/account-and-billing/personal-tokens/). - The ID of the workload. - `curl`. --- ## Create the stream The stream collects from the workload you choose, through a workload filter. Unlike sampling, a workload filter leaves your other streams active. **Console** To create the stream in Azion Console: 1. **Open Data Stream** Access [Azion Console](https://console.azion.com/) > **Data Stream**. 2. **Select + Stream** 3. **Name the stream** In the **General** section, enter a **Name**. For example: `logs-to-s3`. 4. **Select the data source** In the **Input** section, select *Applications* in **Data Source**. 5. **Turn off Sampling** In the **Transform** section, turn off **Sampling** while **Option** is still *All Current and Future Workloads*, its starting value. A stream cannot carry sampling and a workload filter together. 6. **Choose the workload** In the **Transform** section, set **Option** to *Filter Workloads*. In **Available Workload**, select your workload and move it to **Chosen Workload** with the arrow. 7. **Select the template** In the **Render Template** section, select *Applications Event Collector* in **Template**. 8. **Select the S3 endpoint** In the **Output** section, select *Simple Storage Service (S3)* in **Connector**. 9. **Enter the bucket location** Enter the S3 host of the bucket's region in **URL**, such as `https://s3.us-east-1.amazonaws.com`. Enter the bucket name in **Bucket Name** and its region code in **Region**. 10. **Enter the credential** Enter the access key ID in **Access Key** and the secret access key in **Secret Key**. The Console masks both fields. 11. **(Optional) Enter the object prefix** In **Object Key Prefix**, enter the start of each object name. For example: `azion/logs`. 12. **Select the content type** In **Content Type**, select *plain/text* or *application/gzip*. With *plain/text*, each object holds one log line per line. 13. **Keep the stream active** In the **Status** section, keep **Active** turned on. 14. **Select Save** The Console shows `Your data stream has been created`. The stream appears in the **Data Stream** list with `Amazon S3` in the **Connector** column and the **Active** status. **API** To create the stream with the API, send a `POST` request to `https://api.azion.com/v4/workspace/stream/streams`. Replace `[TOKEN VALUE]` with your personal token, `` with the ID of your workload, and the bucket values with your own: ```bash curl -X POST 'https://api.azion.com/v4/workspace/stream/streams' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{ "name": "logs-to-s3", "active": true, "inputs": [ { "type": "raw_logs", "attributes": { "data_source": "workloads" } } ], "transform": [ { "type": "filter_workloads", "attributes": { "workloads": [] } }, { "type": "render_template", "attributes": { "template": 2 } } ], "outputs": [ { "type": "s3", "attributes": { "host_url": "https://s3.us-east-1.amazonaws.com", "bucket_name": "", "region": "us-east-1", "access_key": "[ACCESS KEY]", "secret_key": "[SECRET KEY]", "object_key_prefix": "azion/logs", "content_type": "application/gzip" } } ] }' ``` The `workloads` data source is *Applications* in the Console, and template `2` is *Applications Event Collector*. The API answers `201` with the stored stream: ```json { "state": "executed", "data": { "id": 12349, "name": "logs-to-s3", "last_editor": "user@example.com", "created": "2026-01-01T12:10:26.000000Z", "last_modified": "2026-01-01T12:10:26.000000Z", "product_version": "1.0", … "outputs": [ { "type": "s3", "attributes": { "access_key": "[ACCESS KEY]", "secret_key": "[SECRET KEY]", "region": "us-east-1", "object_key_prefix": "azion/logs", "bucket_name": "", "content_type": "application/gzip", "host_url": "https://s3.us-east-1.amazonaws.com" } } ] } } ``` Keep the `id`: it identifies the stream in every later request, such as `/v4/workspace/stream/streams/12349`. For every key of the body, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#stream-object). The API and the Console save the stream without contacting the bucket. A wrong URL, region, or credential surfaces only when the stream sends. An activation takes effect after one to two minutes. --- ## Confirm the delivery [Real-Time Events](/en/documentation/platform/real-time-events/data-sources/#data-stream) records every send of a stream, delivered or not, with the status code the endpoint returned. Send a few requests to the workload, then wait about a minute: a stream sends a batch every 60 seconds, or sooner when it reaches 2,000 log lines. **Console** To find the sends in Azion Console: 1. **Open Real-Time Events** Access [Azion Console](https://console.azion.com/) > **Real-Time Events**. 2. **Select the Data Stream data source** 3. **Read the latest sends** Each row is one send. Find the rows with `S3` in **Endpoint Type**, and read their **Status Code**. A **Status Code** of `200` means the bucket accepted the batch. **Streamed Lines** gives the number of log lines in the batch. **API** To read the sends with the API, query the `dataStreamedEvents` dataset of the Real-Time Events GraphQL API. Replace the dates with a range that covers the activation of the stream: ```bash curl -X POST 'https://api.azion.com/v4/events/graphql' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{"query":"query { dataStreamedEvents(limit: 20, filter: {tsRange: {begin: \"2026-01-01T12:00:00\", end: \"2026-01-01T12:45:00\"}}, orderBy: [ts_DESC]) { ts endpointType statusCode streamedLines dataStreamed } }"}' ``` The API answers `200` with one record for each send, the latest first: ```json { "data": { "dataStreamedEvents": [ { "ts": "2026-01-01T12:02:04Z", "endpointType": "S3", "statusCode": 200, "streamedLines": 2, "dataStreamed": 2797 } ] } } ``` A `statusCode` of `200` means the bucket accepted the batch, and `streamedLines` and `dataStreamed` give its size in log lines and bytes. An empty `dataStreamedEvents` list means the stream has not sent in the range. For every field, refer to [Real-Time Events GraphQL fields](/en/documentation/devtools/graphql/gql-real-time-events-fields/#datastreamedevents-data-stream). A status other than `200` is the answer of the endpoint, and `503` means Data Stream found the endpoint unavailable. For the causes, refer to [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting/). In the bucket, each object name is the **Object Key Prefix**, a `/`, the date and time of the send in the `YYYY/MM/DD/hh/mm/` format, and a UUID. With the prefix `azion/logs`, an object name starts with a path such as `azion/logs/2026/01/01/12/02/`, followed by the UUID. --- ## Next steps - [Endpoints](/en/documentation/platform/data-stream/endpoints.md#simple-storage-service-s3): Every field of the S3 endpoint, with its type, bounds, and API name. - [Send Data Stream data to Object Storage](/en/documentation/guides/platform/observability/connector-azion-object-storage.md): Send the logs to an Azion Object Storage bucket through the same S3 endpoint. - [Edit, stop, or delete a stream](/en/documentation/guides/platform/observability/delete-data-stream.md): Change the bucket or the credential, pause the stream, or remove it. - [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting.md): Find what to change when a send returns a status other than 200.