--- name: azion-send-logs-to-aws-kinesis-data-firehose description: >- Create a stream that sends the logs of your applications to an Amazon Kinesis Data Firehose delivery stream, in Azion Console or with the Azion API. --- # Send logs to AWS Kinesis Data Firehose You can send the logs of a stream to an [Amazon Kinesis Data Firehose](https://aws.amazon.com/kinesis/data-firehose/) delivery stream from Azion Console or with the Azion API. To write the logs straight to an Amazon S3 bucket, with no delivery stream in between, refer to [Send logs to Amazon S3](/en/documentation/guides/platform/observability/endpoint-amazon-s3/). [Data Stream](/en/documentation/platform/data-stream/) sends the log lines to the delivery stream in batches. In the stream form, the endpoint is set in the field labeled **Connector**, and Kinesis is its *AWS Kinesis Data Firehose* option. For every field and its bounds, refer to [Endpoints](/en/documentation/platform/data-stream/endpoints/#aws-kinesis-data-firehose). The example collects the requests of one workload with the *Applications* data source. --- Select your interface once. The prerequisites and every task below show only that path. ## Prerequisites - An Azion account with the **Edit Data Stream** permission. For the permissions, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#permissions). - A [workload](/en/documentation/platform/workloads/) on the account that receives requests. - An AWS account with an Amazon Kinesis Data Firehose delivery stream. The delivery stream must use **Direct PUT** as its source, and it needs a destination, such as an Amazon S3 bucket. - The name of the delivery stream, up to 64 characters. - The region code where the delivery stream runs, such as `us-east-1`. - An AWS access key: an access key ID and its secret access key, created in AWS Identity and Access Management (IAM). **Console** - Access to Azion Console. To sign in, refer to [How to access Azion Console](/en/documentation/guides/platform/account-and-billing/how-to-access-azion-console/). **API** - A personal token. To create one, refer to [How to manage a personal token](/en/documentation/guides/platform/account-and-billing/personal-tokens/). - The ID of the workload. - `curl`. --- ## Create the stream A workload filter limits the stream to the workload you choose. Sampling would deactivate your other streams, and a workload filter keeps them active. **Console** To create the stream in Azion Console: 1. **Open Data Stream** Access [Azion Console](https://console.azion.com/) > **Data Stream**. 2. **Select + Stream** 3. **Name the stream** In the **General** section, enter a **Name**. For example: `logs-to-kinesis`. 4. **Select the data source** In the **Input** section, select *Applications* in **Data Source**. 5. **Turn off Sampling** In the **Transform** section, turn off **Sampling** while **Option** is still *All Current and Future Workloads*, its starting value. A stream cannot carry sampling and a workload filter together. 6. **Choose the workload** In the **Transform** section, set **Option** to *Filter Workloads*. Select your workload in **Available Workload**, then move it to **Chosen Workload** with the arrow. 7. **Select the template** In the **Render Template** section, select *Applications Event Collector* in **Template**. 8. **Select the Kinesis endpoint** In the **Output** section, select *AWS Kinesis Data Firehose* in **Connector**. 9. **Enter the delivery stream** Enter the name of the delivery stream in **Stream Name**. Enter its region code in **Region**, such as `us-east-1`. 10. **Enter the access key** Enter the access key ID in **Access Key** and the secret access key in **Secret Key**. The Console masks both fields. 11. **Keep the stream active** In the **Status** section, keep **Active** turned on. 12. **Select Save** The Console shows `Your data stream has been created`. The **Data Stream** list shows the stream with `AWS Kinesis Firehose` in the **Connector** column and the **Active** status. **API** To create the stream with the API, send a `POST` request to `https://api.azion.com/v4/workspace/stream/streams`. Replace `[TOKEN VALUE]` with your personal token, `` with the ID of your workload, and the delivery stream values with your own: ```bash curl -X POST 'https://api.azion.com/v4/workspace/stream/streams' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{ "name": "logs-to-kinesis", "active": true, "inputs": [ { "type": "raw_logs", "attributes": { "data_source": "workloads" } } ], "transform": [ { "type": "filter_workloads", "attributes": { "workloads": [] } }, { "type": "render_template", "attributes": { "template": 2 } } ], "outputs": [ { "type": "aws_kinesis_firehose", "attributes": { "stream_name": "azion-logs", "region": "us-east-1", "access_key": "[ACCESS KEY]", "secret_key": "[SECRET KEY]" } } ] }' ``` The `workloads` data source is *Applications* in the Console, and template `2` is *Applications Event Collector*. The API answers `201` with the stored stream: ```json { "state": "executed", "data": { "id": 12353, "name": "logs-to-kinesis", "last_editor": "user@example.com", "created": "2026-01-01T12:10:35.000000Z", "last_modified": "2026-01-01T12:10:35.000000Z", "product_version": "1.0", … "outputs": [ { "type": "aws_kinesis_firehose", "attributes": { "access_key": "[ACCESS KEY]", "stream_name": "azion-logs", "region": "us-east-1", "secret_key": "[SECRET KEY]" } } ] } } ``` Keep the `id`. It identifies the stream in every later request, such as `/v4/workspace/stream/streams/12353`. For every key of the body, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#stream-object). The API and the Console save the stream without contacting AWS. A wrong stream name, region, or access key surfaces only when the stream sends. An activation takes effect after one to two minutes. --- ## Confirm the delivery [Real-Time Events](/en/documentation/platform/real-time-events/data-sources/#data-stream) records every send of a stream, delivered or not, with the status code the endpoint returned. Send a few requests to the workload, then wait about a minute. A stream sends a batch to Kinesis every 60 seconds, or sooner when it reaches 500 log lines. **Console** To find the sends in Azion Console: 1. **Open Real-Time Events** Access [Azion Console](https://console.azion.com/) > **Real-Time Events**. 2. **Select the Data Stream data source** 3. **Read the latest sends** Each row is one send. Find the rows with `AWS_KINESIS_FIREHOSE` in **Endpoint Type**, and read their **Status Code**. A **Status Code** of `200` means the delivery stream accepted the batch. **Streamed Lines** gives the number of log lines in the batch. **API** To read the sends with the API, query the `dataStreamedEvents` dataset of the Real-Time Events GraphQL API. Replace the dates with a range that covers the activation of the stream: ```bash curl -X POST 'https://api.azion.com/v4/events/graphql' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{"query":"query { dataStreamedEvents(limit: 20, filter: {tsRange: {begin: \"2026-01-01T12:00:00\", end: \"2026-01-01T12:45:00\"}}, orderBy: [ts_DESC]) { ts endpointType statusCode streamedLines dataStreamed } }"}' ``` The API answers `200` with one record for each send, the latest first: ```json { "data": { "dataStreamedEvents": [ { "ts": "2026-01-01T12:02:04Z", "endpointType": "AWS_KINESIS_FIREHOSE", "statusCode": 200, "streamedLines": 2, "dataStreamed": 2797 } ] } } ``` A send to Kinesis carries `AWS_KINESIS_FIREHOSE` in `endpointType`. A `statusCode` of `200` means the endpoint accepted the batch, and `streamedLines` and `dataStreamed` give its size in log lines and bytes. An empty `dataStreamedEvents` list means the stream has not sent in the range. For every field, refer to [Real-Time Events GraphQL fields](/en/documentation/devtools/graphql/gql-real-time-events-fields/#datastreamedevents-data-stream). A status other than `200` is the answer of the endpoint, and `503` means Data Stream found the endpoint unavailable. For the causes, refer to [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting/). --- ## Next steps - [Endpoints](/en/documentation/platform/data-stream/endpoints.md#aws-kinesis-data-firehose): Every field of the Kinesis endpoint, with its type, bounds, and API name. - [Data Stream limits](/en/documentation/platform/data-stream/limits.md): The batch size and interval of each endpoint, and the other limits of a stream. - [Edit, stop, or delete a stream](/en/documentation/guides/platform/observability/delete-data-stream.md): Change the delivery stream or the access key, pause the stream, or remove it. - [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting.md): Find what to change when a send returns a status other than 200.