--- name: azion-customize-the-http-post-payload description: >- Set the payload format, log line separator, maximum size, and custom headers of a Standard HTTP/HTTPS POST endpoint, in Azion Console or with the Azion API. --- # Customize the HTTP POST payload You can customize the payload that a Standard HTTP/HTTPS POST endpoint receives from a stream, from Azion Console or with the Azion API. To create a stream that sends to an HTTP endpoint, with its data source and template, refer to [Send logs to an HTTP endpoint](/en/documentation/guides/platform/observability/connector-standard-https-post/). The payload is the body of each `POST` request that [Data Stream](/en/documentation/platform/data-stream/) sends. Four fields of the endpoint shape it: **Payload Format**, **Payload Log Line Separator**, **Payload Max Size**, and **Custom Headers**. No other endpoint type has them. In the stream form, the endpoint is set in the field labeled **Connector**. The template decides which variables each log line carries. For every payload field and its bounds, refer to [Templates and payload](/en/documentation/platform/data-stream/templates-and-payload/#payload). --- Select your interface once. The prerequisites and every task below show only that path. ## Prerequisites - An Azion account with the **Edit Data Stream** permission. For the permissions, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#permissions). - A [workload](/en/documentation/platform/workloads/) on the account that receives requests. - An HTTP or HTTPS URL that accepts `POST` requests, and the headers it requires, such as an authentication token. **Console** - Access to Azion Console. To sign in, refer to [How to access Azion Console](/en/documentation/guides/platform/account-and-billing/how-to-access-azion-console/). - An account with fewer than 3,000 workloads. At 3,000 workloads or more, the Console blocks the stream forms, and you manage streams through the API only. **API** - A personal token. To create one, refer to [How to manage a personal token](/en/documentation/guides/platform/account-and-billing/personal-tokens/). - The ID of the workload. - `curl`. --- ## Set the payload fields The fields are in the **Output** section of the stream form, on a stream you create or one you edit. With the defaults, the endpoint receives NDJSON: one log line per line, with no brackets and no comma between lines. **Console** To set the payload fields in Azion Console: 1. **Open Data Stream** Access [Azion Console](https://console.azion.com/) > **Data Stream**. 2. **Open the stream form** Select **+ Stream** to create a stream, or select the row of an existing stream to edit it. 3. **Select the endpoint** In the **Output** section, set **Connector** to *Standard HTTP/HTTPS POST*. In **URL**, enter the address of your endpoint, such as `https://logs.example.com/ingest`. 4. **Set the payload format** In **Payload Format**, keep `$dataset`, or enter up to 250 characters around it. Data Stream replaces `$dataset` with the log lines of the batch, joined by the separator. 5. **Set the log line separator** In **Payload Log Line Separator**, keep `\n` to put each log line on its own line. You can enter up to 100 characters instead. 6. **Set the maximum size** In **Payload Max Size**, enter the maximum size of a data packet in bytes, from `1000000` to `2147483647`. The default is `1000000`. 7. **Add the headers** In **Custom Headers**, enter one header in each **Header** row, as `header-name:value`. For a token, enter `Authorization:Bearer `. To add a row, select **Header**. The Console holds up to five headers and requires at least one. 8. **Select Save** 9. **Confirm the sampling warning** If the stream uses sampling, the **Attention** dialog opens. Saving then deactivates every other stream on the account. Select **Confirm**. The Console shows `Your data stream has been created` for a stream you create, or `Your data stream has been updated` for an edit. **API** To set the payload fields with the API, send them in `outputs[0].attributes` of the create request. This example collects the requests of one workload with the *Applications* data source and template `2`, *Applications Event Collector*. Replace `` with the ID of your workload and `[TOKEN]` with the token your endpoint expects: ```bash curl --request POST \ --url https://api.azion.com/v4/workspace/stream/streams \ --header 'Accept: application/json' \ --header 'Authorization: Token [TOKEN VALUE]' \ --header 'Content-Type: application/json' \ --data '{ "name": "requests-to-http", "active": true, "inputs": [ { "type": "raw_logs", "attributes": { "data_source": "workloads" } } ], "transform": [ { "type": "filter_workloads", "attributes": { "workloads": [] } }, { "type": "render_template", "attributes": { "template": 2 } } ], "outputs": [ { "type": "standard", "attributes": { "url": "https://logs.example.com/ingest", "headers": { "Authorization": "Bearer [TOKEN]" }, "log_line_separator": "\\n", "payload_format": "$dataset", "max_size": 1000000 } } ] }' ``` A `201` carries the stream, with the payload fields as stored: ```json { "state": "executed", "data": { "id": 12347, "name": "requests-to-http", "last_editor": "user@example.com", "created": "2026-01-01T12:10:22.000000Z", "last_modified": "2026-01-01T12:10:22.000000Z", "product_version": "1.0", … "outputs": [ { "type": "standard", "attributes": { "url": "https://logs.example.com/ingest", "log_line_separator": "\\n", "payload_format": "$dataset", "max_size": 1000000, "headers": { "Authorization": "Bearer [TOKEN]" } } } ] } } ``` The stream exists with the payload you sent. Because it uses a workload filter, not sampling, creating it deactivates no other stream. In JSON, `"\\n"` is the escape sequence `\n`, the separator the Console shows by default. `headers` is required, and `{}` sends no header. A request that leaves out the other fields stores `$dataset`, `\n`, and a `max_size` of `null`. To change the payload of an existing stream, send a `PATCH` request to `/v4/workspace/stream/streams/` with the whole `outputs` array. A change to the payload of an existing stream takes a few minutes to propagate. Saving checks the format of the fields, not the endpoint, so a wrong URL or header shows up later as failed sends. --- ## Send the log lines as one JSON array A receiver that reads each request body as a single JSON document needs the log lines in a JSON array, with brackets around them and a comma between them. Two fields change: **Payload Format** wraps `$dataset` in brackets, and the separator becomes a comma. For the NDJSON and JSON array bodies side by side, refer to [Templates and payload](/en/documentation/platform/data-stream/templates-and-payload/#payload). **Console** To send a JSON array in Azion Console: 1. **Open Data Stream** Access [Azion Console](https://console.azion.com/) > **Data Stream**. 2. **Open the stream** In the list, select the row of the stream that sends to the HTTP endpoint. 3. **Wrap the log lines in brackets** In the **Output** section, set **Payload Format** to `[$dataset]`. 4. **Separate the log lines with a comma** Set **Payload Log Line Separator** to `,`. 5. **Select Save** 6. **Confirm the sampling warning** If the **Attention** dialog opens, select **Confirm**. The Console shows `Your data stream has been updated`. Each request body is then one JSON array of log lines. **API** To send a JSON array with the API, set `payload_format` to `[$dataset]` and `log_line_separator` to `,` in the endpoint of the stream. The `outputs` item reads: ```json { "type": "standard", "attributes": { "url": "https://logs.example.com/ingest", "headers": { "Authorization": "Bearer [TOKEN]" }, "log_line_separator": ",", "payload_format": "[$dataset]", "max_size": 1000000 } } ``` Send this item in the `outputs` array of a create request, or of a `PATCH` request to `/v4/workspace/stream/streams/`. Each request body is then one JSON array of log lines. --- ## Confirm the delivery [Real-Time Events](/en/documentation/platform/real-time-events/data-sources/#data-stream) records every send of a stream, accepted or not, with the status code your endpoint returned. Send a few requests to the workload, then wait about a minute. A batch leaves after 60 seconds, or sooner at 2,000 log lines or at the **Payload Max Size**. **Console** To find the sends in Azion Console: 1. **Open Real-Time Events** Access [Azion Console](https://console.azion.com/) > **Real-Time Events**. 2. **Select the Data Stream data source** 3. **Read the latest sends** Each row is one send. Find the rows with `HTTP_POST` in **Endpoint Type**, and read their **Status Code**. A **Status Code** of `200` means your endpoint accepted the batch. **Streamed Lines** gives the number of log lines in the batch. **API** To read the sends with the API, query the `dataStreamedEvents` dataset of the Real-Time Events GraphQL API. Replace the dates with a range that covers the save of the stream: ```bash curl -X POST 'https://api.azion.com/v4/events/graphql' \ -H 'Content-Type: application/json' \ -H 'Authorization: Token [TOKEN VALUE]' \ -d '{"query":"query { dataStreamedEvents(limit: 20, filter: {tsRange: {begin: \"2026-01-01T11:40:00\", end: \"2026-01-01T12:00:00\"}}, orderBy: [ts_DESC]) { ts jobName endpointType statusCode streamedLines url } }"}' ``` The API answers `200` with one record for each send, the latest first: ```json { "data": { "dataStreamedEvents": [ { "ts": "2026-01-01T11:44:00Z", "jobName": "Data Streaming RTM Activity", "endpointType": "HTTP_POST", "statusCode": 405, "streamedLines": 2, "url": "https://example.com/logs" }, { "ts": "2026-01-01T11:43:00Z", "jobName": "Data Streaming RTM Activity", "endpointType": "HTTP_POST", "statusCode": 405, "streamedLines": 1, "url": "https://example.com/logs" } ] } } ``` A `statusCode` of `405` means the endpoint at `url` refused the batch, and `streamedLines` gives the number of log lines in it. A `statusCode` of `200` means your endpoint accepted the batch. An empty `dataStreamedEvents` list means the stream has not sent in the range. For every field, refer to [Real-Time Events GraphQL fields](/en/documentation/devtools/graphql/gql-real-time-events-fields/#datastreamedevents-data-stream). A status other than `200`, such as `405`, is the answer of your endpoint. A `503` means Data Stream found the endpoint unavailable. For the causes, refer to [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting/). --- ## Next steps - [Templates and payload](/en/documentation/platform/data-stream/templates-and-payload.md#payload): Compare the NDJSON and JSON array bodies, and look up each payload field. - [Send logs to an HTTP endpoint](/en/documentation/guides/platform/observability/connector-standard-https-post.md): Create a stream that sends its logs to an HTTP or HTTPS URL. - [Create a custom template](/en/documentation/guides/application-development/frameworks/data-stream-custom-template.md): Choose the variables and keys each log line of the payload carries. - [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting.md): Fix a stream whose endpoint refuses the batches or receives nothing.