Endpoints
Look up the 11 endpoint types a stream sends its log lines to, with every field, type, and bound in Azion Console and the Azion API.
An endpoint is the platform where Data Stream delivers the log lines of a stream: a SIEM, a big-data platform, a stream-processing platform, an HTTP server, or a bucket. Each stream sends to one endpoint. Azion Console sets it in the Output section of the stream form, where the field is labeled Connector and each option shows its own fields. The API carries the endpoint in outputs[0]: the endpoint type goes in outputs[0].type, and its fields go in outputs[0].attributes.
| Console option | API type | Credential |
|---|---|---|
| Standard HTTP/HTTPS POST | standard | The custom headers you set |
| Apache Kafka | kafka | None |
| Simple Storage Service (S3) | s3 | Access key and secret key |
| Google BigQuery | big_query | Service account key |
| Elasticsearch | elasticsearch | Encoded API key |
| Splunk | splunk | HTTP Event Collector token |
| AWS Kinesis Data Firehose | aws_kinesis_firehose | Access key and secret key |
| Datadog | datadog | API key |
| IBM QRadar | qradar | None |
| Azure Monitor | azure_monitor | Shared key |
| Azure Blob Storage | azure_blob_storage | SAS token |
In the tables below, the Required column gives the API rule. Where the Console differs, the cell names both. The Console marks each field it requires with an asterisk. Every request to /v4/workspace/stream/streams carries the header Authorization: Token [TOKEN VALUE].
Standard HTTP/HTTPS POST
Standard HTTP/HTTPS POST sends the log lines in the body of POST requests to a URL you choose. Use it for any platform that receives data over HTTP or HTTPS and has no option of its own in the Connector list.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| URL | outputs[0].attributes.url | string, URL | Yes | A URL that starts with http:// or https:// | The URL that receives the log lines, such as https://app.domain.com/. |
| Payload Format | outputs[0].attributes.payload_format | string | Yes in the Console, no in the API | 1 to 250 characters | The body of each request. $dataset stands for the log lines of the batch, joined by the separator. Default $dataset. |
| Payload Log Line Separator | outputs[0].attributes.log_line_separator | string | Yes in the Console, no in the API | 1 to 100 characters | The characters at the end of each log line. Default \n, which puts one log line per line, in NDJSON format. |
| Payload Max Size | outputs[0].attributes.max_size | integer, bytes | No | 1,000,000 to 2,147,483,647 bytes | The maximum size of each data packet. Default 1000000 in the Console, and null when an API request leaves it out. |
| Custom Headers | outputs[0].attributes.headers | object | Yes. The API accepts {}; the Console requires at least one header, as name:value | Each value 1 to 1,024 characters. The Console holds up to five headers. | The headers each request carries, such as an access key the platform requires. The Console takes one Header row per header, written header-name:value. The API takes an object of header names and values, and {} sends none. |
For how Payload Format, the separator, and the template build the request body, refer to Templates and payload. For the steps, refer to Send logs to an HTTP endpoint.
Apache Kafka
Apache Kafka sends the log lines as messages to one topic of a Kafka cluster.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| Bootstrap Servers | outputs[0].attributes.bootstrap_servers | string | Yes | 1 to 150 characters | The hosts and ports of the cluster, separated by a comma and no space, such as myownhost.com:2021,imaginaryhost.com:4525,anotherhost:4030. Only the servers for the initial connection are needed, not every server of the cluster. |
| Kafka Topic | outputs[0].attributes.kafka_topic | string | Yes | 1 to 150 characters. One topic. | The topic that receives the messages, such as analytics.fct.pageviews.0. |
| Enable Transport Layer Security (TLS) | outputs[0].attributes.use_tls | boolean | Yes in the API. The Console shows a switch. | true or false | true sends the data encrypted with Transport Layer Security (TLS). |
With TLS on, the receiving servers need a digital certificate from a trusted certificate authority (CA), such as IdenTrust, DigiCert, Sectigo, GoDaddy, GlobalSign, or Let’s Encrypt. For the steps, refer to Send logs to Apache Kafka.
Simple Storage Service (S3)
Simple Storage Service (S3) writes the log lines as objects in a bucket. It accepts any provider that works with the S3 protocol, Object Storage included.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| URL | outputs[0].attributes.host_url | string, URL | Yes | 1 to 200 characters, scheme included | The S3 host, such as https://myownhost.s3.us-east-1.myprovider.com. For Amazon S3, the default AWS endpoint https://s3.amazonaws.com works. |
| Bucket Name | outputs[0].attributes.bucket_name | string | Yes | 1 to 150 characters | The bucket that receives the objects, such as mys3bucket. The bucket must exist before the stream sends to it. |
| Region | outputs[0].attributes.region | string | Yes | 1 to 50 characters | The region of the bucket, such as us-east-1. |
| Access Key | outputs[0].attributes.access_key | string | Yes | 1 to 150 characters | The public key of the credential. Masked in the Console. |
| Secret Key | outputs[0].attributes.secret_key | string | Yes. The Console requires it on create only. | 1 to 150 characters | The secret key of the credential. Masked in the Console. On edit, the Console does not send an empty Secret Key. |
| Object Key Prefix | outputs[0].attributes.object_key_prefix | string or null | No | 1 to 150 characters | The start of each object name, such as user/logs. Without a prefix, the objects go to the root of the bucket. Masked in the Console. |
| Content Type | outputs[0].attributes.content_type | enum | Yes | plain/text or application/gzip | The format of each object. With plain/text, the object holds one log line per line. |
Each object name is the Object Key Prefix, a /, the date and time of the send in the YYYY/MM/DD/hh/mm/ format, and a UUID. With the prefix activity, an object name reads activity/2026/01/01/12/02/11111111-1111-1111-1111-111111111111. Data Stream adds the / after the prefix.
The credential needs permission to list the bucket and to write objects in it. On Amazon S3, these are the s3:ListBucket and s3:PutObject permissions. For the steps, refer to Send logs to Amazon S3.
Azion Object Storage
An Object Storage bucket takes the URL https://s3.us-east-005.azionstorage.net and the region us-east-005. The S3 credential needs four capabilities: listAllBucketNames, listBuckets, listFiles, and writeFiles. Without listAllBucketNames and listBuckets, every send is recorded with status 503, and no object reaches the bucket. The secret key of a credential shows only once, when the credential is created.
The outputs entry below sends to an Object Storage bucket:
For the steps, refer to Send Data Stream data to Object Storage.
Google BigQuery
Google BigQuery sends the log lines as rows to a table of a BigQuery dataset.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| Project ID | outputs[0].attributes.project_id | string | Yes | 1 to 100 characters | The ID of the project on Google Cloud, such as mycustomGBQproject01. |
| Dataset ID | outputs[0].attributes.dataset_id | string | Yes | 1 to 1,024 characters, case sensitive | The name of the dataset, unique in its project, such as myGBQdataset. |
| Table ID | outputs[0].attributes.table_id | string | Yes | 1 to 1,024 characters | The name of the table that receives the rows, such as mypagaviewtable01. |
| Service Account Key | outputs[0].attributes.service_account_key | string | Yes | 1 to 65,535 characters | The content of the JSON key file of a Google Cloud service account. The Console takes it in a JSON editor. |
Google BigQuery needs four things in place before the stream sends:
- The dataset exists.
- The table exists, with a schema for the data.
- The BigQuery API is enabled on the project.
- Billing is enabled on the project, because the free tier does not accept rows streamed into a table.
Google Cloud provides the service account key as a JSON file with the keys below. In the API, service_account_key is a string that holds the JSON of the file:
For the steps, refer to Send logs to Google BigQuery.
Elasticsearch
Elasticsearch sends the log lines to an index of an Elasticsearch instance, which can run on any cloud platform.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| URL | outputs[0].attributes.url | string, URL | Yes | A URL | The address of the instance followed by the index, such as https://elasticsearch-domain.com/myindex. |
| Encoded API Key | outputs[0].attributes.api_key | string | Yes | 1 to 255 characters | The Base64 encoded value that Elasticsearch returns when it creates the API key, such as VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw==. |
For the steps, refer to Send logs to Elasticsearch.
Splunk
Splunk sends the log lines to the HTTP Event Collector (HEC) of a Splunk instance. The HEC token must be enabled in Splunk.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| URL | outputs[0].attributes.url | string, URL | Yes | A URL | The HEC URL. To send to another index, add it at the end of the URL, such as https://inputs.splunkcloud.com:8080/services/collector?index=myindex. |
| API Key | outputs[0].attributes.api_key | string | Yes | 1 to 255 characters | The HEC token of the Splunk installation. |
The HEC URL depends on the type of Splunk instance:
| Splunk instance | HEC URL |
|---|---|
| Self-hosted | https://<host>:<port>/services/collector/event |
| Self-service Splunk Cloud plans | https://input-<host>:<port>/services/collector/event |
| Other Splunk Cloud plans | <protocol>://http-inputs-<host>:<port>/services/collector/event |
For the steps, refer to Send logs to Splunk.
AWS Kinesis Data Firehose
AWS Kinesis Data Firehose sends the log lines to a Firehose delivery stream that uses Direct PUT as its source. Data Stream sends to this endpoint in batches of up to 500 log lines or every 60 seconds, whichever comes first.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| Stream Name | outputs[0].attributes.stream_name | string | Yes | 1 to 64 characters | The name of the delivery stream, such as MyKDFConnector. |
| Region | outputs[0].attributes.region | string | Yes | 1 to 50 characters | The region of the delivery stream, such as us-east-1. |
| Access Key | outputs[0].attributes.access_key | string | Yes | 1 to 150 characters | The public key AWS gives for the delivery stream. Masked in the Console. |
| Secret Key | outputs[0].attributes.secret_key | string | Yes, on create and on edit | 1 to 150 characters | The secret key AWS gives for the delivery stream. Masked in the Console. |
For the batch limits of every endpoint, refer to Data Stream limits. For the steps, refer to Send logs to AWS Kinesis Data Firehose.
Datadog
Datadog sends the log lines to a Datadog log intake URL.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| URL | outputs[0].attributes.url | string, URL | Yes | A URL | The URL of the Datadog endpoint, such as https://http-intake.logs.datadoghq.com/v1/input. |
| API Key | outputs[0].attributes.api_key | string | Yes | 1 to 255 characters | The API key created in the Datadog dashboard. |
For the steps, refer to Send logs to Datadog.
IBM QRadar
IBM QRadar sends the log lines to a URL of a QRadar instance. The URL is the only field of this endpoint.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| URL | outputs[0].attributes.url | string, URL | Yes | A URL | The URL that receives the log lines. |
For the steps, refer to Send logs to IBM QRadar.
Azure Monitor
Azure Monitor sends the log lines to a workspace in Azure Monitor.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| Log Type | outputs[0].attributes.log_type | string | Yes | 1 to 100 characters: letters, numbers, and _ only | The record type of the data, which names the table where Azure Monitor stores the logs, such as AzureMonitorTest. |
| Shared Key | outputs[0].attributes.shared_key | string | Yes | 1 to 150 characters | The Primary Key of the workspace. Masked in the Console. |
| Time Generated Field | outputs[0].attributes.time_generated_field | string or null | No | 1 to 50 characters | Optional. Without it, the ingestion time is used. Example: myCustomTimeField. |
| Workspace ID | outputs[0].attributes.workspace_id | string | Yes | 1 to 150 characters | The ID of the workspace. |
For the steps, refer to Send logs to Azure Monitor.
Azure Blob Storage
Azure Blob Storage writes the log lines to a container of an Azure storage account. The storage account and the container must exist before the stream sends.
| Console label | API field | Type | Required | Bounds | Description |
|---|---|---|---|---|---|
| Storage Account | outputs[0].attributes.storage_account | string | Yes | 1 to 100 characters | The name of the storage account, such as mystorageaccount. |
| Container Name | outputs[0].attributes.container_name | string | Yes | 1 to 150 characters | The name of the container, such as mycontainer. |
| Blob SAS Token | outputs[0].attributes.blob_sas_token | string | Yes | 1 to 250 characters | The SAS token that Blob Storage generates, with create, read, write, and list access. |
For the steps, refer to Send logs to Azure Blob Storage.
Credentials and masked fields
An endpoint credential belongs to your account on the receiving platform, and the stream uses it to send the log lines its template shapes. Apache Kafka and IBM QRadar take no credential field. Standard HTTP/HTTPS POST carries a credential only in the custom headers you set.
Azion Console masks six fields, with an icon that reveals the value:
- S3 Access Key, Secret Key, and Object Key Prefix.
- AWS Kinesis Data Firehose Access Key and Secret Key.
- Azure Monitor Shared Key.
The other credential fields show their value in the form: Encoded API Key, the Splunk and Datadog API Key, Blob SAS Token, and Service Account Key. An account with View Data Stream only sees a lock icon instead of the reveal icon. For the permissions, refer to Stream settings.
Errors
The API refuses each endpoint request below with HTTP 400, and the stream is not saved. The pointer in source.pointer names the field inside outputs[0].
| Code | Title | Pointer | Cause | What to do |
|---|---|---|---|---|
10050 | Min Value | /data/outputs/0/max_size | The max_size of a standard endpoint is below the floor: Ensure this value is greater than or equal to 1000000. | Set max_size to 1000000 or more, or omit it. |
10059 | Required Field | /data/outputs/0/headers | A standard endpoint has no headers key. | Add headers, as {} when the endpoint needs no header. |
10059 | Required Field | /data/outputs/0/kafka_topic | A kafka endpoint has no kafka_topic key. | Add the topic in kafka_topic. |
32006 and 10032 | Invalid URL Scheme and Invalid Url | /data/outputs/0/url | The url of a standard endpoint is not an HTTP or HTTPS URL, such as not-a-url: Only http and https schemes are allowed. | Write the full URL, with http:// or https://. |
The API does not test the endpoint when it saves a stream. A standard endpoint that answers 405 to every send is still saved with 201. Real-Time Events records each send in dataStreamedEvents, with the endpoint type, the number of log lines, and the status code. A delivered send is recorded with 200, and a send to an unavailable endpoint with 503. Data Stream checks each endpoint once a minute and discards the log lines of an interval in which the endpoint is unavailable. For the delivery path, refer to How Data Stream works.