# Endpoints An endpoint is the platform where [Data Stream](/en/documentation/platform/data-stream/) delivers the log lines of a stream: a SIEM, a big-data platform, a stream-processing platform, an HTTP server, or a bucket. Each stream sends to one endpoint. Azion Console sets it in the **Output** section of the stream form, where the field is labeled **Connector** and each option shows its own fields. The API carries the endpoint in `outputs[0]`: the endpoint type goes in `outputs[0].type`, and its fields go in `outputs[0].attributes`. | Console option | API `type` | Credential | | ----------------------------------------------------------- | ---------------------- | -------------------------- | | [*Standard HTTP/HTTPS POST*](#standard-httphttps-post) | `standard` | The custom headers you set | | [*Apache Kafka*](#apache-kafka) | `kafka` | None | | [*Simple Storage Service (S3)*](#simple-storage-service-s3) | `s3` | Access key and secret key | | [*Google BigQuery*](#google-bigquery) | `big_query` | Service account key | | [*Elasticsearch*](#elasticsearch) | `elasticsearch` | Encoded API key | | [*Splunk*](#splunk) | `splunk` | HTTP Event Collector token | | [*AWS Kinesis Data Firehose*](#aws-kinesis-data-firehose) | `aws_kinesis_firehose` | Access key and secret key | | [*Datadog*](#datadog) | `datadog` | API key | | [*IBM QRadar*](#ibm-qradar) | `qradar` | None | | [*Azure Monitor*](#azure-monitor) | `azure_monitor` | Shared key | | [*Azure Blob Storage*](#azure-blob-storage) | `azure_blob_storage` | SAS token | In the tables below, the Required column gives the API rule. Where the Console differs, the cell names both. The Console marks each field it requires with an asterisk. Every request to `/v4/workspace/stream/streams` carries the header `Authorization: Token [TOKEN VALUE]`. --- ## Standard HTTP/HTTPS POST *Standard HTTP/HTTPS POST* sends the log lines in the body of `POST` requests to a URL you choose. Use it for any platform that receives data over HTTP or HTTPS and has no option of its own in the **Connector** list. | Console label | API field | Type | Required | Bounds | Description | | ------------------------------ | ------------------------------------------ | -------------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **URL** | `outputs[0].attributes.url` | string, URL | Yes | A URL that starts with `http://` or `https://` | The URL that receives the log lines, such as `https://app.domain.com/`. | | **Payload Format** | `outputs[0].attributes.payload_format` | string | Yes in the Console, no in the API | 1 to 250 characters | The body of each request. `$dataset` stands for the log lines of the batch, joined by the separator. Default `$dataset`. | | **Payload Log Line Separator** | `outputs[0].attributes.log_line_separator` | string | Yes in the Console, no in the API | 1 to 100 characters | The characters at the end of each log line. Default `\n`, which puts one log line per line, in NDJSON format. | | **Payload Max Size** | `outputs[0].attributes.max_size` | integer, bytes | No | 1,000,000 to 2,147,483,647 bytes | The maximum size of each data packet. Default `1000000` in the Console, and `null` when an API request leaves it out. | | **Custom Headers** | `outputs[0].attributes.headers` | object | Yes. The API accepts `{}`; the Console requires at least one header, as `name:value` | Each value 1 to 1,024 characters. The Console holds up to five headers. | The headers each request carries, such as an access key the platform requires. The Console takes one **Header** row per header, written `header-name:value`. The API takes an object of header names and values, and `{}` sends none. | For how **Payload Format**, the separator, and the template build the request body, refer to [Templates and payload](/en/documentation/platform/data-stream/templates-and-payload/#payload). For the steps, refer to [Send logs to an HTTP endpoint](/en/documentation/guides/platform/observability/connector-standard-https-post/). --- ## Apache Kafka *Apache Kafka* sends the log lines as messages to one topic of a Kafka cluster. | Console label | API field | Type | Required | Bounds | Description | | ----------------------------------------- | ----------------------------------------- | ------- | ------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Bootstrap Servers** | `outputs[0].attributes.bootstrap_servers` | string | Yes | 1 to 150 characters | The hosts and ports of the cluster, separated by a comma and no space, such as `myownhost.com:2021,imaginaryhost.com:4525,anotherhost:4030`. Only the servers for the initial connection are needed, not every server of the cluster. | | **Kafka Topic** | `outputs[0].attributes.kafka_topic` | string | Yes | 1 to 150 characters. One topic. | The topic that receives the messages, such as `analytics.fct.pageviews.0`. | | **Enable Transport Layer Security (TLS)** | `outputs[0].attributes.use_tls` | boolean | Yes in the API. The Console shows a switch. | `true` or `false` | `true` sends the data encrypted with Transport Layer Security (TLS). | With TLS on, the receiving servers need a digital certificate from a trusted certificate authority (CA), such as IdenTrust, DigiCert, Sectigo, GoDaddy, GlobalSign, or Let's Encrypt. For the steps, refer to [Send logs to Apache Kafka](/en/documentation/guides/platform/observability/endpoint-apache-kafka/). --- ## Simple Storage Service (S3) *Simple Storage Service (S3)* writes the log lines as objects in a bucket. It accepts any provider that works with the S3 protocol, [Object Storage](/en/documentation/platform/object-storage/) included. | Console label | API field | Type | Required | Bounds | Description | | --------------------- | ----------------------------------------- | ---------------- | -------------------------------------------- | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | | **URL** | `outputs[0].attributes.host_url` | string, URL | Yes | 1 to 200 characters, scheme included | The S3 host, such as `https://myownhost.s3.us-east-1.myprovider.com`. For Amazon S3, the default AWS endpoint `https://s3.amazonaws.com` works. | | **Bucket Name** | `outputs[0].attributes.bucket_name` | string | Yes | 1 to 150 characters | The bucket that receives the objects, such as `mys3bucket`. The bucket must exist before the stream sends to it. | | **Region** | `outputs[0].attributes.region` | string | Yes | 1 to 50 characters | The region of the bucket, such as `us-east-1`. | | **Access Key** | `outputs[0].attributes.access_key` | string | Yes | 1 to 150 characters | The public key of the credential. Masked in the Console. | | **Secret Key** | `outputs[0].attributes.secret_key` | string | Yes. The Console requires it on create only. | 1 to 150 characters | The secret key of the credential. Masked in the Console. On edit, the Console does not send an empty **Secret Key**. | | **Object Key Prefix** | `outputs[0].attributes.object_key_prefix` | string or `null` | No | 1 to 150 characters | The start of each object name, such as `user/logs`. Without a prefix, the objects go to the root of the bucket. Masked in the Console. | | **Content Type** | `outputs[0].attributes.content_type` | enum | Yes | `plain/text` or `application/gzip` | The format of each object. With *plain/text*, the object holds one log line per line. | Each object name is the **Object Key Prefix**, a `/`, the date and time of the send in the `YYYY/MM/DD/hh/mm/` format, and a UUID. With the prefix `activity`, an object name reads `activity/2026/01/01/12/02/11111111-1111-1111-1111-111111111111`. Data Stream adds the `/` after the prefix. The credential needs permission to list the bucket and to write objects in it. On Amazon S3, these are the `s3:ListBucket` and `s3:PutObject` permissions. For the steps, refer to [Send logs to Amazon S3](/en/documentation/guides/platform/observability/endpoint-amazon-s3/). ### Azion Object Storage An Object Storage bucket takes the URL `https://s3.us-east-005.azionstorage.net` and the region `us-east-005`. The S3 credential needs four capabilities: `listAllBucketNames`, `listBuckets`, `listFiles`, and `writeFiles`. Without `listAllBucketNames` and `listBuckets`, every send is recorded with status `503`, and no object reaches the bucket. The secret key of a credential shows only once, when the credential is created. The `outputs` entry below sends to an Object Storage bucket: ```json { "type": "s3", "attributes": { "host_url": "https://s3.us-east-005.azionstorage.net", "bucket_name": "", "region": "us-east-005", "access_key": "[ACCESS KEY]", "secret_key": "[SECRET KEY]", "object_key_prefix": "activity", "content_type": "plain/text" } } ``` For the steps, refer to [Send Data Stream data to Object Storage](/en/documentation/guides/platform/observability/connector-azion-object-storage/). --- ## Google BigQuery *Google BigQuery* sends the log lines as rows to a table of a BigQuery dataset. | Console label | API field | Type | Required | Bounds | Description | | ----------------------- | ------------------------------------------- | ------ | -------- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | **Project ID** | `outputs[0].attributes.project_id` | string | Yes | 1 to 100 characters | The ID of the project on Google Cloud, such as `mycustomGBQproject01`. | | **Dataset ID** | `outputs[0].attributes.dataset_id` | string | Yes | 1 to 1,024 characters, case sensitive | The name of the dataset, unique in its project, such as `myGBQdataset`. | | **Table ID** | `outputs[0].attributes.table_id` | string | Yes | 1 to 1,024 characters | The name of the table that receives the rows, such as `mypagaviewtable01`. | | **Service Account Key** | `outputs[0].attributes.service_account_key` | string | Yes | 1 to 65,535 characters | The content of the JSON key file of a Google Cloud service account. The Console takes it in a JSON editor. | Google BigQuery needs four things in place before the stream sends: - The dataset exists. - The table exists, with a schema for the data. - The BigQuery API is enabled on the project. - Billing is enabled on the project, because the free tier does not accept rows streamed into a table. Google Cloud provides the service account key as a JSON file with the keys below. In the API, `service_account_key` is a string that holds the JSON of the file: ```json { "type": "service_account", "project_id": "", "private_key_id": "", "private_key": "", "client_email": "", "client_id": "", "auth_uri": "", "token_uri": "", "auth_provider_x509_cert_url": "", "client_x509_cert_url": "" } ``` For the steps, refer to [Send logs to Google BigQuery](/en/documentation/guides/platform/observability/endpoint-google-bigquery/). --- ## Elasticsearch *Elasticsearch* sends the log lines to an index of an Elasticsearch instance, which can run on any cloud platform. | Console label | API field | Type | Required | Bounds | Description | | ------------------- | ------------------------------- | ----------- | -------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | **URL** | `outputs[0].attributes.url` | string, URL | Yes | A URL | The address of the instance followed by the index, such as `https://elasticsearch-domain.com/myindex`. | | **Encoded API Key** | `outputs[0].attributes.api_key` | string | Yes | 1 to 255 characters | The Base64 `encoded` value that Elasticsearch returns when it creates the API key, such as `VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw==`. | For the steps, refer to [Send logs to Elasticsearch](/en/documentation/guides/platform/observability/endpoint-elasticsearch/). --- ## Splunk *Splunk* sends the log lines to the HTTP Event Collector (HEC) of a Splunk instance. The HEC token must be enabled in Splunk. | Console label | API field | Type | Required | Bounds | Description | | ------------- | ------------------------------- | ----------- | -------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | **URL** | `outputs[0].attributes.url` | string, URL | Yes | A URL | The HEC URL. To send to another index, add it at the end of the URL, such as `https://inputs.splunkcloud.com:8080/services/collector?index=myindex`. | | **API Key** | `outputs[0].attributes.api_key` | string | Yes | 1 to 255 characters | The HEC token of the Splunk installation. | The HEC URL depends on the type of Splunk instance: | Splunk instance | HEC URL | | ------------------------------- | ----------------------------------------------------------------- | | Self-hosted | `https://:/services/collector/event` | | Self-service Splunk Cloud plans | `https://input-:/services/collector/event` | | Other Splunk Cloud plans | `://http-inputs-:/services/collector/event` | For the steps, refer to [Send logs to Splunk](/en/documentation/guides/platform/observability/endpoint-splunk/). --- ## AWS Kinesis Data Firehose *AWS Kinesis Data Firehose* sends the log lines to a Firehose delivery stream that uses **Direct PUT** as its source. Data Stream sends to this endpoint in batches of up to 500 log lines or every 60 seconds, whichever comes first. | Console label | API field | Type | Required | Bounds | Description | | --------------- | ----------------------------------- | ------ | -------------------------- | ------------------- | ------------------------------------------------------------------------ | | **Stream Name** | `outputs[0].attributes.stream_name` | string | Yes | 1 to 64 characters | The name of the delivery stream, such as `MyKDFConnector`. | | **Region** | `outputs[0].attributes.region` | string | Yes | 1 to 50 characters | The region of the delivery stream, such as `us-east-1`. | | **Access Key** | `outputs[0].attributes.access_key` | string | Yes | 1 to 150 characters | The public key AWS gives for the delivery stream. Masked in the Console. | | **Secret Key** | `outputs[0].attributes.secret_key` | string | Yes, on create and on edit | 1 to 150 characters | The secret key AWS gives for the delivery stream. Masked in the Console. | For the batch limits of every endpoint, refer to [Data Stream limits](/en/documentation/platform/data-stream/limits/). For the steps, refer to [Send logs to AWS Kinesis Data Firehose](/en/documentation/guides/platform/observability/endpoint-amazon-kinesis/). --- ## Datadog *Datadog* sends the log lines to a Datadog log intake URL. | Console label | API field | Type | Required | Bounds | Description | | ------------- | ------------------------------- | ----------- | -------- | ------------------- | ------------------------------------------------------------------------------------------- | | **URL** | `outputs[0].attributes.url` | string, URL | Yes | A URL | The URL of the Datadog endpoint, such as `https://http-intake.logs.datadoghq.com/v1/input`. | | **API Key** | `outputs[0].attributes.api_key` | string | Yes | 1 to 255 characters | The API key created in the Datadog dashboard. | For the steps, refer to [Send logs to Datadog](/en/documentation/guides/platform/observability/endpoint-datadog/). --- ## IBM QRadar *IBM QRadar* sends the log lines to a URL of a QRadar instance. The URL is the only field of this endpoint. | Console label | API field | Type | Required | Bounds | Description | | ------------- | --------------------------- | ----------- | -------- | ------ | ------------------------------------ | | **URL** | `outputs[0].attributes.url` | string, URL | Yes | A URL | The URL that receives the log lines. | For the steps, refer to [Send logs to IBM QRadar](/en/documentation/guides/platform/observability/endpoint-ibm-qradar/). --- ## Azure Monitor *Azure Monitor* sends the log lines to a workspace in Azure Monitor. | Console label | API field | Type | Required | Bounds | Description | | ------------------------ | -------------------------------------------- | ---------------- | -------- | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | **Log Type** | `outputs[0].attributes.log_type` | string | Yes | 1 to 100 characters: letters, numbers, and `_` only | The record type of the data, which names the table where Azure Monitor stores the logs, such as `AzureMonitorTest`. | | **Shared Key** | `outputs[0].attributes.shared_key` | string | Yes | 1 to 150 characters | The Primary Key of the workspace. Masked in the Console. | | **Time Generated Field** | `outputs[0].attributes.time_generated_field` | string or `null` | No | 1 to 50 characters | Optional. Without it, the ingestion time is used. Example: `myCustomTimeField`. | | **Workspace ID** | `outputs[0].attributes.workspace_id` | string | Yes | 1 to 150 characters | The ID of the workspace. | For the steps, refer to [Send logs to Azure Monitor](/en/documentation/guides/platform/observability/endpoint-azure-monitor/). --- ## Azure Blob Storage *Azure Blob Storage* writes the log lines to a container of an Azure storage account. The storage account and the container must exist before the stream sends. | Console label | API field | Type | Required | Bounds | Description | | ------------------- | --------------------------------------- | ------ | -------- | ------------------- | ------------------------------------------------------------------------------------- | | **Storage Account** | `outputs[0].attributes.storage_account` | string | Yes | 1 to 100 characters | The name of the storage account, such as `mystorageaccount`. | | **Container Name** | `outputs[0].attributes.container_name` | string | Yes | 1 to 150 characters | The name of the container, such as `mycontainer`. | | **Blob SAS Token** | `outputs[0].attributes.blob_sas_token` | string | Yes | 1 to 250 characters | The SAS token that Blob Storage generates, with create, read, write, and list access. | For the steps, refer to [Send logs to Azure Blob Storage](/en/documentation/guides/platform/observability/endpoint-azure-blob/). --- ## Credentials and masked fields An endpoint credential belongs to your account on the receiving platform, and the stream uses it to send the log lines its template shapes. Apache Kafka and IBM QRadar take no credential field. *Standard HTTP/HTTPS POST* carries a credential only in the custom headers you set. Azion Console masks six fields, with an icon that reveals the value: - S3 **Access Key**, **Secret Key**, and **Object Key Prefix**. - AWS Kinesis Data Firehose **Access Key** and **Secret Key**. - Azure Monitor **Shared Key**. The other credential fields show their value in the form: **Encoded API Key**, the Splunk and Datadog **API Key**, **Blob SAS Token**, and **Service Account Key**. An account with **View Data Stream** only sees a lock icon instead of the reveal icon. For the permissions, refer to [Stream settings](/en/documentation/platform/data-stream/stream-settings/#permissions). --- ## Errors The API refuses each endpoint request below with HTTP `400`, and the stream is not saved. The pointer in `source.pointer` names the field inside `outputs[0]`. | Code | Title | Pointer | Cause | What to do | | ------------------- | -------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | | `10050` | `Min Value` | `/data/outputs/0/max_size` | The `max_size` of a `standard` endpoint is below the floor: `Ensure this value is greater than or equal to 1000000.` | Set `max_size` to `1000000` or more, or omit it. | | `10059` | `Required Field` | `/data/outputs/0/headers` | A `standard` endpoint has no `headers` key. | Add `headers`, as `{}` when the endpoint needs no header. | | `10059` | `Required Field` | `/data/outputs/0/kafka_topic` | A `kafka` endpoint has no `kafka_topic` key. | Add the topic in `kafka_topic`. | | `32006` and `10032` | `Invalid URL Scheme` and `Invalid Url` | `/data/outputs/0/url` | The `url` of a `standard` endpoint is not an HTTP or HTTPS URL, such as `not-a-url`: `Only http and https schemes are allowed.` | Write the full URL, with `http://` or `https://`. | The API does not test the endpoint when it saves a stream. A `standard` endpoint that answers `405` to every send is still saved with `201`. Real-Time Events records each send in [`dataStreamedEvents`](/en/documentation/devtools/graphql/gql-real-time-events-fields/#datastreamedevents-data-stream), with the endpoint type, the number of log lines, and the status code. A delivered send is recorded with `200`, and a send to an unavailable endpoint with `503`. Data Stream checks each endpoint once a minute and discards the log lines of an interval in which the endpoint is unavailable. For the delivery path, refer to [How Data Stream works](/en/documentation/platform/data-stream/how-it-works/). --- ## Related resources - [Stream settings](/en/documentation/platform/data-stream/stream-settings.md#output): Every field of the stream form, including the Output section that picks the endpoint. - [Data Stream limits](/en/documentation/platform/data-stream/limits.md): The batch sizes, timeouts, and field bounds that apply to each endpoint. - [Troubleshoot Data Stream](/en/documentation/platform/data-stream/troubleshooting.md): The status codes a send can return, and what to change when log lines do not arrive. - [Data Stream guides and tutorials](/en/documentation/platform/data-stream/guides.md): The step-by-step guide for each endpoint type.