Glossary

This glossary defines the terms that Data Stream uses in Azion Console, in the Azion API, and across its documentation.

Term	Definition
active	The state of a stream that collects and sends log lines, set by the Active switch and the `active` field, `true` by default. Turning it off stops the stream and keeps its settings, and the list of streams shows each one as Active or Inactive. A change of state takes effect after one to two minutes, and Stream settings describes the field.
batch	The group of log lines that a stream sends in one delivery, at 2,000 log lines or after 60 seconds, whichever comes first. A Standard HTTP/HTTPS POST endpoint also sends a batch that reaches its Payload Max Size. AWS Kinesis Data Firehose sends at 500 log lines or 60 seconds, and Data Stream limits lists both.
connector	The Console label of the Output field that picks the endpoint type, which the API carries in `outputs[0].type`. The documentation calls the destination an endpoint, and the field has no relation to Connectors, a separate Platform Resource. Stream settings maps each Connector option to its API type.
custom template	A template that your account creates, with a name and a data set that lists the variables you choose. The Console creates it in the Create Custom Template drawer of the Render Template section and lists it under Custom Templates. Templates and payload describes the format, and Create a custom template gives the steps.
data set	The JSON object of a template that maps each key of a log line to a variable, such as `"title": "$title"`. The Console shows it in the read-only Data Set field, and the API stores it as the `data_set` string, up to 65,535 characters. A data source is where events come from, and the data set picks the values each log line carries, as Templates and payload describes.
data source	The source of the events a stream collects: Activity History, Applications, Functions, or WAF Events. A stream has one data source, set in the Data Source field and in `inputs[0].attributes.data_source` as `activity_history`, `workloads`, `functions_console`, or `waf`. Data sources and variables lists the variables each one carries.
endpoint	The destination that receives the log lines of a stream, such as a SIEM, a big-data platform, or a stream-processing platform. A stream sends to one endpoint of 11 types, set in the `outputs` array, and the Console labels the field Connector. Endpoints lists the fields of each type.
endpoint check	The test that Data Stream runs once a minute to mark each endpoint as available, which needs every Azion server to report it available. While it is unavailable, Data Stream sends nothing to it and discards the logs of that interval. The check runs again the next minute, and How Data Stream works describes the sequence.
event	One occurrence that a data source records, such as a request to a workload or a change made in the account. A stream renders each event it sends as one log line, and sampling sets the share of events it sends. Data sources and variables lists what each data source records about an event.
filter workloads	The option that limits a stream to the events of the workloads you pick, from 1 to 600 workload IDs on the account. The Console shows it as Filter Workloads under Option, and the API carries it as a `filter_workloads` transform item. Unlike sampling, it leaves other streams active, and Associate workloads with a stream gives the steps.
log line	The text that a template renders from one event, which the stream sends to the endpoint. Batch sizes count log lines, which some pages call records. Real-Time Events counts them per delivery as streamed lines, and Templates and payload shows how a template shapes them.
log line separator	The characters that end each log line in the payload of a Standard HTTP/HTTPS POST endpoint, set in Payload Log Line Separator and in `log_line_separator`. The default is `\n`, which puts each log line on its own line, and the value takes up to 100 characters. It is an endpoint setting, not part of the template, as Templates and payload describes.
NDJSON	Newline-delimited JSON, a format with one JSON object per line, no enclosing `[]`, and no comma between lines. The receiver can process one record at a time. A Standard HTTP/HTTPS POST endpoint sends NDJSON by default, as Templates and payload describes.
payload	The body of one delivery that a stream sends to its endpoint, which carries the log lines of one batch. Only a Standard HTTP/HTTPS POST endpoint lets you shape it, with the payload format and the log line separator. Customize the HTTP POST payload gives the steps.
payload format	The pattern that a Standard HTTP/HTTPS POST endpoint fills to build each payload, set in Payload Format and in `payload_format`, up to 250 characters. The default `$dataset` stands for the log lines of the batch, each rendered from the data set and ended with the log line separator. A format such as `[$dataset]` with a `,` separator sends a JSON array, and Templates and payload describes both settings.
preset template	One of the five templates that Azion provides, listed under Azion’s Templates and marked `custom: false` in the API. The presets are Activity History Collector, Applications Event Collector, Applications + WAF Event Collector, Functions Event Collector, and WAF Event Collector. Their data sets are read-only, and Templates and payload lists the variables of each one.
sampling	The transform item that sends a percentage of the events of a stream, from 1 to 100, set by Sampling Rate (%) or `transform[sampling].attributes.rate`. Saving an active stream with sampling, at any rate including `100`, deactivates every other stream on the account. Configure sampling on a stream gives the steps.
stream	The Data Stream object that collects the events of one data source and renders each one as a log line with a template. It sends the lines in batches to one endpoint, and the API serves streams at `/v4/workspace/stream/streams`. Stream settings shows a complete stream object.
streamed lines	The number of log lines in one delivery, which Real-Time Events records in the `streamedLines` field of `dataStreamedEvents`. Each delivery also records `endpointType`, `statusCode`, `dataStreamed`, and `url`, so a rejected delivery shows with the status the endpoint returned. Real-Time Events GraphQL fields lists every field.
template	The definition that turns each event into a log line, made of a data set that maps keys to variables. Every stream carries one, a preset template or a custom template, in `transform[render_template].attributes.template`, picked in the Template field of the Render Template section. Templates and payload describes both kinds.
transform	The array of a stream that holds the items applied to events before delivery: one `render_template` item and exactly one `sampling` or `filter_workloads` item. In the Console, the Transform and Render Template sections fill it. Stream settings describes each item.
variable	A name that starts with `$`, such as `$time`, that a data set maps to a key. The stream replaces it with the value from each event, and each data source has its own variables. Data sources and variables lists them with an example value for each one.