Associate workloads with a stream
Choose the workloads a Data Stream stream collects logs from, in Azion Console or with the Azion API, and confirm the stream sends them.
You can associate workloads with a Data Stream stream from Azion Console or with the Azion API. To collect from every workload with sampling instead, refer to Configure sampling on a stream.
A stream collects the logs of the workloads associated with it and sends them to its endpoint. One stream can collect from a single workload or from several. Unlike sampling, a workload filter leaves your other streams active. For every field of the filter, refer to Stream settings.
Prerequisites
Console
- One or more workloads on your account.
- The Edit Data Stream permission. For details, refer to Stream settings.
- Access to Azion Console. To sign in, refer to How to access Azion Console.
- An account with fewer than 3,000 workloads. At 3,000 workloads or more, the Console blocks the stream forms, and you manage streams through the API only.
Create a stream for your workloads
You choose the workloads in the same form that sets the stream’s data source, template, and endpoint. The example uses the Applications data source. Activity History events belong to the account rather than to a workload, so an Activity History stream uses sampling instead.
Console
To create the stream in Azion Console:
Access Azion Console > Data Stream.
In the General section, enter a Name. For example: workloads-to-http.
In the Input section, select Applications in Data Source. For the other data sources, refer to Data sources and variables.
In the Transform section, Option starts at All Current and Future Workloads, with Sampling on. Turn off Sampling before you change Option, so the stream carries no sampling.
All Current and Future Workloads covers every workload on the account, including the ones you create later. A stream saved with Sampling on deactivates every other stream on the account.
Set Option to Filter Workloads. The Workloads pick list opens. If your Console names domains instead, the control reads All Current and Future Domains and Filter Domains. The pick list then reads Available Domain and Chosen Domain.
Use the controls of the pick list:
- To find a workload in Available Workload, enter characters in the search box, or scroll through the list.
- To move a workload to Chosen Workload, select it and then select the
>arrow. - To move several at once, select them with
ctrlon Windows and Linux orcommandon Mac, and then select>. - To move every workload the list shows, select
>>. - To remove workloads from Chosen Workload, select them and then select
<. To remove all of them, select<<.
In the Render Template section, select Applications Event Collector in Template. For the other templates, refer to Templates and payload.
In the Output section, select your endpoint in the field labeled Connector, and fill in its fields. For the fields of each endpoint, refer to Endpoints.
In the Status section, keep Active turned on.
If the Attention dialog opens, Sampling is still on. Select Cancel, set Option back to All Current and Future Workloads, and turn off Sampling. Set Option to Filter Workloads again, make sure your workloads are in Chosen Workload, and select Save.
The Console shows Your data stream has been created, and the Data Stream list shows the stream with the status Active.
The stream saves without contacting the endpoint, and an activation takes effect after one to two minutes.
Confirm the delivery
Real-Time Events records every send of a stream, delivered or not, with the status code the endpoint returned. Send a few requests to a chosen workload, then wait about a minute.
Console
To find the sends in Azion Console:
Access Azion Console > Real-Time Events.
Each row is one send. Read its Endpoint Type, Status Code, and Streamed Lines.
A Status Code of 200 means the endpoint accepted the batch.
A status other than 200 is the answer of the endpoint, and 503 means Data Stream found the endpoint unavailable. For the causes, refer to Troubleshoot Data Stream.