How to create a custom template on Data Streaming

Data Streaming is an Observe product that allows you to feed your SIEM, big data, and stream processing platforms with the event logs from your applications on Azion in real time.

While configuring a data streaming on Real-Time Manager (RTM), after choosing a data source on Data Streaming, you have two options:

  • Use the pre-set template for that specific data source, which sends data for all available variables.
  • Use the Custom Template, which allows you to customize which variables you want to use on your analysis according to your needs.

You can inform the exact variables you want to send as data through a JSON format, and each request is separated from the other with the \n character.

To use the Custom Template option on Data Streaming, follow these steps:

  1. Access RTM.
  2. On the upper-left corner, select Data Streaming on the OBSERVE section.
  3. Click Add Streaming.
  4. Choose a name for your data streaming.
  5. On the Data Source dropdown menu, select the one you want to use.
  6. On the Template dropdown menu, select Custom Template.
  7. On the Data Set code box, add the variables you want to use to receive your data.

You can see the entire list of variables and their descriptions for each data set on the Data Streaming documentation. Or, you can navigate through the pre-set templates on RTM by clicking on the Template dropdown menu and then copying the variables you desire.

Here’s an example of a custom template for the Edge Applications data source. The pre-set template currently contains about 50 variables, but you may only need 10 variables for your data analysis. Then, you can add the variables you want to use in the Data Set code box:

"time": "$time",
"session_id": "$session_id",
"host": "$host",
"status": "$status",
"bytes_sent": "$bytes_sent",
"upstream_bytes_received": "$upstream_bytes_received",
"server_port": "$server_port",
"remote_port": "$remote_port",
"country": "$country",
"state": "$state"

After adding the variables of your choice:

  1. Fill in the remaining mandatory fields to configure your data streaming.
  2. Make sure the Active switch is on.
  3. Click Save.

Now, your configured data streaming will send only the data related to the variables you added in the Custom Template.