How to use Data Streaming

Data Streaming is an Observe product that allows you to feed your SIEM, big data, and stream processing platforms with the event logs from your applications on Azion in real time. You can find more detailed information about the product and its first steps on the reference documentation.

Continue reading this guide to find out how to use and configure a data streaming on Azion.

Before being able to access the Data Streaming product, you need to have an Azion account to use with Real-Time Manager (RTM). Create one through the sign-up page or see the documentation page for more information.

After creating your Azion account, you can access Data Streaming:

  1. Access RTM.
  2. On the upper-left corner, on the Products Menu, select Data Streaming on the OBSERVE section.
  3. Click Add Streaming.

Next, fill in the fields to configure your data streaming. Fields marked with an asterisk * are mandatory.

  1. Choose a name for your data streaming: try to use a meaningful name to help you remember for what purpose you’ve created this data streaming.
  2. On the Data Source dropdown menu, select the one you want to use: Activity History, Edge Applications, Edge Functions, or WAF Events. For Edge Functions and WAF Events, you must be subscribed to the products.
  3. On the Template dropdown menu, select the one you want to use: a pre-set template according to your data source choice or a Custom Template.
  4. On Options, select between Filter Domains or All Domains. See more about each option on How to associate domains on Data Streaming.
  5. On the Destination section, select an Endpoint Type on the dropdown menu: Standard HTTP/HTTPS POST, Apache Kafka, Simples Storage Service (S3), Google BigQuery, Elasticsearch, Splunk, AWS Kinesis Data Firehose, Datadog, IBM QRadar, Azure Monitor, or Azure Blob Storage.
    • You’ll see different fields depending on the endpoint type you choose. Find more information on each of them on the Setting an endpoint page.

Finally, to finish your Data Streaming setup:

  1. Make sure the Active switch is turned on.
  2. Click the Save button.

You’ve successfully created a data streaming on RTM.

To make sure your logs are being sent to your endpoint through the configured data streaming, you can use Real-Time Events to query detailed analysis.

To see all your created data streamings, go to the Data Streaming page on RTM and select one from the list. Feel free to create multiple ones.

Watch a video tutorial on how to use Data Streaming on Azion’s YouTube channel: