Using Amazon S3 to receive data from Azion Data Streaming
Data Streaming is one of Azion’s Observe products designed to help you access your content and application data in real time. To successfully use Data Streaming with connectors, you first need to set up the endpoints.
After completing the initial setup, you can use Amazon Simple Storage Service (Amazon S3) to receive data from Azion Data Streaming. Once you finish connecting the endpoint, you can improve your monitoring and use other Azion products to continue exploring information on your data.
Continue reading this hands-on guide to see step by step of how to connect Amazon S3 endpoint to Data Streaming.
To get started, you must complete a few configurations in Amazon S3:
- Create an Amazon Web Services (AWS) account.
- Register an Identity and Access Management (IAM) credential.
- Add an Amazon S3 bucket.
- If desired, you can also upload an object.
- Create an access key and a private key.
- Save your newly created access key and private key.
- Optionally, you can set up and manage Server Side Encryption (SSE) with Amazon S3-Managed Keys (SSE-S3).
For more details on setting up Amazon S3, see its documentation page.
2. Configuring the new endpoint in Azion DataStreaming
Next, you’ll follow these steps to configure the new endpoint you created in Amazon S3 in your Azion Data Streaming.
- If you are a new user of Data Streaming, access Real-Time Manager (RTM) > Account Menu > Billing & Subscriptions and enable the product in the Subscriptions tab.
- On the upper left corner of the page, select Products Menu > Data Streaming.
- Click the Add Streaming button.
- Type a name for your Data Streaming.
- In the Data Source dropdown list, select the option you’ll use:
- Edge Applications
- Edge Functions
- WAF Events
For more information on creating Edge Applications, see the documentation page.
- In Template, choose the option that fits your Data Source choice:
- WAF Event Collector
- Edge Functions Event Collector
- Edge Applications + WAF Event Collector
- Custom Template
In case you select Custom Template, you’ll need to provide information in a JSON format in the Data Set field.
- In Options, you can:
- Select Filter Domains and add the domains you want to receive the logs by selecting them in the Available Domains box.
- Select All Domains.
- In the Destination configurations, select Simple Storage Service (S3) from the Endpoint Type dropdown list.
- Fill the following fields:
- Host URL: URL of your S3 host.
- Bucket Name: Name of your S3 bucket.
- Region: Region of your S3 bucket.
- Access Key: Access key of your S3 bucket.
- Secret Key: Secret key of your S3 bucket.
- Object Key Prefix (optional): Prefix name of your uploaded object, for example: WAF_.
- Content Type (optional): Format of the object that will be created in your S3 bucket — plain/text or application/gzip.
- Make sure the Active switch is on.
- Click the Save button.
After saving the configurations, you can keep track of the calls made by Data Streaming to Amazon S3 in the Real-Time Events product, available at the Products Menu in RTM. To do so, select the Data Source > Data Streaming and choose the filters options as you desire.
Amazon Simple Storage Service is a registered trademark of Amazon Web Services in the United States and other countries.
Didn’t find what you were looking for? Open a support ticket.