How to use Amazon Kinesis Data Firehose to receive data from Data Streaming

While configuring data streamings, you need to set up a specific endpoint to stream your Azion data.

Continue reading for a step by step on how to connect an Amazon Kinesis Data Firehose endpoint to receive data from Data Streaming.


Amazon Kinesis Data Firehose requirements

Section titled Amazon Kinesis Data Firehose requirements

To start using Amazon Kinesis Data Firehose, follow the steps:

  1. Create an account on Amazon AWS.
  2. In the AWS dashboard, create an instance.
  3. Fill in the requested information.
  4. Create an Amazon S3 bucket.

You’ll also need to create and save the following information:

  • Stream Name
  • Region
  • Access Key
  • Secret Key

See how to obtain them while still on the AWS platform:

  1. Go to the Amazon Kinesis console homepage.
  2. On Get started, select Kinesis Data Firehose as an option.
  3. Click the Create delivery stream button. You’ll be redirected to your stream creation page.
  4. In the Choose source and destination section, on Source, select Direct PUT.
  5. On Destination, select Amazon S3. The rest of the configuration options will load.
  6. In the Delivery stream name section, choose a delivery stream name according to what you find most convenient.
  7. Keep the other options with the default value.
  8. In the Destination Settings section, on S3 bucket, click browse and select a previously created S3 bucket. You can also click the Create button and create a new one now.
  9. Keep the other options with the default value.
  10. Click on Create a delivery flow.

You can find more details on the AWS Creating the Kinesis Data Firehose delivery stream documentation.

This is the region where your AWS Kinesis instance is running.

  1. On the top bar of your dashboard, right next to the help button with the ’?’ icon, you find a dropdown menu, the Regions selector, where you can see the name of the region and its code.
  2. Find the code and keep it to use it later. It looks similar to this: us-east-1

You can find more details on the AWS Regions and Zones documentation.

  1. Access the Identify and Access Management (IAM) panel > click the Create new access key button.
  2. A pop-up will appear on the screen with your key information.
  3. Click on Display Access Key to display your key information as indicated below:
    • Access Key ID, with a format similar to AKIA5LVCRTER524RAIA3
    • Secret Access Key, with a format similar to N9UTjunb299M5ryfKdG2cKh2+Pe2mBiGZXaBpVff
  4. Save the two keys information in a safe place to use them later on Azion Data Streaming.

You can also download the EMP file with the information for the keys.


Configuring the endpoint in Data Streaming

Section titled Configuring the endpoint in Data Streaming

Next, follow these steps to configure the new endpoint you created in Amazon Kinesis Data Firehose in your Azion Data Streaming.

You can find detailed steps for the entire configuration on the How to use Data Streaming guide.

In the Destination configurations:

  1. On the Endpoint Type dropdown menu, select AWS Kinesis Data Firehose.
  2. On Stream Name, add the name you gave your delivery Kinesis Firehose stream.
  3. On Region, add the region configured on your stream.
  4. On Access Key, add the access key you created for your stream.
  5. On Secret Key, add the secret key you created for your stream.
  6. Make sure the Active switch is on.
  7. Click the Save button.

After saving the configurations, your data will be streamed to the newly configured endpoint.

You can keep track of the calls made by Data Streaming to Amazon Kinesis Data Firehose on Real-Time Events. To do so, select Data Source > Data Streaming and choose the filters options as you wish.

Watch a video tutorial on how to use Amazon Kinesis Data Firehose with Data Streaming on Azion’s YouTube channel:


Amazon Web Services, AWS, Amazon Kinesis and affiliates are registered trademarks of Amazon Inc in the United States and other countries.


Contributors