Stream request and WAF records to a SIEM

You send every request to an application, with the WAF fields of the same request, to a security information and event management (SIEM) platform from Azion Console or the Azion API. To send only the requests WAF analyzed, refer to Stream WAF events to a SIEM.

A Data Stream stream on the Applications data source, shaped by the Applications + WAF Event Collector preset, carries the request, the response, and the client address with the WAF score, matches, and block flag of the same request. One log line then answers a question about one request, whether WAF flagged it or not. The cost is volume: the preset sends 51 keys, the most of the five presets, and Data Stream is billed on requests and data transfer.

1. Each request to a chosen workload becomes an event of the Applications data source.
2. The workload filter keeps only the workloads you choose, and leaves the account’s other streams active.
3. Template 184 renders each event as one log line with the request fields and the WAF fields.
4. The stream sends the lines to the SIEM’s endpoint in batches, and Real-Time Events records each send with the status the endpoint returned.

Prerequisites

• An account user with the Edit Data Stream permission. For the permissions, refer to Stream settings.
• A workload whose firewall applies a WAF rule set with a Set WAF rule. To build it, refer to Apply a rule set to every request.
• An endpoint that your SIEM reads. The examples use an Apache Kafka cluster, with the host and port of its servers and one topic. For the other endpoints, refer to Endpoints.
• A personal token and the ID of the workload, for the API procedure.
• Access to Azion Console, for the Console procedure. Refer to Access Azion Console.

The examples send the requests of the workload <workload-id> to the topic azion.requests on kafka1.example.com:9092 and kafka2.example.com:9092. Replace them with your workload and endpoint.

Create the stream

The stream takes its scope from a workload filter rather than sampling. A stream carries exactly one of the two, and saving an active stream with sampling, at any rate including 100, deactivates every other stream on the account with no error. A filter holds up to 600 workloads, and a workload created later is not collected until you add it.

The stream starts sending one to two minutes after it is saved. Saving checks the format of each field and does not contact the endpoint, so a wrong address or topic shows only when the stream sends. For a SIEM with its own connector, select it in Connector, or send its type in outputs[0]: Splunk takes the HTTP Event Collector URL and token, IBM QRadar a URL, and Elasticsearch a URL and an encoded API key. A stream keeps one endpoint, and the API drops any second entry in outputs with no error. For every field, refer to Endpoints.

Confirm the delivery

Real-Time Events records every send of a stream, delivered or not, with the status code the endpoint returned. A stream sends a batch every 60 seconds, or sooner when it reaches 2,000 log lines. A failed send does not stop the stream, and its lines are not sent again. Send a few requests to the workload, then wait about a minute.

A status other than 200 is the answer of the endpoint, and 503 means Data Stream found the endpoint unavailable. For the causes, refer to Troubleshoot Data Stream.

At the SIEM, each log line carries the keys of the template, named after its variables. These keys tie the security decision to the request:

• request_id identifies the request, and remote_addr, host, request_uri, and status describe it.
• waf_score and waf_match hold the score of the request and the infractions it matched.
• waf_block reads 1 when WAF blocked the request. While waf_learning reads 1, WAF blocks no request, whatever waf_block reads.

The SIEM receives one line per request to the workload, each with its WAF fields. The SIEM keeps the lines for as long as you configure it to.

Next steps