Real⁠-⁠Time Events

Preview

Real-Time Events is an Observe product that provides raw data, logs, from other Azion products in real time.

A set of preorganized variables are available to make queries manually using different data sources. This allows you to get extensive, detailed information on behaviors, occurrences, and performance of your applications through logs.

You can use Real-Time Events to:

Perform complex searches.
Inspect possible attacks.
Perform debugging investigations.
Analyze application’s performance.
Analyze applications and platform savings.
Increase reliability of your data.
Decrease problem-solving time.
Improve content delivery based on actual data.

Implementation

Task	Guide
See first steps	Real-Time Events first steps

Events storage

Real-Time Events stores events logs from the last 168 hours, equivalent to 7 days. You’re able to query detailed data during that period.

Data sources

Data Source represents the Azion product or service that generated the events you’ll query for. When submitting a query, the data source represents the index from where you want to collect data.

Selecting a data source in the dropdown menu is mandatory. You can choose between:

Each data source has a specific set of available variables, representing the specific information you can receive in your query. See each data source’s prerequisites and variables and their description next.

Data Streaming

Requires: Data Streaming

Displays the event records of data sent to your configured endpoints.

Variable	Description
$client_id	Unique Azion customer identifier. Example: `4529r`
$data_streamed	Total amount of data streamed in bytes to the configured endpoint (attempt/sink/send). Example: `1270`
$endpoint_type	Type of endpoint used in the configured data streaming. Can be: `HTTP_POST`, `S3`, `ELASTICSEARCH`, `QRADAR`, `AWS_KINESIS_FIREHOSE`, `KAFKA`, `DATADOG`, `BIG_QUERY`, `SPLUNK`, `AZURE_MONITOR`, `AZURE_BLOB_STORAGE`
$job_name	Unique Azion identification for the type of streaming created. Example: `Data Streaming HTTP`, `Data Streaming WAF`.
$status_code	The status code of the request. Example: `200`
$streamed_lines	Total amount of lines streamed to the configured endpoint. Maximum value of `2000`. Example: `837`
$time	Date and time when the data was sent to the configured endpoint. Example: `31 October, 2023, 21:10:55`
$url	The URL to which the client data was sent/sink. Example: `https://s3.amazonaws.com`

Edge Applications

Displays the event records from requests made to your edge applications.

Variable	Description
$asn	Autonomous System Number (ASN) Allocation, which are IP address networks managed by one or more network operators that have a clear and unique routing policy. Example: `AS52580`
$bytes_sent	Number of bytes sent to a client, including header and body. Example: `191`
$client	Unique Azion customer identifier. Example: `4529r`
$city	The remote client’s city name detected via IP address geolocation. Example: `Chicago`, `Boston`
$country	Remote client’s country detected via IP address geolocation. Example: `United States`, `Russian Federation`
$host	Host information sent on the request line. Stores: host name from the request line or host name from the `Host` request header field, or the server name matching a request. Example: `website.com.br`
$http_referrer	Address of the page the user made the request from. Example: `https://example.com`
$http_user_agent	End user’s application, operating system, vendor, and/or version. Value of the `User-Agent` header. Example: `Mozilla/5.0 (Windows NT 10.0; Win64; x64)`
$proxy_status	HTTP error status code or origin when no response is obtained from the upstream. Example: `520`. In case of cache, the response is `-`.
$remote_addr	IP address of the origin that generated the request. Example: `54.233.153.15`
$remote_port	Remote port of the origin that generated the request. Example: `26081`
$request_id	Unique request identifier. Example: `5f222ae5938482c32a822dbf15e19f0f`
$request_length	Request length, including request line, headers, and body. Example: `1133`
$request_method	HTTP request method. Example: `GET` or `POST`.
$request_time	Request processing time elapsed since the first bytes were read from the client with resolution in milliseconds. Example: `1.19`
$request_uri	URI of the request made by the end user, without the host and protocol information and with arguments. Example: `/v1?v=bo%20dim`
$scheme	Request scheme. Example: `HTTP` or `HTTPS`
$sent_http_content_type	`Content-Type` header sent in the origin’s response. Example: `text/html; charset=UTF-8`
$server_addr	IP address of the server that received the request. Example: `179.191.169.73`
$server_port	Remote port of the server that received the request. Example: `443`
$server_protocol	Request protocol. Example: `HTTP/1.1`, `HTTP/2.0`, `HTTP/3.0`
$ssl_cipher	Cipher string used to establish TLS connection. Example: `TLS_AES_256_GCM_SHA384`
$ssl_protocol	Protocol for an established TLS connection. Example: `TLS v1.2`
$state	Remote client’s state detected via IP address geolocation. Example: `CA`, `RS`
$status	HTTP status code of the request. Example: `200`
$time	Request date and time. Example: `31 October, 2023, 21:10:55`
$traceback	Provides the names of the Rules Engine from your edge application or your edge firewall that are run by the request. Example: `export NODE_OPTIONS="--max-old-space-size=4096"{\"edge_firewall\":[\"BODY DS\",\"WAF Rules\"]}`
$upstream_addr	Client’s IP address and port. Can also store multiple servers or server groups. Example: `192.168.1.1:80`. When the response is `127.0.0.1:1666`, the upstream is Azion Cells Runtime.
$upstream_bytes_received	Number of bytes received by the origin’s edge if the content isn’t cached. Example: `8304`
$upstream_bytes_sent	Number of bytes sent to the origin. Example: `2733`
$upstream_cache_status	Status of the local edge cache. Can be: `MISS`, `BYPASS`, `EXPIRED`, `STALE`, `UPDATING`, `REVALIDATED`, or `HIT`
$upstream_response_time	Time it takes for the edge to receive a default response from the origin in milliseconds, including headers and body. Example: `0.876`. In case of cache, the response is `-`.
$upstream_status	HTTP status code of the origin. If a server cannot be selected, the variable keeps the 502 (Bad Gateway) status code. Example: `200`. In case of cache, the response is `-`.
$waf_attack_action	Reports WAF’s action regarding the action. Can be: `$BLOCK`, `$PASS`, `$LEARNING_BLOCK`, or `$LEARNING_PASS`
$waf_attack_family	Informs the attack’s families, which are categories of attack identified by WAF and classified according to the OWASP Top 10. Example: `$XSS`. See the categories.
$waf_block	Informs whether the WAF blocked the action or not. `0` when action wasn’t blocked and `1` when action was blocked. When in Learning Mode, it won’t be blocked regardless of the return.
$waf_headers	When the request headers sent by the user are analyzed by the WAF module and tagged as blocked with `$waf_block = 1`, it contains a `base64` encoded string. Otherwise, it contains a dash character `-`. It applies to both WAF Learning or Blocking modes.
$waf_learning	Informs if WAF is in Learning mode. Can be `0` or `1`.
$waf_match	List of infractions found in the end user’s request. It’s formed by key-value elements: the key refers to the type of violation detected; the value shows the string that generated the infraction. Example: `0:1311:BODY:ctl00_cph_jp1_dados_container_clientstate`. Find out more on WAF Allowed Rules.
$waf_score	Reports the score that’ll be increased in case of a match with the rules set for the WAF. Example: `0:$SQL:2`

The $traceback variable can be used if you have the Debug Rules feature activated in your application. Find out more on How to debug rules created with Rules Engine.

Edge Functions

Requires: Edge Functions

Displays the event records of requests made to your edge functions.

Variable	Description
$client_id	Unique Azion customer identifier. Example: `4529r`
$configuration_id	Unique Azion configuration identifier. Example: `1595368520`
$function_id	Unique Azion function identifier number. Can be found on RTM’s function URL path or via API request. Example: `1111`
$message_content	Open field with a message used in the console.log for debugging. Example: `[Send event to endpoint] Generic error handler; TypeError: error sending request for url (https://http-intake.logs.datadoghq.com/v1/input): connection closed before message completed`
$message_level	Message with the level type for the function. Can be `MDN`, `DEBUG`, `INFO`, `ERROR`, `LOG`, or `WARN`
$message_source	The source of the message. Can be: `CONSOLE` when messages are generated by the Console API, or `RUNTIME` when it’s related to an error message.
$time	Request date and time. Example: `31 October, 2023, 21:10:55`

Edge Pulse

Displays the event records from requests made to your edge pulse using the performance data measured from the user’s browser.

Variable	Description
$browser	The UUID generated for the client’s browser used on the request. Example: `2648698a-61cb-45ad-bbb5-c569313185d4`
$client_id	Unique Azion customer identifier. Example: `4529r`
$contentdownload	Time used to download the requested content. Example: `5`
$dns	DNS resolution time. Example: `0`
$downlink	Returns the average volume of data received in Mb/s. Example: `10`
$effectivetype	The effective type of the connection. Example: `3g`, `4g`, `5g`
$hostname	Hostname of the current URL. Example: `website.com.br`
$locationhref	The complete URL of the current page. Example: `https://www.azion.com/pt-br/sobre-nos/`
$navigation.contentDownload	Time used to download the content.
$navigation.dns	DNS resolution time.
$navigation.networkDuration	Duration without query browser waiting.
$navigation.PageLoadTime	Time from the start of navigation to the full page load.
$navigation.redirectCount	The number of redirects since the last navigation without redirection in the context of the current navigation.
$navigation.renderTime	Time the browser was rendered for after browsing.
$navigation.ssl	Standard protocol for an established TLS connection.
$navigation.tcp	Internet protocol that returns the data that makes up the page.
$navigation.ttfb	Time until the arrival of the first byte of the requested page.
$navigation.type	Type of navigation without redirection.
$navigation.typeBackForward	Type of navigation through the session history.
$navigation.typeNavigate	Type of the last navigation without redirection. Example: by clicking on a link, by entering the URL in the address bar, or by submitting a form.
$navigation.typeReload	Type of navigation for the reload operation: when the page was reloaded.
$navigation.typeReserved	Any type of navigation not defined by other specific variables (navigation.typeNavigate and navigation.typeReload).
$networkApi.downlink	The average volume of data received in Mb/s.
$networkduration	Duration without query browser waiting. Example: `52`
$pageloadtime	Time from the start of navigation until the full page load. Example: `1267`
$platform	Operating system architecture. Example: `Linux x86_64`, `Iphone`
$redirectcount	Number of redirects since the last navigation without redirection in the context of the current navigation. Example: `0`
$referrer	Address of the page the user made the request from. The URL by which the user arrived at “locationHref”. If the access originated directly from the page (for example, through a bookmark), the value will be an `empty string`. It doesn’t provide DOM access to the reference page. Example: `https://www.azion.com/pt-br/`
$rendertime	The amount of time it took to render the page. Example: `1242`
$rtt	Round-Trip Time (RTT) information. Example: `250`
$ssl	Standard protocol for an established TLS connection. Example: `0`. If the requested URL has a secure connection, it returns the time it took for authenticating.
$tcp	Internet protocol that returns the data that makes up the page. The time it takes for the TCP handshake is the time between the connection start and the connection end. Example: `0`
$time	Request date and time. Example: `31 October, 2023, 21:10:55`
$timestamp	Request date and time. Example: `31 October, 2023, 21:10:55`
$ttfb	Time To First Byte: time until the arrival of the first byte of the requested page in milliseconds. Includes 1 round trip of latency and the time the server took to prepare the response. Example: `1`
$type	Type of navigation without redirection. Indicates how the navigation to this page/script was done. Example: `navigation`, `reload`
$typebackforward	Type of navigation through the session history. Example: `2`
$typenavigate	Type of the last navigation without redirection. Example: by clicking on a link, by entering the URL in the address bar, or by submitting a form. Example: `0`
$typereload	Type of navigation for the reload operation: when the page was reloaded. Example: `1`
$typereserved	Any type of navigation not defined by other specific variables (typeNavigate and typeReload). Example: `1`
$userAgent	End user’s application, operating system, vendor, and/or version. Value of the `User-Agent` header. Example: `Mozilla/5.0 (Windows NT 10.0; Win64; x64)`
$version	The Azion Log version used. Example: `v5`

WAF Events

Requires: Web Application Firewall

Displays the event records of requests analyzed by WAF to allow you to map the score assigned to the request, the WAF rules that matched, the reason for the block, and more.

Variable	Description
$attack_family	Informs the attack’s families, which are categories of attack identified by WAF and classified according to the OWASP Top 10. Example: `$XSS`. See the categories
$blocked	Informs whether the WAF blocked the action or not. `0` when action wasn’t blocked and `1` when action was blocked. When in Learning Mode, it won’t be blocked regardless of the return.
$client_id	Unique Azion customer identifier. Example: `4529r`
$geoloc_country_name	Remote client’s country detected via IP address geolocation. Example: `United States`, `Russian Federation`
$host	Host information sent on the request line. Stores: host name from the request line, or host name from the `Host` request header, or the server name matching a request. Example: `website.com.br`
$remote_address	IP address of the origin that generated the request. Example: `54.233.153.15`
$server_protocol	The connection established protocol. Example: `HTTP/1.1`, `HTTP/2.0`, `HTTP/3.0`
$time	Request date and time. Example: `31 October, 2023, 21:10:55`
$total_blocked	Informs the total number of requests blocked by WAF. Example: `2`
$total_processed	Informs the total number of requests processed by WAF. Example: `1`
$waf_action	Reports WAF’s action regarding the action. Can be: `$BLOCK`, `$PASS`, `$LEARNING_BLOCK`, or `$LEARNING_PASS`.
$waf_args	The request arguments. Example: `quantidade_periodos=10`
$waf_learning	Informs if WAF is in Learning mode. Can be `0` or `1`
$waf_match	List of infractions found in the end user’s request. It’s formed by key-value elements: the key refers to the type of violation detected; the value shows the string that generated the infraction. Example: `0:1311:BODY:ctl00_cph_jp1_dados_container_clientstate`. Find out more on WAF Allowed Rules
$waf_score	Reports the score that’ll be increased in case of a match with the rules set for the WAF. Example: `0:$SQL:2`
$waf_server	Hostname used in the WAF request. Example: `api-login.azion.com.br`
$waf_uri	URI used in the WAF request. Example: `/access/v2/after-login`

Time filter

The Time filter allows you to refine the period for the events record search result. It’s selected by default for Last 15 minutes.

You can filter by:

Last 15 minutes
Last 30 minutes
Last 1 hour
Last 3 hours
Last 6 hours
Last 12 hours
Last day
Last 2 days
Last 3 days
Last 5 days
Last 7 days
Custom

By using the Custom field, you can customize your search by selecting a date and time range during the last 168 hours.

You can change the time range as many times as you want to investigate your logs.

Filter by

In the Filter by field, you filter your search results by using a keyword or phrase. This makes your search more specific and makes it easier to find the logs you want to analyze.

By submitting a search with a blank Filter by field, you’ll get all existing records for the variables of the selected data source available during the selected time filter.

The field uses SQL language to query results. Your search must be in one of two formats:

key='value'

Exact match, where:
- key: one of the variables from the data source you’re querying for.
- =: means the search must query for the exact value passed.
- value: a value of either string or integer format.

key like '%value%'

Similar value, where:
- key: one of the variables from the data source you’re querying for.
- like: means the search must query for a similar value to the one passed.
- %value%: a value of either string or integer format surrounded.

In the second format, you can use with value:

%value%: filters for values that contain the entire specified value.
%value: filters for values that end with the specified value.
value%: filters for values that begin with the specified value.

You can also search for more complex queries with the AND, OR, and NOT notations to combine the fields.

Some examples of SQL queries:

Variable	SQL query
$status	`status='404'`
$status + $scheme	`status='200' AND scheme='https'`
$endpoint_type	`endpoint_type='datadog'`
$geoloc_country_name	`geoloc_country_name='Brazil'`
$message_content	`message_content like '%unavailable%'`
$message_content	`message_content like '%available%'`

Data exhibition

After you complete the filters and search for results, your logs will appear in a table.

Each line is a different log, which equals to a different action performed by the edge. If you click on a log, the line will expand and provide more detailed information about it. The information shown varies according to the specifics of each variable.

The interface paginates for a given amount of results at first, but it’ll continue to load all available results as you scroll through the page.

Refresh

After performing a query, you can use the Refresh button to update the returned data. Real-Time Events will repeat the last search performed, updating the data but keeping the time filter and the SQL filter you used.

If you had the Last 30 minutes time filter, for example, and it was 4:00, you’d have logs from 3:30-4:00. If you use the Refresh button at 4:45, you’ll have logs from 4:15 P.M-4:45.

The search continues to return the results ordered by most recent to oldest.

Limits

Depending on the size of your data, the query limit may exceed. If this happens, filter by a short time filter.

Contributors