Real-Time Events
Preview
Real-Time Events is an Observe product that provides raw data, logs, from other Azion products in real time.
A set of preorganized variables are available to make queries manually using different data sources. This allows you to get extensive, detailed information on behaviors, occurrences, and performance of your applications through logs.
You can use Real-Time Events to:
- Perform complex searches.
- Inspect possible attacks.
- Perform debugging investigations.
- Analyze application’s performance.
- Analyze applications and platform savings.
- Increase reliability of your data.
- Decrease problem-solving time.
- Improve content delivery based on actual data.
Implementation
Section titled ImplementationTask | Guide |
---|---|
See first steps | Real-Time Events first steps |
Events storage
Section titled Events storageReal-Time Events stores events logs from the last 168 hours, equivalent to 7 days. You’re able to query detailed data during that period.
Data sources
Section titled Data sourcesData Source represents the Azion product or service that generated the events you’ll query for. When submitting a query, the data source represents the index from where you want to collect data.
Selecting a data source in the dropdown menu is mandatory. You can choose between:
Each data source has a specific set of available variables, representing the specific information you can receive in your query. See each data source’s prerequisites and variables and their description next.
Data Streaming
Section titled Data StreamingRequires: Data Streaming
Displays the event records of data sent to your configured endpoints.
Variable | Description |
---|---|
$client_id | Unique Azion customer identifier. Example: 4529r |
$data_streamed | Total amount of data streamed in bytes to the configured endpoint (attempt/sink/send). Example: 1270 |
$endpoint_type | Type of endpoint used in the configured data streaming. Can be: HTTP_POST , S3 , ELASTICSEARCH , QRADAR , AWS_KINESIS_FIREHOSE , KAFKA , DATADOG , BIG_QUERY , SPLUNK , AZURE_MONITOR , AZURE_BLOB_STORAGE |
$job_name | Unique Azion identification for the type of streaming created. Example: Data Streaming HTTP , Data Streaming WAF . |
$status_code | The status code of the request. Example: 200 |
$streamed_lines | Total amount of lines streamed to the configured endpoint. Maximum value of 2000 . Example: 837 |
$time | Date and time when the data was sent to the configured endpoint. Example: 31 October, 2023, 21:10:55 |
$url | The URL to which the client data was sent/sink. Example: https://s3.amazonaws.com |
Edge Applications
Section titled Edge ApplicationsDisplays the event records from requests made to your edge applications.
Variable | Description |
---|---|
$asn | Autonomous System Number (ASN) Allocation, which are IP address networks managed by one or more network operators that have a clear and unique routing policy. Example: AS52580 |
$bytes_sent | Number of bytes sent to a client, including header and body. Example: 191 |
$client | Unique Azion customer identifier. Example: 4529r |
$city | The remote client’s city name detected via IP address geolocation. Example: Chicago , Boston |
$country | Remote client’s country detected via IP address geolocation. Example: United States , Russian Federation |
$host | Host information sent on the request line. Stores: host name from the request line or host name from the Host request header field, or the server name matching a request. Example: website.com.br |
$http_referrer | Address of the page the user made the request from. Example: https://example.com |
$http_user_agent | End user’s application, operating system, vendor, and/or version. Value of the User-Agent header. Example: Mozilla/5.0 (Windows NT 10.0; Win64; x64) |
$proxy_status | HTTP error status code or origin when no response is obtained from the upstream. Example: 520 . In case of cache, the response is - . |
$remote_addr | IP address of the origin that generated the request. Example: 54.233.153.15 |
$remote_port | Remote port of the origin that generated the request. Example: 26081 |
$request_id | Unique request identifier. Example: 5f222ae5938482c32a822dbf15e19f0f |
$request_length | Request length, including request line, headers, and body. Example: 1133 |
$request_method | HTTP request method. Example: GET or POST . |
$request_time | Request processing time elapsed since the first bytes were read from the client with resolution in milliseconds. Example: 1.19 |
$request_uri | URI of the request made by the end user, without the host and protocol information and with arguments. Example: /v1?v=bo%20dim |
$scheme | Request scheme. Example: HTTP or HTTPS |
$sent_http_content_type | Content-Type header sent in the origin’s response. Example: text/html; charset=UTF-8 |
$server_addr | IP address of the server that received the request. Example: 179.191.169.73 |
$server_port | Remote port of the server that received the request. Example: 443 |
$server_protocol | Request protocol. Example: HTTP/1.1 , HTTP/2.0 , HTTP/3.0 |
$ssl_cipher | Cipher string used to establish TLS connection. Example: TLS_AES_256_GCM_SHA384 |
$ssl_protocol | Protocol for an established TLS connection. Example: TLS v1.2 |
$state | Remote client’s state detected via IP address geolocation. Example: CA , RS |
$status | HTTP status code of the request. Example: 200 |
$time | Request date and time. Example: 31 October, 2023, 21:10:55 |
$traceback | Provides the names of the Rules Engine from your edge application or your edge firewall that are run by the request. Example: export NODE_OPTIONS="--max-old-space-size=4096"{\"edge_firewall\":[\"BODY DS\",\"WAF Rules\"]} |
$upstream_addr | Client’s IP address and port. Can also store multiple servers or server groups. Example: 192.168.1.1:80 . When the response is 127.0.0.1:1666 , the upstream is Azion Cells Runtime. |
$upstream_bytes_received | Number of bytes received by the origin’s edge if the content isn’t cached. Example: 8304 |
$upstream_bytes_sent | Number of bytes sent to the origin. Example: 2733 |
$upstream_cache_status | Status of the local edge cache. Can be: MISS , BYPASS , EXPIRED , STALE , UPDATING , REVALIDATED , or HIT |
$upstream_response_time | Time it takes for the edge to receive a default response from the origin in milliseconds, including headers and body. Example: 0.876 . In case of cache, the response is - . |
$upstream_status | HTTP status code of the origin. If a server cannot be selected, the variable keeps the 502 (Bad Gateway) status code. Example: 200 . In case of cache, the response is - . |
$waf_attack_action | Reports WAF’s action regarding the action. Can be: $BLOCK , $PASS , $LEARNING_BLOCK , or $LEARNING_PASS |
$waf_attack_family | Informs the attack’s families, which are categories of attack identified by WAF and classified according to the OWASP Top 10. Example: $XSS . See the categories. |
$waf_block | Informs whether the WAF blocked the action or not. 0 when action wasn’t blocked and 1 when action was blocked. When in Learning Mode, it won’t be blocked regardless of the return. |
$waf_headers | When the request headers sent by the user are analyzed by the WAF module and tagged as blocked with $waf_block = 1 , it contains a base64 encoded string. Otherwise, it contains a dash character - . It applies to both WAF Learning or Blocking modes. |
$waf_learning | Informs if WAF is in Learning mode. Can be 0 or 1 . |
$waf_match | List of infractions found in the end user’s request. It’s formed by key-value elements: the key refers to the type of violation detected; the value shows the string that generated the infraction. Example: 0:1311:BODY:ctl00_cph_jp1_dados_container_clientstate . Find out more on WAF Allowed Rules. |
$waf_score | Reports the score that’ll be increased in case of a match with the rules set for the WAF. Example: 0:$SQL:2 |
The $traceback
variable can be used if you have the Debug Rules feature activated in your application. Find out more on How to debug rules created with Rules Engine.
Edge Functions
Section titled Edge FunctionsRequires: Edge Functions
Displays the event records of requests made to your edge functions.
Variable | Description |
---|---|
$client_id | Unique Azion customer identifier. Example: 4529r |
$configuration_id | Unique Azion configuration identifier. Example: 1595368520 |
$function_id | Unique Azion function identifier number. Can be found on RTM’s function URL path or via API request. Example: 1111 |
$message_content | Open field with a message used in the console.log for debugging. Example: [Send event to endpoint] Generic error handler; TypeError: error sending request for url (https://http-intake.logs.datadoghq.com/v1/input): connection closed before message completed |
$message_level | Message with the level type for the function. Can be MDN , DEBUG , INFO , ERROR , LOG , or WARN |
$message_source | The source of the message. Can be: CONSOLE when messages are generated by the Console API, or RUNTIME when it’s related to an error message. |
$time | Request date and time. Example: 31 October, 2023, 21:10:55 |
Edge Pulse
Section titled Edge PulseDisplays the event records from requests made to your edge pulse using the performance data measured from the user’s browser.
Variable | Description |
---|---|
$browser | The UUID generated for the client’s browser used on the request. Example: 2648698a-61cb-45ad-bbb5-c569313185d4 |
$client_id | Unique Azion customer identifier. Example: 4529r |
$contentdownload | Time used to download the requested content. Example: 5 |
$dns | DNS resolution time. Example: 0 |
$downlink | Returns the average volume of data received in Mb/s. Example: 10 |
$effectivetype | The effective type of the connection. Example: 3g , 4g , 5g |
$hostname | Hostname of the current URL. Example: website.com.br |
$locationhref | The complete URL of the current page. Example: https://www.azion.com/pt-br/sobre-nos/ |
$navigation.contentDownload | Time used to download the content. |
$navigation.dns | DNS resolution time. |
$navigation.networkDuration | Duration without query browser waiting. |
$navigation.PageLoadTime | Time from the start of navigation to the full page load. |
$navigation.redirectCount | The number of redirects since the last navigation without redirection in the context of the current navigation. |
$navigation.renderTime | Time the browser was rendered for after browsing. |
$navigation.ssl | Standard protocol for an established TLS connection. |
$navigation.tcp | Internet protocol that returns the data that makes up the page. |
$navigation.ttfb | Time until the arrival of the first byte of the requested page. |
$navigation.type | Type of navigation without redirection. |
$navigation.typeBackForward | Type of navigation through the session history. |
$navigation.typeNavigate | Type of the last navigation without redirection. Example: by clicking on a link, by entering the URL in the address bar, or by submitting a form. |
$navigation.typeReload | Type of navigation for the reload operation: when the page was reloaded. |
$navigation.typeReserved | Any type of navigation not defined by other specific variables (navigation.typeNavigate and navigation.typeReload). |
$networkApi.downlink | The average volume of data received in Mb/s. |
$networkduration | Duration without query browser waiting. Example: 52 |
$pageloadtime | Time from the start of navigation until the full page load. Example: 1267 |
$platform | Operating system architecture. Example: Linux x86_64 , Iphone |
$redirectcount | Number of redirects since the last navigation without redirection in the context of the current navigation. Example: 0 |
$referrer | Address of the page the user made the request from. The URL by which the user arrived at “locationHref”. If the access originated directly from the page (for example, through a bookmark), the value will be an empty string . It doesn’t provide DOM access to the reference page. Example: https://www.azion.com/pt-br/ |
$rendertime | The amount of time it took to render the page. Example: 1242 |
$rtt | Round-Trip Time (RTT) information. Example: 250 |
$ssl | Standard protocol for an established TLS connection. Example: 0 . If the requested URL has a secure connection, it returns the time it took for authenticating. |
$tcp | Internet protocol that returns the data that makes up the page. The time it takes for the TCP handshake is the time between the connection start and the connection end. Example: 0 |
$time | Request date and time. Example: 31 October, 2023, 21:10:55 |
$timestamp | Request date and time. Example: 31 October, 2023, 21:10:55 |
$ttfb | Time To First Byte: time until the arrival of the first byte of the requested page in milliseconds. Includes 1 round trip of latency and the time the server took to prepare the response. Example: 1 |
$type | Type of navigation without redirection. Indicates how the navigation to this page/script was done. Example: navigation , reload |
$typebackforward | Type of navigation through the session history. Example: 2 |
$typenavigate | Type of the last navigation without redirection. Example: by clicking on a link, by entering the URL in the address bar, or by submitting a form. Example: 0 |
$typereload | Type of navigation for the reload operation: when the page was reloaded. Example: 1 |
$typereserved | Any type of navigation not defined by other specific variables (typeNavigate and typeReload). Example: 1 |
$userAgent | End user’s application, operating system, vendor, and/or version. Value of the User-Agent header. Example: Mozilla/5.0 (Windows NT 10.0; Win64; x64) |
$version | The Azion Log version used. Example: v5 |
WAF Events
Section titled WAF EventsRequires: Web Application Firewall
Displays the event records of requests analyzed by WAF to allow you to map the score assigned to the request, the WAF rules that matched, the reason for the block, and more.
Variable | Description |
---|---|
$attack_family | Informs the attack’s families, which are categories of attack identified by WAF and classified according to the OWASP Top 10. Example: $XSS . See the categories |
$blocked | Informs whether the WAF blocked the action or not. 0 when action wasn’t blocked and 1 when action was blocked. When in Learning Mode, it won’t be blocked regardless of the return. |
$client_id | Unique Azion customer identifier. Example: 4529r |
$geoloc_country_name | Remote client’s country detected via IP address geolocation. Example: United States , Russian Federation |
$host | Host information sent on the request line. Stores: host name from the request line, or host name from the Host request header, or the server name matching a request. Example: website.com.br |
$remote_address | IP address of the origin that generated the request. Example: 54.233.153.15 |
$server_protocol | The connection established protocol. Example: HTTP/1.1 , HTTP/2.0 , HTTP/3.0 |
$time | Request date and time. Example: 31 October, 2023, 21:10:55 |
$total_blocked | Informs the total number of requests blocked by WAF. Example: 2 |
$total_processed | Informs the total number of requests processed by WAF. Example: 1 |
$waf_action | Reports WAF’s action regarding the action. Can be: $BLOCK , $PASS , $LEARNING_BLOCK , or $LEARNING_PASS . |
$waf_args | The request arguments. Example: quantidade_periodos=10 |
$waf_learning | Informs if WAF is in Learning mode. Can be 0 or 1 |
$waf_match | List of infractions found in the end user’s request. It’s formed by key-value elements: the key refers to the type of violation detected; the value shows the string that generated the infraction. Example: 0:1311:BODY:ctl00_cph_jp1_dados_container_clientstate . Find out more on WAF Allowed Rules |
$waf_score | Reports the score that’ll be increased in case of a match with the rules set for the WAF. Example: 0:$SQL:2 |
$waf_server | Hostname used in the WAF request. Example: api-login.azion.com.br |
$waf_uri | URI used in the WAF request. Example: /access/v2/after-login |
Time filter
Section titled Time filterThe Time filter allows you to refine the period for the events record search result. It’s selected by default for Last 15 minutes.
You can filter by:
- Last 15 minutes
- Last 30 minutes
- Last 1 hour
- Last 3 hours
- Last 6 hours
- Last 12 hours
- Last day
- Last 2 days
- Last 3 days
- Last 5 days
- Last 7 days
- Custom
By using the Custom field, you can customize your search by selecting a date and time range during the last 168 hours.
You can change the time range as many times as you want to investigate your logs.
Filter by
Section titled Filter byIn the Filter by field, you filter your search results by using a keyword or phrase. This makes your search more specific and makes it easier to find the logs you want to analyze.
By submitting a search with a blank Filter by field, you’ll get all existing records for the variables of the selected data source available during the selected time filter.
The field uses SQL language to query results. Your search must be in one of two formats:
key='value'
- Exact match, where:
key
: one of the variables from the data source you’re querying for.=
: means the search must query for the exact value passed.value
: a value of either string or integer format.
key like '%value%'
- Similar value, where:
key
: one of the variables from the data source you’re querying for.like
: means the search must query for a similar value to the one passed.%value%
: a value of either string or integer format surrounded.
In the second format, you can use with value
:
%value%
: filters for values that contain the entire specified value.%value
: filters for values that end with the specified value.value%
: filters for values that begin with the specified value.
You can also search for more complex queries with the AND
, OR
, and NOT
notations to combine the fields.
Some examples of SQL queries:
Variable | SQL query |
---|---|
$status | status='404' |
$status + $scheme | status='200' AND scheme='https' |
$endpoint_type | endpoint_type='datadog' |
$geoloc_country_name | geoloc_country_name='Brazil' |
$message_content | message_content like '%unavailable%' |
$message_content | message_content like '%available%' |
Data exhibition
Section titled Data exhibitionAfter you complete the filters and search for results, your logs will appear in a table.
Each line is a different log, which equals to a different action performed by the edge. If you click on a log, the line will expand and provide more detailed information about it. The information shown varies according to the specifics of each variable.
The interface paginates for a given amount of results at first, but it’ll continue to load all available results as you scroll through the page.
Refresh
Section titled RefreshAfter performing a query, you can use the Refresh button to update the returned data. Real-Time Events will repeat the last search performed, updating the data but keeping the time filter and the SQL filter you used.
If you had the Last 30 minutes time filter, for example, and it was 4:00, you’d have logs from 3:30-4:00. If you use the Refresh button at 4:45, you’ll have logs from 4:15 P.M-4:45.
The search continues to return the results ordered by most recent to oldest.
Limits
Section titled LimitsDepending on the size of your data, the query limit may exceed. If this happens, filter by a short time filter.
Contributors