Azion Bot Manager

Azion Bot Manager uses advanced intelligent algorithms that allow machine learning and Reputation Intelligence to analyze the behavior of incoming data. This enables the detection of suspicious traffic and bad bots, facilitating the implementation of preventive measures against malicious activities such as credential stuffing, vulnerability scanning, and site scraping.

This Edge Firewall add-on assigns a score to every request based on rules, behaviors, and Reputation Intelligence. If the score exceeds a predetermined threshold, Bot Manager executes the predetermined action.

By using Bot Manager, you can:

Enhance user experience
- Reducing the impact of bots on the entire infrastructure.
- Providing bot protection by IP reputation.
- Defining custom rule management to act on individual bots based on previous content extraction activity from requests.
Increase visibility
- Measuring the amount and characteristics of the bot traffic trying to access your website, APIs, and applications.
- Using the observability tools provided by Azion to monitor the malicious activity.
- Combining other integrations to enhance Bot Manager efficiency, through the use of fingerprint, captchas, JavaScript injection, or SDKs, to create robust rules.
Reduce financial risk
- Protecting your website and applications against credential abuse, card balance verification, and other forms of online fraud.

Implementation

Azion provides the Bot Manager add-on, a comprehensive solution for bot management. Contact the Sales team for more details on the Bot Manager subscription.

Additionally, a lite version is also available in the Marketplace.

How Azion Bot Manager works

On a high-level, Azion Bot Manager works this way:

A request reaches a domain using Bot Manager.
Edge Firewall receives the request.
Bot Manager starts all the analytics processes, including:

Retrieving the requested data, including device, browser, and network data, fingerprint, among others.
Identifying and classifying the request according to advanced intelligent algorithms and Reputation Intelligence.
Defining the behavior according to the rules engine criteria.

Bot Manager assigns a score to the request.
If the score is equal to or higher than the predetermined threshold, the predefined preventive action is executed.

Bot Manager is able to execute 7 different actions whenever the request’s score is greater or equals than the defined threshold:

allow: allows the continuation of the request.
deny: delivers a standard Status Code 403 response.
drop: terminates the request without a response to the user.
redirect: allows the request to be redirected to a new URL/location when the security threshold is reached.
custom_html: allows customized HTML content to be delivered to the user in case of a threshold violation.
random_delay: makes the function wait for a random period between 1 and 10 seconds before allowing the request to proceed.
hold_connection: holds the request, keeping the connection open for 1 minute before dropping it.

All these actions can be configured for web and mobile applications, as well as APIs, offering protection in different environments.

Use cases

Bot Manager was developed by Azion to attend to use cases involving common practices of malicious bots and traffic.

Reputation Intelligence
Bot attacking
Account takeover
Credential stuffing
Vulnerability scanning
Brute force attacks
Web scraping

Features

Azion Bot Manager is composed by different features that you can take advantage of.

Reputation Intelligence

By using Reputation Intelligence, Bot Manager establishes an additional security perimeter, cataloging the inbound and outbound traffic, based on Network Lists, maintained and constantly updated by Azion. Through these lists, Bot Manager is able to identify the profile of each request trying to reach your site.

Network Lists used by Bot Manager include criterias such as:

Tor Exit Nodes
Reputation
Proxies
Malware
Fraud

Go to network lists reference

Bot classification

Based on the scores and Reputation Intelligence, Bot Manager is able to classify different types of bots and traffic.

Legitimate traffic: This category is assigned to requests that don’t match either “good bot” or “bad bot” criteria. It typically refers to regular user traffic (non-bots) showing legitimate access patterns without signs of automation or suspicious behavior.
Good bots: Identified based on known and trusted User-Agents. These bots are classified as “good” when their User-Agent matches an allowed bot category, like the following ones:
- Social network bot
- Monitoring bot
- Aggregator bot
- Enterprise bot
- Search engine bot
Bad Bots: Classified when identifying suspicious or malicious User-Agents, missing or unusual header patterns, and anomalous behaviors like automation attempts. Bad bots fail to meet “good bot” criteria and show signs of malicious activities.

The following table describes the possible classification values and categories (attack types) of Bot Manager and how they are identified, as they can be seen in Real Time Metrics:

classified	botCategory	Identification Method
Good Bot	Good Bot	Identified by specific User-Agents associated with social networks, content aggregators, monitoring bots and search engines.
Bad Bot	Bad Bot Signatures	Detected through User-Agents known for malicious behavior. Includes checking malicious User-Agent signatures and analyzing missing or anomalous headers.
Bad Bot	Scripted Bots	Identified by suspicious User-Agents that typically indicate automation, such as “headless” or “dalvik”. Also considers unusual User-Agent length.
Bad Bot	Malicious Browser Behavior	Based on suspicious behaviors, such as missing or forged essential cookies, missing required HTTP headers, and cookie validation failures.
Bad Bot	Malicious Intent Detected	Uses checks of unusual HTTP headers and methods, like TRACE, to detect potentially malicious intent.
Bad Bot	Reputation Intelligence	Analyzes user IP addresses against known reputation lists to identify IPs with history of suspicious network activity.
Bad Bot	Brute Force	Detected based on high frequency of login attempts, IP address variations, and error patterns indicating credential discovery attempts.
Bad Bot	Scraping	Identified by high URL access variability and request frequency, indicating mass data extraction attempts.
Bad Bot	Crawling	Detected based on URL variation patterns and request frequency typical of content crawlers systematically navigating websites.
Bad Bot	Credential Stuffing	Detected by frequency of login attempts, error patterns, and multiple account access attempts typical of credential stuffing attacks.
Bad Bot	Credential Cracking	Detected by request frequency and specific error patterns indicating password cracking attempts.
Bad Bot	Account Takeover	Detected by anomalous request patterns and high geographic variation typical of account takeover attempts.
Legitimate	Non-Bot Like	Classification assigned when no suspicious behavior or bot pattern is identified.
Under Evaluation	Under Evaluation	When there is insufficient data for complete classification, traffic is placed “under evaluation” until more information becomes available.

The “Under Evaluation” classification

Due to the delay of up to 15 minutes to consolidate the score classification of a fingerprint, we can’t mark it as “legitimate” or “bad/good bot”, as this could “poison” the host’s score with imprecise data. Therefore, for the first few minutes a new fingerprint appears on a host, it’ll be classified as “under evaluation”.

This doesn’t interfere with future detections and occurs if: a) the overall traffic through a specific host ceases for more than 15 minutes; or b) the fingerprint wasn’t seen for 15 minutes or more.

Device identification

Bot Manager leverages advanced techniques to identify and distinguish between legitimate devices and potentially malicious bots attempting to access your digital assets. It’s capable of generating a user ID for each device.

To further enhance its protective capabilities, it allows for the incorporation of additional security layers through other integrations and resources, such as SDKs, JavaScript injection, and Fingerprint to collect more granular data.

Redirect

One of the actions Bot Manager is able to execute is redirect. It allows the request to be redirected to a new URL/location, specified in the JSON args, when the security threshold is reached.

Custom HTML

Bot Manager allows customized HTML content to be delivered to the user in case of a threshold violation, thanks to the custom_html action. You can create a custom message to exhibit to users in case of threshold violation.

Delayed response

This action allows for introducing delays in responses in cases where bots attempt to make requests. It increases the cost of the attack by holding the attacker for a longer time in a request that won’t return a valid response, thereby increasing the probability that the attacker will abort or give up on the attack.

Modes

Azion Bot Manager allows you to define the environment in which the function is expected to run, being API or a web application the possible modes. The default mode is web. If any value other than the string api (case-sensitive in lowercase) is provided, the web mode will be used as the default configuration.

By enabling the api mode, no Set-Cookie will be executed, and any rules related to the use of cookies in Bot Manager will be ignored.

Logs

Azion Observe products

The requests will generate logs that can be seen in Real-Time Events and Data Stream. By analyzing the logs generated by Bot Manager, you can get insights to understand if any changes in the configuration are needed.

Go to Data Stream Reference
Go to Real-Time Events Reference

You can also check the Bot Manager graphs in Real-Time Metrics. The Bot Manager tab displays metrics related to Bot Manager activity, being generated almost in real time, with an aggregation interval of up to 60 seconds. Depending on the chart, the data can be stored for up to 2 years.

The two sections, Overview and Breakdown, include charts such as:

Bad Bot Hits: the total number of requests identified as bad bots within the defined period.
Good Bot Hits: the number of requests identified as good bots within the defined period.
Bot Hits: the total number of requests identified as bots within the defined period.
Transactions: a sum referring to the total number of requests evaluated by Azion Bot Manager.
Bot Traffic: it presents historical data by identifying the traffic as legitimate, bad bot, good bot, and under evaluation.
Top Bot Azion: the actions performed by Azion Bot Manager for accesses identified as bots.
Bot CAPTCHA line and pie graphs: the results of the challenge returned for requests classified as bots.
Top Bot Classifications: the sum of requests classified according to the tactics used and the purpose of the bots.
Bot Activity Map: the geographic origin of bot attacks, with a color-based visualization to detect the places with more detected bot attacks.
Impacted URLs: the total number of distinct URLs bots request.
Top Impacted URLs: the total number of requests detected as bots, broken down by the most affected URLs.
Top Bad Bot IPs: the total number of requests detected as bad bots from different IP addresses.

The variety of charts, combined with the use of filters and variation tags, allows you to perform detailed analyses of individual bots, as well as trends and patterns in bot activity over time.

Go to Real-Time Metrics Reference
Go to Bot Manager Graphs

GraphiQL Playground

Use the GraphiQL Playground to access detailed information about Bot Manager’s performance and operations. For example, you can query Bot Manager data or query the top URLs impacted by bots to analyze bot activity, and retrieve data to optimize your configurations.

Go to GraphiQL Playground
Go to Bot Manager Fields

Report Log explained

Example of a report log generated by the Bot Manager:

[Report-Bot-Protection][er18ewiohw.map.azionedge.net] Report: {
  "action": "redirect",
  "asn": "52580",
  "azion_fingerprint": "5f852499202f7fa889157ce8b39a613b0d8feef80a7552e0aab0df2b098f8146",
  "bot_category": "Bad Bot Signatures",
  "bot_characteristics": [
    "Bad Bot Signatures"
  ],
  "bot_mode": "web",
  "bytes_sent": "0",
  "challenge_solved": false,
  "classified": "bad bot",
  "concat_headers": "accept,cdn-loop,content-length,cookie,host,user-agent",
  "disabled_matched_rules": [],
  "geoip_country": "Brazil",
  "geoip_region": "São Paulo",
  "host": "er18ewiohw.map.azionedge.net",
  "http_user_agent": "Mozilla/5.0 (X11; U; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/111.0.5505.226 Chrome/111.0.5505.226 Safari/537.36",
  "log_tag": "er18ewiohw.map.azionedge.net",
  "matched_rules": [
    10
  ],
  "persist_cookies": false,
  "remote_addr": "186.195.68.17",
  "request_headers": [
    "accept:Ki8q",
    "accept-encoding:bnVsbA==",
    "accept-language:bnVsbA==",
    "content-type:bnVsbA==",
    "host:ZXIxOGV3aW9ody5tYXAuYXppb25lZGdlLm5ldA==",
    "referer:bnVsbA==",
    "user-agent:TW96aWxsYS81LjAgKFgxMVw7IFVcOyBMaW51eCB4ODZfNjQpIEFwcGxlV2ViS2l0LzUzNy4zNiAoS0hUTUwsIGxpa2UgR2Vja28pIFVidW50dSBDaHJvbWl1bS8xMTEuMC41NTA1LjIyNiBDaHJvbWUvMTExLjAuNTUwNS4yMjYgU2FmYXJpLzUzNy4zNg==",
    "x-forwarded-for:bnVsbA==",
    "x-request-id:bnVsbA==",
    "x-requested-with:bnVsbA=="
  ],
  "request_id": "5934c01027a55541e3e8aa9b2241d8f8",
  "request_method": "GET",
  "request_uri": "/",
  "score": 8,
  "sent_http_content_type": null
}

The report log of the Bot Manager function has the following fields:

Field	Data Type	Description
action	String	Describes the action applied by the Bot Protection at the end of the execution. It will be “allow” whenever the request score doesn’t reach the maximum threshold defined in the JSON Args and no attack is identified. It will be the same value defined in the JSON Args when the request score is greater than the limit and/or an attack is identified.
asn	String	The ASN used by the request.
azion_fingerprint	String	The fingerprint Azion identified from the request.
bot_category	String	Defines the bot category which the request best fits. If the request is classified as legitimate (see more about in the classified field), then the Bot Category will be “Non-Bot Like”. If the request is identified as good bot, then the Bot Category will be the same as the good bot type the request matches. The possible values in this case are: Aggregator Bot, Enterprise Bot, Monitoring Bot, Search Engine Bot, Social Network Bot. If the request is classified as bad bot by violating the threshold limit due to too many violations in static rules, then the Bot Category will be the “Bot Characteristic” it violated the most. The possible values are: Bad bot signatures, Malicious browser behavior, Malicious intent detected, Reputation Intelligence. Finally, if the request is classified as “bad bot” by being identified as an attack, the Bot Category will be the same as the attack. The possible values are: Account Takeover, Brute Force, Crawling, Credential Cracking, Credential Stuffing, Scrapping.
bot_characteristics	Array of String	An array containing all the “static rules violation categories” and attacks the request matched (if any).
bot_mode	String	Describes the bot protection mode used in the request.
bytes_sent	Number	The Content-Length of the request.
challenge_solved	Boolean	A boolean value which informs whenever the user has solved a CAPTCHA challenge. Requires the integration of a CAPTCHA function alongside the Bot Protection.
classified	String	Defines the final classification the edge function identifies the request as. The possible values are: legitimate (when the request isn’t identified as a bot and there’s enough fingerprint data to ensure it’s not an attack), under evaluation (when the request isn’t identified as a bot, but there’s not enough fingerprint data to ensure it’s not an attack), bad bot (when the request reaches the score threshold or it’s identified as an attack), good bot (when the request isn’t identified as an attack, the User-Agent matches any of the Good User Agent patterns.
disabled_matched_rules	Array of Numbers	Defines the IDs of all disabled static rules which were matched during the request. Note that when a disabled rule is matched, it doesn’t increment the request score.
geoip_country	String	The code of the country from where the request was made.
geoip_region	String	The name of the region from where the request was made.
host	String	The host to which the request was made.
http_user_agent	String	The user-agent sent in the request.
log_tag	String	The identifier of the Bot Manager instance (defined via JSON Args).
matched_rules	Array of Numbers	Defines the IDs of all (enabled) static rules which were matched during the request.
persist_cookies	Boolean	Describes whenever the function identifies if the user’s device is persisting with the bot protection cookies or not.
remote_addr	String	The IP address which initiated the request.
request_headers	Array of Strings	Lists every header defined in the log_headers key of the JSON Args (when they are not in the forbidden list). Every header is logged in a “key value” fashion and the values are base64 encoded. Note: The array can have a maximum of 10,000 (ten thousand characters). For each key/value included, we deduct from this limit and once the limit is reached, no other key/value is included.
request_id	String	Unique identifier of the request.
request_method	String	The request method.
request_uri	String	The request URI.
score	Number	The calculated request score.
sent_http_content_type	String	The content of the Content-Type header sent in the request.

Setup Options

Configure your Bot Manager instance through the following JSON Args parameters to customize its behavior, detection capabilities, and response actions.

The table below outlines all available configuration options that can be included in your JSON Args:

Field	Data Type	Is Required	Default value	Description
action	Enum of Strings	No	`allow`	Defines an action to be taken by the function whenever the request’s score is greater than or equal to the defined threshold. The possible values for this field are: `allow`: Will let the request to continue, no matter the score received. `custom_html`: Will finalize the request with a new HTML response. This action requires two additional arguments: `custom_html`, `custom_status_code`. `deny`: Triggers the Deny behavior. `drop`: Triggers the Drop behavior. `hold_connection`: Will make the request “sleep” for a minute before continuing it. `random_delay`: Will make the request “sleep” for a random period of time (between 1 and 10 seconds) before continuing the request. `redirect`: Will redirect the request to a new location. This action requires an additional argument: `redirect_to`, which defines the location to where the request will be redirected. If an invalid value is set, the function will use the action `allow`.
custom_html	String	Yes (only if the `action` is `custom_html`)		Defines the HTML to be used in the response the function will return to users who violate the bot protection. If this key is not defined or has an invalid value, the action triggered will be `allow`.
custom_status_code	Number	No	`200`	Only used when the action `custom_html` is triggered. Defines the status code of the response returned to the users.
disable_dynamic_rules	Boolean	No	`False`	Disables the usage of the dynamic rules method.
disabled_static_rules	Array of Numbers	No	`[]`	In this field you can define rules to be disabled. When a rule is disabled, it will still be executed (and whenever the request matches it, the id will be pushed to the `disabled_matched_rules` in the edge function logs) but it won’t increase the request score.
dynamic_rules_baseline	Number	No	`0`	Is a relative fine adjustment for the calculated baseline. It implies a percentage multiplier to the values. Lower baseline means more strict detections. For example: 0.1 = increase baseline by 10% or -0.234 = decrease baseline by 23,4%.
dynamic_rules_logs_enabled	Boolean	No	`False`	When enabled, the function will write dynamic rules debugging logs. This is expected to be used for debugging purposes only.
dynamic_rules_tolerance	String	No	`soft`	Defines how strict the dynamic rules algorithm will be. The possible values are: `soft` (least strict), `medium`, `hard` (most strict). If an invalid value is set, the function will use `soft` tolerance.
log_headers	Array of Strings	No	`-`	This field defines which request headers should be included in the function’s report log. The user can define almost any request header they want to write in the logs, except any of the forbidden headers list below: `authorization`, `cookie`, `proxy-authorization`, `set-cookie`, `x-csrf-token`, `x-api-key`, `x-amz-security-token`. Note: The headers’ values will be printed with base64 encoding.
log_tag	String	No	Value of the variable: Received `host` header of the request	This field sets a tag to be used to identify the Function Instance which generated the request in the Edge Functions event log.
mode	String (possible values are `api` and `web`)	No	`web`	Defines the traffic mode: The possible values are: `web`: for cookie compatible HTTP clients such as web browsers; `api`: for traffic such as web services and API based services that don’t use cookies in their requests.
redirect_to	String (URI)	Yes (only if the `action` is `redirect`)	`-`	Only used when the action `redirect` is triggered. Defines the URL/relative path to where the Bot Manager should redirect the users. If this key is not defined or has an invalid value, the action triggered will be `allow`, as the function will not have anywhere to redirect the users.
reputation_network_lists	Array of Numbers	No	`[IDs of Network Lists configured for the account managed by Azion]`	Defines every Network List that should be used to validate the request IP. If the request IP is found in a list, then the request score will be increased. If the request IP is found in multiple lists, the score will be increased multiple times.
session_signature_key	String	No		This string is used to sign the `az_asm` cookie value. When this field has no value or an invalid value, the function will use the default value `azion`.
threshold	Number	No	`Infinity`	Defines a maximum score the request can reach before the function takes an action. When set to `0`, it will cause the Bot Manager’s `action` to always be triggered (except if the user has already solved the ALTCHA Challenge, if ALTCHA is in use). When no value is set, the threshold will be defined as `Infinity`, therefore, all requests will always be allowed to pass.

Additional resources

SDKs

Azion Bot Manager can work together with Software Development Kits (SDKs), for both Android and iOS systems, allowing you to customize and tailor security protocols to meet the specific needs of your mobile applications. With SDKs and Bot Manager, you can implement fine-grained controls, address application-specific vulnerabilities, and adapt to evolving threats more effectively.

You can use SDKs to track mobile devices and identify behaviors (such as touching the screen) and device data (model, manufacturer, operational system, etc.) to use as insights for Bot Manager detect and mitigate malicious threats.

JavaScript injection

When the JavaScript file is inserted in your edge application, it collects data to identify the access in the request by calculating it’s fingerprint. It’s available for use with web browsers. With JavaScript injection, more non-sensitive data will be collected from the user, but exclusively for fingerprinting identification of the device.

This data can be used to create more robust rules and behaviors on the Bot Manager args in order to detect and mitigate threats more effectively.

Rate Limiting

Rate limiting integration establishes thresholds for the number of requests a user or system can make within a specified timeframe, effectively mitigating the impact of brute force attacks or excessive bot activities. By working jointly with rate limiting, the bot management measures gain an additional layer of defense against automated threats.

Fingerprint

A set of information (IP, User-Agent header) creates a hash for devices accessing your edge applications. The information is gathered by tracing the device’s session and provides a more accurate detailing of the request’s device, increasing the precision of Bot Manager logs.

If you use Fingerprint with Bot Manager, you can also enable the use of Azion Real-Time Metrics to query consolidated data via GraphQL API related to the access to the application protected by Bot Manager, facilitating the identification of patters and use this intelligence to optimize the rules. With this feature, you can define a threshold and take a specific action when the threshold is violated and the device or user is identified as malicious, based on the fingerprint data.

Captcha

By using the redirect action, the defined URL/location can contain a Captcha integration to add an additional security layer. It helps you to increase security and malicious traffic detection, challenging all the request previously violating any threshold to guarantee is legitimate.

Custom rules

Azion will provide you with easy-to-go configurations, that should be enough for most of the cases. If you need a more detailed configuration, you can add new custom rules based on your business needs. It’s also possible to add more criteria and behaviors to be executed by the Rules Engine, building more comprehensive responses to possible attacks.