# Troubleshoot Applications Use this page when an [application](/en/documentation/platform/applications/) does not handle requests the way you configured it, or when the Azion API refuses an object that belongs to it. Each section names one symptom, its cause, and its fix, and reads on its own. The application's own symptoms come first, then those of [Cache](/en/documentation/platform/applications/how-it-works/#cache), [Application Accelerator](/en/documentation/platform/applications/how-it-works/#application-accelerator), and [Image Processor](/en/documentation/platform/applications/how-it-works/#image-processor), the Products enabled on an application. --- ## Requests never reach the origin Clients send requests to the domain of a workload, and your origin logs none of them. Two links connect a request to an origin. The workload's deployment names the application, and a Request Phase rule with **Set Connector** names the [connector](/en/documentation/platform/connectors/) that holds the origin's address, since the application record has no origin field. - **Name the application on the workload's deployment**: in Azion Console, select it in the **Application** field of the workload's **Deployment Settings**, or pass its ID to Azion CLI in `--application-id`: ```bash azion create workload-deployment --workload-id --name \ --application-id --strategy-type default --active true --current true ``` The CLI prints `Created Workload Deployment with ID `. - **Add a rule that sends requests to the connector**: a [new application](/en/documentation/platform/applications/how-it-works/#what-a-new-application-has) has no rules. Create one with **Set Connector** in Azion Console, as [Applications quickstart](/en/documentation/platform/applications/quickstart/) shows, or with `set_connector` through the [API](/en/documentation/platform/applications/rules-engine/#api). - **Give the change time to propagate**: send the request again until the origin receives it, for the times that [Propagation](/en/documentation/platform/applications/how-it-works/#propagation) gives. Requests that match the rule then reach the origin through the connector, and the origin's responses return to the client. --- ## The origin returns an error Your origin answers a request with a `4xx` or `5xx` status, and the client receives an error page. The connector that a **Set Connector** rule names receives the origin's error. The error page is set on the workload, not on the application: its deployment selects the page beside the application and the firewall. - **Serve your own error page**: create it in [Custom Pages](/en/documentation/platform/workloads/#custom-pages) and select it in the **Custom Page** field of the workload's **Deployment Settings**, `strategy.attributes.custom_page` in the API. - **Serve the last stored copy instead of a `5xx`**: turn on [Stale cache](/en/documentation/platform/applications/cache/expiration-and-freshness/#stale-cache) in the cache setting that a **Set Cache Policy** rule applies to the path. A client then receives your page for an origin error, or the expired copy of a cached object within the stale window. --- ## A rule does not act on the requests you expect A rule is saved, and requests you meant it to catch keep the previous behavior. A rule acts only on the requests its criteria match, and only after the change propagates. Check three causes: - **The rule has not propagated yet**: the API accepts a rule with `202` and `state` set to `pending`. Send the request again before you change the rule, for as long as [Propagation](/en/documentation/platform/applications/how-it-works/#propagation) describes. - **The criteria do not match the request**: [Criteria](/en/documentation/platform/applications/rules-engine/#criteria) lists each variable, operator, and conditional. - **An earlier rule ended the processing**: a behavior such as **Finish Request Phase** stops the behaviors and rules after it, as [How rules run](/en/documentation/platform/applications/how-it-works/#how-rules-run) explains. To tell the three apart, check which rules ran on the request. **Debug Rules** logs the Rules Engine rules that run on each request, and it is off on a new application. - **Turn on Debug Rules**: on the **Main Settings** tab of the application, turn on **Debug Rules** and select **Save**. - **Read the rules that ran**: [Debug Rules](/en/documentation/platform/applications/main-settings/#debug-rules) names the field that lists them in each log. A rule missing from the log of a request did not run on it: rework its criteria, or move it ahead of the rule that ended the processing. The rule then appears in the log of each request it matches, and its behaviors run. --- ## A problem shows in traffic but not in the responses you request Users report errors or slow responses, yet each request you send to the application returns what you expect. A request you send shows one response, from one server, at one moment, as [Debug headers](/en/documentation/platform/applications/cache/cache-keys/#debug-headers) shows. A problem that touches some servers, some paths, or some hours does not show in a single response. - **Read aggregated metrics**: [Real-Time Metrics](/en/documentation/platform/real-time-metrics/build-dashboards/#applications) charts aggregated data for your applications, and keeps it over longer storage periods. - **Read the raw log of each request**: [Real-Time Events](/en/documentation/platform/real-time-events/) holds the raw data of every request. - **Query only the data you need**: the GraphQL API returns [aggregated and raw data](/en/documentation/devtools/graphql/features/#datasets), limited to the fields a query asks for, as [Query usage data from Applications](/en/documentation/guides/platform/observability/query-applications-usage-data-with-graphql/) shows. - **Trace the rules that ran**: [Debug Rules](/en/documentation/platform/applications/main-settings/#debug-rules) logs them per request. The metrics and logs then show which requests, paths, or periods carry the problem. --- ## A rule is refused with Missing Required Modules Saving a request rule that reads `${request_uri}` fails with `400` and code `25047` while Application Accelerator is off, as [Errors](/en/documentation/platform/applications/rules-engine/#errors) shows in full. Some variables and behaviors belong to a Product, and the API refuses a rule that uses one while that Product is off: `meta.missing_required_modules` names it. - **Match on `${uri}` instead**: it needs no Product, so the same rule returns `202`, as [Variables](/en/documentation/platform/applications/rules-engine/#variables) lists for every variable. - **Turn on the Product the error names**: its switch is in the **Modules** section of the application's **Main Settings** tab, and **Save** applies it. [Products](/en/documentation/platform/applications/main-settings/#products) gives its Azion CLI flag and API key. With Application Accelerator on, the rule on `${request_uri}` is accepted. --- ## A behavior is refused with Invalid Choice A rule body copied from an older example fails with `400`, code `10039`, and `Invalid Choice`, which [Errors](/en/documentation/platform/applications/rules-engine/#errors) lists with its detail and pointer. The API accepts only the behavior types it defines. The one that adds a header to the request is `add_request_header`, **Add Request Header** in Azion Console, and not `add_header`. - **Send `add_request_header`**: carry the header as `name: value` in `attributes.value`, as in `{ "type": "add_request_header", "attributes": { "value": "Accept: image/webp" } }`. - **Look up the other types**: [Behaviors](/en/documentation/platform/applications/rules-engine/#behaviors) lists every behavior and its type. The rule is then accepted and reads back with the type `add_request_header`. --- ## A WebSocket connection does not return 101 Switching Protocols A WebSocket connection through the application returns a status other than `101 Switching Protocols`, even a `2xx` or a `3xx`. Only `101` means an upgraded connection: any other status means that the client, the application, or the origin did not complete it. - **Confirm native WebSocket support**: the client and the origin both need it, as [Client and origin support](/en/documentation/platform/applications/websocket/#client-and-origin-support) requires. - **Send both upgrade headers**: `Upgrade: websocket` and `Connection: upgrade`, as [Upgrade headers](/en/documentation/platform/applications/websocket/#upgrade-headers) requires. - **Confirm the account can use WebSocket Proxy**: [Availability](/en/documentation/platform/applications/websocket/#availability) names who has access. The connection then returns `101 Switching Protocols`. --- ## Cache Every new application has Cache on, yet a cache setting changes only the requests that a rule applies it to. For how Cache answers a request, refer to [How Applications works](/en/documentation/platform/applications/how-it-works/#cache). ### A saved cache setting has no effect and responses stay MISS `x-cache` reads `MISS` on each response for the path, and your origin gets as many requests as before. A saved cache setting acts only on the requests that a propagated Request Phase rule applies it to through **Set Cache Policy**. - **Apply the setting with a rule**: add **Set Cache Policy** with the setting to a Request Phase rule that covers the path, as [Create a cache setting](/en/documentation/guides/application-performance/cache-and-purge/tune-cache-settings/) shows. - **Wait for the rule to propagate**: [Propagation](/en/documentation/platform/applications/how-it-works/#propagation) gives the time it takes. - **Read the status of several requests**: each one can come from a different server. [Check the cache status of a response](/en/documentation/guides/application-performance/cache-and-purge/check-page-cache-time/) shows how to read it. With the rule in effect, the request that stores the copy reports `MISS`, and later responses served from that copy report `HIT`. ### The x-cache header reports a dash on every response for a path Both debug headers hold a dash, `x-cache: -` and `x-cache-key: -`, so the response has neither a cache status nor a key. The dash marks content restricted from caching, as [Cache status](/en/documentation/platform/applications/cache/cache-keys/#cache-status) lists. `GET` and `HEAD` responses are cached natively, while `POST` and `OPTIONS` responses need Application Accelerator and a cache setting that varies by their method. - **Turn on Application Accelerator first**: with it off, the API refuses the method field with error `21013`. [Products](/en/documentation/platform/applications/main-settings/#products) has the switch. - **Cache the method**: select it in **Cache vary by Method**, in the **Application Accelerator** section of the cache setting, which then reads back as `"cache_vary_by_method":["post"]` for *POST*. [Cache POST and OPTIONS responses](/en/documentation/guides/application-performance/cache-and-purge/cache-post-and-options-responses/) has the steps. - **Tell the cases apart by method**: on a `GET` or a `HEAD`, the dash comes from elsewhere, such as a response that a function builds. Responses to the cached method then carry a status, and a key that ends with an MD5 hash of the request body. ### Content is still served from cache after a Bypass Cache rule Requests that a **Bypass Cache** rule matches keep getting a stored copy, and the origin does not see them. **Bypass Cache** reaches Azion's cache but not the [Tiered Cache](/en/documentation/platform/applications/cache/tiered-cache/#bypass-cache) layer, which keeps objects for the minimum TTL, so the first layer can still answer from it. - **Turn off Tiered Cache**: in the **Cache** section of the cache setting that serves the path. - **Purge both layers, the tiered one first**: purge Tiered Cache by cache key, then Cache, so the first layer cannot refill from a stale copy, as [Purge cached content](/en/documentation/guides/application-performance/cache-and-purge/purge-cached-content/) shows. Requests that the rule sends to the origin then carry `x-cache: BYPASS`. ### A purge did not expire a variation of the object The purge was accepted, yet some visitors, or some query strings, keep getting the old content. A URL purge maps each URL to one cache key without variation, so copies that vary by cookie, device group, or image format survive it, however often you repeat it. It reaches a query string variation only when the URL lists the arguments in the order the key stores them, alphabetical when **Sort** is on. - **Purge by cache key**: list every variation of the object, at most 50 keys in one request, each copied from the `x-cache-key` header of its response. - **Purge by wildcard**: end the expression with `@@*` for the variations that add the `@@` separator, such as cookie and device group, or with `?*` for query string variations. On its next request, each purged variation reports `x-cache: MISS`, as [Purge content that varies](/en/documentation/platform/applications/cache/real-time-purge/#purge-content-that-varies) shows for each kind of variation. ### A Max Age below 60 seconds is refused with error 21021 Creating or updating a cache setting with a **Max Age** below 60 seconds returns `400` and code `21021`, as [Errors](/en/documentation/platform/applications/cache/cache-settings/#errors) details. Without Application Accelerator, the floor is 60 seconds, with Tiered Cache on or off. With it, a **Max Age** of `30` or of `0` saves, unless Tiered Cache is on, whose floor is 3 seconds. - **Turn on Application Accelerator**: its switch is in **Main Settings** › **Modules**. - **Keep it off and raise the value**: a **Max Age** of 60 seconds or more saves without it. The cache setting is then accepted, within the bounds that [Applications limits](/en/documentation/platform/applications/limits/#cache) lists. ### Tiered Cache cannot be turned on and the API returns 21001 or 21020 A cache setting that sends `tiered_cache.enabled` as `true` is refused with `400`: code `21001` answers the `honor` cache behavior, and code `21020` a **Max Age** below 3 seconds, as [Errors](/en/documentation/platform/applications/cache/cache-settings/#errors) details for each. Tiered Cache needs the *Override cache behavior*, `override` in the API, and a **Max Age** of at least 3 seconds. With Application Accelerator off, the 60-second floor of error `21021` applies first, so 3 to 59 seconds is still refused. - **Send the behavior and the Max Age in one request**: `modules.cache.behavior` set to `override`, with a `max_age` of at least 3, or of at least 60 with Application Accelerator off. - **Use `--file` in Azion CLI**: no CLI flag sets the cache behavior, so `--tiered-caching-enabled` on its own always ends in `21001`. This command writes the body to a file and creates the setting from it: ```bash cat > cache-setting.json <<'EOF' {"name":"tiered-from-file","modules":{"cache":{"behavior":"override","max_age":600,"tiered_cache":{"enabled":true,"topology":"nearest-region"}}}} EOF azion create cache-setting --application-id --file cache-setting.json ``` The CLI prints `Created Cache Settings configuration with ID `, and the setting reads back with `enabled` set to `true` under `tiered_cache`. For the topologies, refer to [Tiered Cache](/en/documentation/platform/applications/cache/tiered-cache/). ### A cache setting cannot be deleted and the API returns 21014 A `DELETE` on a cache setting answers `400` and code `21014`, which [Errors](/en/documentation/platform/applications/cache/cache-settings/#errors) lists with its detail. The **Set Cache Policy** behavior of some rule still names the setting, and a setting in use cannot be removed. - **Release the setting**: on the **Rules Engine** tab, point that rule's **Set Cache Policy** at a different setting, or delete the rule. - **Delete it again**: with Azion CLI, the delete takes the application ID and the setting ID: ```bash azion delete cache-setting --application-id --cache-setting-id --format json ``` With no rule naming the setting, the CLI prints `{"message": "Caches settings configuration was successfully deleted"}`, and the setting leaves the **Cache Settings** list. ### A purge request is rejected with 30001, 30003, or 30005 A purge comes back `400` rather than `201`, with one of three codes that [Errors](/en/documentation/platform/applications/cache/real-time-purge/#errors) lists with its detail: | Code | Title | Cause | Fix | | ------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | `30001` | `Invalid Purge Layer For Purge Type` | `layer` is `tiered_cache` on a purge by URL or by wildcard | Purge by cache key to reach the Tiered Cache layer, or change `layer` to `cache` | | `30003` | `Unauthorized Domain` | An item names a domain outside the account. A purge only reaches domains that belong to the account sending it | Fix the domain in the item | | `30005` | `Invalid Purge Cachekey` | A cache key purge carries an item that is not a valid key | Copy the key from the `x-cache-key` header as it is, without a scheme separator or spaces | Azion CLI reports `30001` with the detail of the API, as this URL purge on the Tiered Cache layer shows: ```bash azion purge --urls "https:///" --layer tiered_cache ``` The command prints `Error: ["Invalid purge layer for purge type \"url\"."]`. A purge by cache key on the same layer prints `Purge carried out successfully`, and accepted purges return `201` with `state` set to `executed`, as [Request body](/en/documentation/platform/applications/cache/real-time-purge/#request-body) shows. ### Azion CLI prints an empty error object A cache setting `create` or `update` fails, and under `--format json` Azion CLI prints only `{"error": {}}`. With JSON output, Azion CLI 4.23.0 loses the code and the title that the API answered with. - **Repeat the command without `--format json`**: plain output carries the message of the API, such as `Error: Failed to create the Cache Settings configuration: ["It's not possible to use this edge cache behavior while using Tiered Cache."]` for Tiered Cache with `honor`. - **Look the message up**: on this page, or in [Errors](/en/documentation/platform/applications/cache/cache-settings/#errors), which lists every code of the cache settings endpoint. Correct the request with that reason, and send it again. --- ## Application Accelerator Application Accelerator decides three things on an application: what goes into a cache key, which HTTP methods get cached, and which Rules Engine behaviors a rule may use. It is off on a new application. For how a variation changes the key, refer to [How Applications works](/en/documentation/platform/applications/how-it-works/#application-accelerator). ### A POST request reports no cache status A `POST` reports `-` in `X-Cache` until Application Accelerator is on and the cache setting names *POST* in **Cache vary by Method**, a field that [HTTP methods](/en/documentation/platform/applications/application-accelerator/settings/#http-methods) lists for each interface. For the fix, refer to [The x-cache header reports a dash on every response for a path](/en/documentation/platform/applications/troubleshooting/#the-x-cache-header-reports-a-dash-on-every-response-for-a-path). ### Cookie and device group variations survive a purge These variations come from the **Application Accelerator** section of the cache setting, and each one is an object under its own key, which a URL purge never reaches. For the purge by cache key or by the `@@*` wildcard that does, refer to [A purge did not expire a variation of the object](/en/documentation/platform/applications/troubleshooting/#a-purge-did-not-expire-a-variation-of-the-object). ### A URL purge misses a query string variation A URL purge reaches a query string variation only when it lists the arguments the key varies by, in the order the key stores them, which is alphabetical when **Sort** is on in **Cache vary by Query String**. For that purge and the `?*` wildcard, refer to [A purge did not expire a variation of the object](/en/documentation/platform/applications/troubleshooting/#a-purge-did-not-expire-a-variation-of-the-object). ### A behavior or a variable cannot be selected in Rules Engine On the **Rules Engine** tab of an application, a behavior such as **Bypass Cache**, **Forward Cookies**, or **Rewrite Request** cannot be added to a rule, and the `${device_group}` variable is missing from the criteria. Seven behaviors and the `${device_group}` variable stay locked while Application Accelerator is off on the application. - **Turn on Application Accelerator**: its switch is in **Main Settings** › **Modules**. - **Turn on Functions for Run Function**: **Run Function** also needs [Functions](/en/documentation/platform/functions/) on the application, as [Run Function is missing from the behaviors list](/en/documentation/platform/functions/troubleshooting/#run-function-is-missing-from-the-behaviors-list) explains. A rule can then take the seven behaviors, and its criteria can use `${device_group}`, as [Rules Engine behaviors](/en/documentation/platform/applications/application-accelerator/settings/#rules-engine-behaviors) lists by phase. ### Content is served from cache with Bypass Cache configured **Bypass Cache** needs Application Accelerator, and a rule that applies it can still be answered from the Tiered Cache layer, which [Bypass Cache](/en/documentation/platform/applications/cache/tiered-cache/#bypass-cache) leaves untouched. The rule works, so changing it fixes nothing: turn off Tiered Cache and purge both layers, as [Content is still served from cache after a Bypass Cache rule](/en/documentation/platform/applications/troubleshooting/#content-is-still-served-from-cache-after-a-bypass-cache-rule) describes. ### A cache variation is refused with error 21013 Saving a cache setting that sets a field of its **Application Accelerator** section returns `400` and code `21013`, as [Errors](/en/documentation/platform/applications/cache/cache-settings/#errors) details. Those fields, under `modules.application_accelerator`, vary the cache by method, query string, cookie, and device group, and each one needs Application Accelerator. - **Turn on Application Accelerator, then save the setting again**: the switch is in **Main Settings** › **Modules**. - **Read the reason when Azion CLI hides it**: with `--format json`, Azion CLI 4.23.0 prints this refusal as `{"error": {}}` for a setting with a query string allowlist, as [Azion CLI prints an empty error object](/en/documentation/platform/applications/troubleshooting/#azion-cli-prints-an-empty-error-object) explains. The setting then saves, and its variation reads back as sent. ### A query string variation is refused with error 21018 A cache setting whose query string behavior is *Allowlist* or *Denylist*, sent with an empty list of fields, returns `400` and code `21018`, as [Errors](/en/documentation/platform/applications/cache/cache-settings/#errors) details. Those behaviors name the fields the key varies by, so each needs at least one, and Azion Console makes **Fields** required. - **List the fields**: enter at least one query string parameter in **Fields**, one per line, or send them in `fields` through the API. - **Keep the default when the key should not vary**: *Ignore*, the default behavior, saves with an empty `fields` list. The setting then saves, with `cache_vary_by_querystring` holding what you sent, as [Cache variation](/en/documentation/platform/applications/application-accelerator/settings/#cache-variation) lists. --- ## Image Processor Image Processor builds a derived image from the source image when a rule applies **Optimize Images**, and it is off on a new application. For the path from the source image to the derived image, refer to [How Applications works](/en/documentation/platform/applications/how-it-works/#image-processor). ### A 504 error on a request whose ims is not the last parameter A request for an image returns a `504` error, and its URL has another query string parameter after `ims=`. `ims=` has to come last among the parameters of the URL: a parameter after it may make the request return a `504`, though not every such request fails. - **Put `ims` last**: move the other parameters ahead of `ims=`, as in `example.com/image.jpeg?ts=1234&ims=1000x1000`. For the rule, refer to [Position of the ims parameter](/en/documentation/platform/applications/image-processor/url-parameters/#position-of-the-ims-parameter). With `ims=` last, the URL returns the derived image, not a `504`. ### The image is delivered unprocessed and its ims query string is ignored The `ims` query string of a request is valid, yet the source image comes back as it is, without a resize, a crop, or a filter. An `ims` query string does not start a transformation by itself: a rule has to match the request and apply **Optimize Images**. - **Check what the rule matches**: on the **Rules Engine** tab, compare the rule's criterion with the request. A criterion for images compares `${uri}`, or `${request_uri}` with Application Accelerator on, through `matches` with an argument such as `\.(jpg|jpeg|gif|bmp|png|ico|webp|avif)`. - **Check the behavior**: **Optimize Images** has to be a Request Phase behavior of the rule, beside **Set Cache Policy** when the rule also caches the result, as [Rules Engine behaviors](/en/documentation/platform/applications/image-processor/settings/#rules-engine-behaviors) describes. - **Follow the guide**: [Configure Image Processor on an application](/en/documentation/guides/application-performance/delivery-optimization/process-images/) builds the rule step by step. A request that matches the rule then gets a derived image, with `x-ims: Enabled` in the response headers. ### The image is not converted to WEBP You request a WEBP conversion, and the image arrives in another format. A WEBP conversion happens only for a request whose `Accept` header names the format, `Accept: image/webp`, as [Format conversion](/en/documentation/platform/applications/image-processor/url-parameters/#format-conversion) requires. - **Set the header from a rule**: give the rule an **Add Request Header** behavior in its Request Phase, with the value `Accept: image/webp`, `add_request_header` in the API. For the behavior, refer to [Image Processor settings](/en/documentation/platform/applications/image-processor/settings/#rules-engine-behaviors). Matching requests then get the image back as `image/webp`. ### Every request for the same derived image is processed again Each request for the same image with the same `ims` value adds a processed image to the Images meter of the account, instead of coming from cache. **Optimize Images** processes the image, and keeping the result takes a cache setting that a rule applies with **Set Cache Policy**. Varying that setting by `ims` is a query string variation, so it needs Application Accelerator, which processing does not. - **Apply a cache setting to the image requests**: add **Set Cache Policy**, with the cache setting that serves images, to the rule that applies **Optimize Images**. - **Add ims to the query string allowlist**: in **Cache vary by Query String**, in the **Application Accelerator** section of that setting, choose *Allowlist* as the **Behavior** and enter `ims` in **Fields**, as [Cache variation](/en/documentation/platform/applications/image-processor/settings/#cache-variation) describes. Read back, the cache setting then holds `{ "behavior": "allowlist", "fields": ["ims"], "sort_enabled": false }` in `cache_vary_by_querystring`. ### A resize larger than the original does not enlarge the image You ask for `fit-in/WidthxHeight`, or for dimensions beyond those of the source image, and the derived image comes back no bigger than the original. The resize works as designed, and no setting changes it. `fit-in` does not scale an image up, and without `fit-in`, a request for more resolution than the original has returns the highest resolution the source allows. The result keeps the size of the source image, or takes the largest size that fits the requested box, as [Resize](/en/documentation/platform/applications/image-processor/url-parameters/#resize) shows for every form. ### A rotation value has no effect With `?ims=filters:rotate(Angle)` set to an angle other than `0`, `90`, `180`, `270`, or `360`, the derived image keeps its orientation. Rotation accepts only those five angles, and any other value leaves the image unrotated. - **Pick an accepted angle**: [Rotation](/en/documentation/platform/applications/image-processor/url-parameters/#rotation) gives the syntax of the filter. The derived image is then turned to the left by the angle you requested. --- ## Related resources - [Rules Engine for Applications](/en/documentation/platform/applications/rules-engine.md): Each variable, operator, and behavior of a rule, and the errors returned when a rule is refused. - [Main Settings](/en/documentation/platform/applications/main-settings.md): The Product switches, the Debug Rules setting, and the errors an application write returns. - [WebSocket Proxy](/en/documentation/platform/applications/websocket.md): The upgrade headers, the support a client and an origin need, and the status a connection returns. - [Real-Time Purge](/en/documentation/platform/applications/cache/real-time-purge.md): The purge types, the two layers, and the purge that reaches each kind of variation.