# Troubleshoot Workloads This page lists the symptoms a [workload](/en/documentation/platform/workloads/) shows on live traffic or in a refused request, each with its cause and its fix. The workload's own symptoms open the page: responses that are missing or out of date, and refused domains, settings, deployments, and certificates. Sections for [Certificate Manager](/en/documentation/platform/workloads/#certificate-manager), which issues and stores the workload's certificates, and [Custom Pages](/en/documentation/platform/workloads/#custom-pages), which replaces its error responses, close it. A quoted refusal is the API's message unless the entry names another source, and Azion CLI prints the same message inside `Error: Failed to ... [...]`. --- ## A workload answers 404 with Azion's error page Requests to the workload domain return `HTTP/2 404` and Azion's HTML error page instead of your application's response. The workload has no application to send the request to: no deployment binds one yet, or the deployment has not propagated, as [The deployment](/en/documentation/platform/workloads/how-it-works/#the-deployment) explains. A workload with **Workload Domain Allow Access** off also answers `404` on its workload domain, while its own domains keep serving. - **Create the deployment**: it names the application, as [Workloads quickstart](/en/documentation/platform/workloads/quickstart/) shows for each interface. - **Repeat the request after several minutes**: a deployment reaches traffic only once it [propagates](/en/documentation/platform/workloads/how-it-works/#propagation). - **Check the access switch**: the workload domain serves only while `workload_domain_allow_access` reads `true`. - **Read the headers**: `curl -sI https:///get` prints the status line and the `x-azion-request-id` header that identifies the request. Once the deployment propagates, the same request returns `200` from your application. --- ## A request receives the old configuration after a change After you save a change to a workload or its deployment, some requests still get the previous behavior, or the answers alternate between old and new. The change spreads across Azion's distributed infrastructure one data center at a time, over several minutes, and propagation is best-effort. Each data center serves either the old configuration or the new one until all of them agree, as [Propagation](/en/documentation/platform/workloads/how-it-works/#propagation) details. - **Wait several minutes before you test**: an early request measures propagation, not the configuration. - **Send several requests, not one**: repeat the request until the answers agree. - **Confirm the API stored the change**: `azion describe workload --workload-id --format json` prints the workload as the API keeps it. When every data center serves the new configuration, each request receives the answer that configuration defines. --- ## A custom domain does not reach the workload Requests to a hostname of your own do not reach the workload, while its workload domain answers. A workload answers only the hostnames listed in its domains, and a client reaches it only after DNS sends the hostname to Azion. Both parts are needed, as [Workload domain and custom domains](/en/documentation/platform/workloads/how-it-works/#workload-domain-and-custom-domains) explains. - **List the hostname on the workload**: add it to `domains`, or as a **Subdomain** and **Domain** row of the **Domains** section, as [Add a custom domain to a workload](/en/documentation/guides/platform/migration/configure-a-domain/) shows. - **Point the hostname at the workload domain**: add a CNAME record at your DNS provider, or host the zone in [Edge DNS](/en/documentation/platform/edge-dns/), as [Point a domain to a workload](/en/documentation/guides/platform/migration/point-domain-to-azion/) shows. - **Give resolvers time**: visitors reach the workload once their resolvers pick up the new record. - **Use a production workload**: a staging workload takes no custom domain. The hostname then answers with the same content as the workload domain. --- ## A domain is refused with This account is not allowed to use the following CNAMEs Saving the workload's domains fails with `This account is not allowed to use the following CNAMEs: example.com.`, where the message names each refused domain. The account has no permission for a domain in `domains`, and the API refuses the whole request, as the [Workload settings](/en/documentation/platform/workloads/settings/#errors) errors show. - **Remove the domain the message names**, and save the rest of the list. - **List only domains your account has permission for**. Azion CLI then prints `Updated Workload with ID `, and the workload answers on the domains it kept. --- ## A custom hostname is refused on a staging workload Adding a domain to a workload fails with `Custom hostname is not available in the environment 'Staging Infrastructure'.` A staging workload, with `infrastructure` set to `2`, takes no custom hostname, an `azion.app` hostname included. It answers only on its `.preview.azionedge.net` workload domain. - **Test on the workload domain**: send `"domains": []` and reach the staging workload through its workload domain. - **Use a production workload for your hostname**: a workload with `infrastructure` set to `1` accepts it. - **Preview your hostname before its DNS changes**: map it to a production workload in your device's hosts file, as [Test an application through the hosts file](/en/documentation/guides/application-development/getting-started/stage-applications-through-hosts-file/) shows. The production workload then accepts the hostname, and the staging workload keeps serving on its workload domain. --- ## An azion.app domain is refused Adding an `azion.app` hostname, which Azion Console calls **Azion Custom Domain**, fails with one of three messages. A workload holds at most one `azion.app` hostname, a name belongs to one workload at a time, and every entry must conform to RFC 1035, as the [Workload settings](/en/documentation/platform/workloads/settings/#errors) errors show. - **Keep one `azion.app` hostname per workload**: `Duplicated usage of suffix in alternate_domains: 'azion.app'.` refuses a second one, even under a different name. - **Choose a name no other workload holds**: `The custom hostname is not available.` means another workload has it, so pick another name or remove it there first. - **List the full hostname, never a wildcard**: `The domain does not conform to the format defined in RFC 1035.` refuses `*.azion.app`. The update then stores the hostname with the case you sent, and Azion serves it over HTTPS with its SAN certificate, as [Create an Azion custom domain](/en/documentation/guides/application-development/getting-started/create-azion-custom-domain/) shows. --- ## The workload domain cannot be closed Turning off **Workload Domain Allow Access** fails with `When the workload hostname access is blocked, the workload requires alternate domains or domains.` A workload needs a hostname to answer on, so `workload_domain_allow_access` can be `false` only while `domains` holds at least one entry. - **Add your own hostname or an `azion.app` hostname first**. - **Close the workload domain after your hostnames answer everywhere**, as [Workloads best practices](/en/documentation/platform/workloads/best-practices/#close-the-workload-domain-once-your-own-domain-serves-traffic) explains. The workload domain then answers `404`, while your hostnames keep serving. --- ## The infrastructure cannot be changed An update that moves a workload between staging and production fails with `The infrastructure cannot be changed after Workload creation.` The `infrastructure` value is fixed when the workload is created, and the Console help text for **Infrastructure** reads "Once this option is saved, it cannot be modified." - **Create a second workload on the other infrastructure**, with `infrastructure` set to `1` for production, and give it its own deployment. - **Leave `infrastructure` out of updates** to the existing workload. The new workload gets the workload domain suffix of its infrastructure, `.map.azionedge.net` on production, as [Infrastructure](/en/documentation/platform/workloads/how-it-works/#infrastructure) explains. --- ## A second deployment is refused Creating a deployment fails with `The maximum number of deployments allowed per workload is 1.` A workload holds one deployment, which names its application, firewall, and custom page set, so a change goes into that deployment, as [The deployment](/en/documentation/platform/workloads/how-it-works/#the-deployment) explains. - **Find the existing deployment**: `azion list workload-deployment --workload-id ` prints its `ID`, and `GET /v4/workspace/workloads//deployments` returns it. - **Edit it in Azion Console**: change **Application**, **Firewall**, or **Custom Page** in **Deployment Settings**, then select **Save**. - **Or send a `PATCH`** to `/v4/workspace/workloads//deployments/` with the changed keys of `strategy.attributes`. The `PATCH` returns `202`, and the change reaches every hostname of the workload once it propagates. --- ## A port or HTTP version change is refused An update to the workload's protocols fails with `Invalid choices for multiple choices field: [80, 8008, 8080, 8880].` or `Missing required choices for multiple choices field: ['http1', 'http2'].` The first message lists the four HTTP ports a workload accepts. The second means `versions` holds `http3` without both `http1` and `http2`. - **Use only `80`, `8008`, `8080`, or `8880` in `http_ports`**, and the HTTPS ports [Protocols and ports](/en/documentation/platform/workloads/settings/#protocols-and-ports) lists. - **Send `["http1", "http2", "http3"]`, or drop `http3`**. - **Turn on HTTPS support before HTTP/3 support** in Azion Console: **HTTP/3 support** requires **HTTPS support**. Azion CLI then prints `Updated Workload with ID `. For the other settings refusals, such as a cipher suite outside `1` to `8`, refer to the [Workload settings](/en/documentation/platform/workloads/settings/#errors) errors. --- ## A client fails the TLS handshake on an mTLS workload A client gets no response: curl exits with code `56` after the server's certificate request, and LibreSSL reports `reason(1116)`, certificate required. The workload has mTLS in `enforce` mode, and the client presented no certificate or one the Trusted CA did not sign. A client that sends no Server Name Indication (SNI) is also closed before a route of your application resolves. - **Present a certificate the Trusted CA signed**: the Trusted CA is the certificate in `mtls.config.certificate`, and curl sends a client certificate with `--cert` and `--key`. - **Send SNI from every client**: on an `enforce` workload, a connection without it never reaches your application, as [mTLS](/en/documentation/platform/workloads/mtls/#requirements) lists. - **Admit clients while you test**: `permissive` completes the handshake and leaves the decision to a firewall rule, as [Verification modes](/en/documentation/platform/workloads/mtls/#verification-modes) describes. - **Wait for a mode change to propagate**: until it does, some data centers answer with the previous mode. The handshake then completes, and the request reaches the application. For the verbose curl output of a refused handshake, refer to the [mTLS](/en/documentation/platform/workloads/mtls/#errors) errors. --- ## A certificate is refused in a workload setting Binding a certificate to a workload fails with `Invalid certificate type, MUST be an Edge Certificate.` or `Invalid certificate type, MUST be a Trusted CA.` Each certificate field takes one type. `tls.certificate` takes a server certificate, of type `edge_certificate`, and `mtls.config.certificate` a Trusted CA certificate, of type `trusted_ca_certificate`, as the [mTLS](/en/documentation/platform/workloads/mtls/#errors) errors show. - **Check the type**: the `TYPE` column of `azion list digital-certificate --details` shows it. - **Put the server certificate in `tls.certificate`**, or `null` for the Azion SAN certificate. - **Put the Trusted CA certificate in `mtls.config.certificate`**. Azion CLI then prints `Updated Workload with ID `, and the bound certificate reads `active` in Certificate Manager. --- ## Certificate Manager Certificate Manager requests Let's Encrypt certificates for a workload's domains, stores the certificates you upload, and reports the state of each one in its `status` field. For where it acts on a request, refer to [How Workloads works](/en/documentation/platform/workloads/how-it-works/#certificate-manager). ### A certificate stays pending A Let's Encrypt certificate keeps the `pending` or `challenge_verification` status, and the workload's **Digital Certificate** field warns "This certificate is pending validation and HTTPS may not work until it’s validated". The challenge cannot validate one of the names. HTTP-01 needs every name to point to Azion, and DNS-01 needs an `_acme-challenge` record, as [Domain validation](/en/documentation/platform/workloads/certificate-manager/issuance-and-renewal/#domain-validation) explains. - **Check the names**: a certificate requested from the workload form covers the workload's **Domains**, so confirm each one is correct. - **For HTTP-01, point every name at Azion**, alternative names included. - **For DNS-01 at another DNS provider, add the CNAME**: `_acme-challenge.` with the value `.letsencrypt.azion.com`. In Edge DNS, Azion writes the record itself. - **After 48 hours pending, check the CNAME records again**: retries then run every 3 hours, and once a day after 7 days, as [Issuance time and retries](/en/documentation/platform/workloads/certificate-manager/issuance-and-renewal/#issuance-time-and-retries) lists. After a later attempt validates, the certificate reads `active` while a workload names it, or `inactive` until then. For the records step by step, refer to [Request a Let's Encrypt certificate](/en/documentation/guides/application-security/tls-and-certificates/how-to-generate-a-lets-encrypt-certificate/). ### A certificate request is refused with This account cannot use certain common or alternative names A Let's Encrypt request or a certificate signing request (CSR) fails with `This account cannot use certain common or alternative names because it does not have permission for their domain names.` A name on the request belongs to a domain the account has no permission for. An `azion.app` hostname is refused too, even one a workload of the account holds, as the [Certificates](/en/documentation/platform/workloads/certificate-manager/certificates/#errors) errors show. - **Name only hostnames in domains your account has permission for**, in `common_name` and `alternative_names`. - **Request no certificate for an `azion.app` hostname**: with `tls.certificate` set to `null`, the workload presents the Azion SAN certificate, which covers the Azion Custom Domain and the workload domain. An accepted request returns the certificate's `id` with the `pending` status. ### A certificate fails to issue A Let's Encrypt certificate reads `failed`, and the workload's **Digital Certificate** field warns "This digital certificate failed and HTTPS cannot be used until the issue is resolved". Validation failed, and Azion records why in the certificate's `status_detail` field. A typical value names the hostname whose record to check: ```text An error has occurred while issuing the requested certificate. Please verify the following domains CNAME: www.example.com ``` Read `status_detail` from any interface, then correct the record or the name it points to: - **Azion Console**: in **Certificate Manager**, the error icon of the certificate carries it. To sign in, refer to [How to access Azion Console](/en/documentation/guides/platform/account-and-billing/how-to-access-azion-console/). - **Azion CLI**: `azion describe digital-certificate --digital-certificate-id --format json` prints it beside `status`. - **API**: `GET /v4/workspace/tls/certificates/` returns it with the certificate. A later attempt in the [retry schedule](/en/documentation/platform/workloads/certificate-manager/issuance-and-renewal/#issuance-time-and-retries) can then issue the certificate, and an issued certificate in use reads `active` with `status_detail` at `""`. ### An uploaded private key is refused Uploading a server certificate fails with `The provided private key is invalid. Please check the key and try again.` The API cannot read the `private_key` sent with the certificate, as the [Certificates](/en/documentation/platform/workloads/certificate-manager/certificates/#errors) errors show. - **Send the key that matches the certificate**. - **Send it in PEM**, with its `-----BEGIN` and `-----END` lines and no passphrase, as [Key algorithms and formats](/en/documentation/platform/workloads/certificate-manager/certificates/#key-algorithms-and-formats) lists. - **Keep the key one JSON string in an API body**, with each line break written as `\n`. Azion CLI then prints `Created Digital Certificate with ID `, and the certificate reads `inactive` until a workload names it. --- ## Custom Pages Custom Pages replaces the error responses of a workload with pages you choose, through a custom page set that the workload's deployment names. For where it acts on a request, refer to [How Workloads works](/en/documentation/platform/workloads/how-it-works/#custom-pages). ### A custom page set is refused with Ensure this field has at least 1 elements Creating a custom page set fails with `Ensure this field has at least 1 elements.`, or Azion Console refuses to save it with `You must have at least one custom page code`. A set holds at least one page, and the request sent `"pages": []`, as the [Custom page settings](/en/documentation/platform/workloads/custom-pages/settings/#errors) errors show. - **Add a page to `pages`**: one page binds a status code to content on a connector, such as `{"code": "404", "page": {"type": "page_connector", "attributes": {"connector": , "ttl": 0, "uri": "/html", "custom_status_code": 404}}}`. - **Add a page code in Azion Console**: the **Page Codes** table needs one row before you save the set. Azion CLI then prints `Created Custom Page with ID `. ### An error page does not appear The connector's error response reaches the client unchanged, even though a custom page set holds a page for its status code. A workload uses a set only when its deployment names it in `strategy.attributes.custom_page`. A deployment without one reads `"custom_page": null` in `GET /v4/workspace/workloads//deployments`. - **Name the set in the deployment**: select it in the **Custom Page** field of **Deployment Settings**, send its ID in a `PATCH` of `strategy.attributes.custom_page`, or pass `--custom-page` when you create the deployment with Azion CLI. - **Check the status code the connector returns**: a page acts only on the code it lists, such as `"404"`. - **Wait for the change to propagate**, and for the `ttl` of an edited page to expire. The client then receives the page's content in place, with the page's `custom_status_code` and no `Location` header. --- ## Related resources - [Workload settings](/en/documentation/platform/workloads/settings.md#errors): Every field of a workload and its deployment, with the full table of refusals and what to do about each. - [How Workloads works](/en/documentation/platform/workloads/how-it-works.md): The path a request follows through the workload, the deployment, and propagation, which most fixes here lean on. - [Certificates](/en/documentation/platform/workloads/certificate-manager/certificates.md): The certificate types, statuses, and key formats a workload accepts, with the errors an upload or a request returns. - [mTLS](/en/documentation/platform/workloads/mtls.md): How enforce and permissive mode treat a client certificate, and what a refused handshake looks like.