Troubleshooting ClientOtherErrors on Azure Storage Account

The blog is inclined towards troubleshooting clientothererrors reflecting in the metrics. It is a step-by-step process to understand what these errors signify and potential reasons. This will also help in analyzing if these are legitimate ones and also deciding on the action plan ahead.

 

ClientOtherError usually means expected client-side errors which are not fatal errors. These operations have been completed successfully and therefore don't affect other metrics, such as availability. Some examples of operations that execute successfully but that can result in unsuccessful HTTP status codes include:

• ResourceNotFound (Not Found 404), for example, from a GET request to a blob that doesn't exist.
• ResourceAlreadyExists (Conflict 409), for example, from a CreateIfNotExist operation where the resource already exists.
• ConditionNotMet (Not Modified 304), for example, from a conditional operation such as when a client sends an ETag value and an HTTP If-None-Match header to request an image only if it has been updated since the last operation.

Below are some of the common reasons for some of status code and resulting into clientothererrors 

 

The client is receiving HTTP 404 (Not found) messages:

If the client receives an HTTP 404 (Not found) message from the server, this implies that the object the client was attempting to use (such as an entity, table, blob, container, or queue) doesn't exist in the storage service. Below are some of the possible reasons:

• The object being requested doesn’t exist. The object might not haven’t been created itself or the client or another process previously deleted the object.
• A head operation for GetBlobProperties prior to PutBlob initially checks whether a blob exist or not. This can also result in 404 and is then followed by actual request to create the blob.

The client receives HTTP 409 (Conflict) messages:

Mostly in the client applications, while creating a blob container, queue, table if make use CreateIfNotExists method. If the object being created already exists, it will eventually fail with the HTTP 409 (Conflict) error.

Another example is that of blob getting created and prior to that a check is done to see whether the container in which blog is being created exist or not. If the container doesn’t exist and we don’t check that, the call to create the blob will tend to fail. So having a sanity check prior is also helpful.

 

Now, let us look at how to analyze them ahead:

 

Step 1: Identifying ClientOtherErrors

 

You will mainly notice ClientOtherErrors in the Insights section of Monitoring under the failures tab.

 

 

Alternatively, if you split the transactions based on the response type, you can filter these explicitly for the clientothererrors.

 

Step 2: Enabling Logging configuration

 

The above 2 options only provide a high-level detail of the number of errors happening or the timelines when the error occurred. This, however, doesn’t provide more robust details as to what the operation was and the exact error resulting in the clientothererrors. For that, we can start by enabling the diagnostic logging on the account.

Monitoring Azure Blob Storage - Azure Storage | Microsoft Learn

 

 

Step 3: Running Kusto Queries

 

For the demo, we have created a diagnostic setting to move the logs to log analytical workspace. After enabling the same, we can make use of Kusto queries to narrow down the transactions. Below are 2 sample queries that can be used:

 

 

StorageBlobLogs | where MetricResponseType contains "ClientOtherError"

In the below example, we got multiple operations that weren’t successful.

OR

StorageBlobLogs | where MetricResponseType !contains "Success"

The first one shall provide only the transactions that fail with response type of ClientOtherErrors specifically while the other one provides all the transactions other than the successful ones which will include authentication errors, clientothererrors, server-side errors etc.

 

We can check the 404 and 409 errors are shown below:

 

StorageBlobLogs | where MetricResponseType has 'ClientOtherError' | where StatusCode == "404" | project StatusCode,OperationName, MetricResponseType

 

StorageBlobLogs | where MetricResponseType has 'ClientOtherError' | where StatusCode == "409" | project StatusCode,OperationName, MetricResponseType

 

 

Step 4: Exporting the Logs

We can export this set of logs to CSV for further assessment.

 

 

Step 5: Analyzing the Logs

 

In the MetricResponsetype column we will filter out specifically clientothererrors. In case we make use of 1^st query then additional filtering isn’t required since we will only have the transactions failing with ClientOtherErrors.

 

 

We could then check the operation and along with the statuscode and statustext field to understand the exact reason for the error.

 

 

You can make use of above steps to analyze the clientothererrors from your side. In case you have validated the above steps but observe something odd or if there is any further understanding required you can reach out to Microsoft support ahead.

 

Hope this helps!

 

Reference Link:

 

Status and error codes (REST API) - Azure Storage | Microsoft Docs

Troubleshoot client application errors in Azure Storage accounts | Microsoft Docs