Cause: #
These errors normally occur because a page does not exist (404), it requires authentication (401) or it is forbidden to access the page (403). Make sure you handle each type of code appropriately to ensure the page can be crawled.
It’s important to regularly monitor these errors and investigate their causes, because they may have a negative impact and lower site authority in users’ eyes.
4xx – Client Error #
This group of HTTP status codes indicates that the request for the resource contains bad syntax or cannot be filled for some other reason, presumably by the fault of the client sending the request. Except when responding to a HEAD request, the server SHOULD include an explanation of the error situation and whether it is a temporary or permanent condition. These status codes are applicable to any request method.
400-Bad request | The request is not understood by the server due to incorrect syntax. The client SHOULD NOT repeat the request without modifications. This means that the web server was unable to understand the request and to process it because the request sent by the client Web browser was not formed properly and did not comply with HTTP protocol standards. |
401-Unauthorized | This error message means User authentication is required by the responding server or that authorization has been refused for the provided credentials with the request. If you have just entered these and then immediately see a 401 error, it means that one or both of your usernames and password were invalid. |
402 – Payment Required | This code is reserved for future use. It was intended for the 402 code to be used as part of some form of digital payment, but that has not happened. |
403 – Forbidden | The request sent to the server was a legal request, the server understood it but is refusing to respond to it and process it. The authorization will not help in this case. |
404 – File Not Found | The server responds with this error when it understands the HTTP request sent by the client browser but does not find any files specified by the requested URL address. No indication is given in the server response if the condition is temporary or permanent. This situation is similar to the “return to sender – address unknown” response from your postal service.
A 404 error should not be confused with “server not found” or similar errors, in which a connection to the destination server cannot be made at all. For URLs like www.my_website.com it is very possible that the request for your website has been redirected to the wrong server. This happens when the DNS entries are corrupted. A 404 error also makes sense if your website is completely dead. If this is a DNS issue as soon as the global DNS entries are updated your website should be accessible. For low-level URLs, usually this error indicates a broken link. |
405 – Method Not Allowed | The methods to access resources on web servers are defined in the HTTP protocol, but all web servers can be configured to allow or disallow any method.
The 405 error means that the method specified in the request is not allowed for the resource identified in the request URL. It may be displayed because the web server is not configured to accept requests from visitors at all. Errors with 405 codes are often returned when trying to use the POST method for sending information through a form, but the server is not configured to allow the POST method necessary to process it. All 405 errors are recorded in the server logs so your hosting company could easily explain them. |
406 – Not Acceptable | When making a request, the client can indicate to the web server what kind of data it will accept back.
The header with the 406 error code is returned if the web server detects that the only response it can generate and return to the client is not acceptable to the client. This error occurs very rarely with web browsers because most clients accept any data returned from the server. |
407 – Proxy Authentication Required | This error code is similar to 401 (Unauthorized) and the server responds with a 407 error when the request from the client (web browser or our monitoring agent) was correct, but access to the URL resource requires that the client must first authenticate itself with the proxy. Usually, this means that you must log in (enter user ID and password) with the proxy server first. |
408 – Request Timeout | The server responds with this error when the client did not produce a request within the time that the server was configured to wait. The client may repeat the request without modifications at any later time.
If there are persistent 408 errors, check the workload on your web server at the times when these errors have been detected. Another possibility might be a connectivity problem. |
409 – Conflict | The web server responds with this error when it considers the client request as legitimate, but the request could not be completed due to a conflict with the current state of the resource. The response body usually includes enough information for the user to recognize the source of the conflict and fix the problem.
The conflict here is usually unrelated to standard web server authority or security, but to specific application conflicts, not defined in the HTTP protocol itself. |
410 – Gone | The web server responds with this error when the requested resource is no longer available at the server, will not be available again and no forwarding address is known. This error is similar to 404, but 410 is permanent. |
411 – Length Required | This error code means that the server refuses to accept the request without a defined content length. The client should repeat the request by adding a valid content-length header field containing the length of the message body in the request message. |
412 – Precondition Failed | The web server responds with this error when the precondition given in one or more of the request-header fields was evaluated as false when it was tested on the server. This response code allows the client to place preconditions on the current resource meta-information (header field data) and thus prevent the requested method from being applied to a resource other than the one intended. |
413 – Request Entity Too Large | The server is refusing to process a request because the request entity is larger than the server is willing or able to process. The server may close the connection to prevent the client from continuing the request. What constitutes ‘too large’ depends in part on the operation being attempted. For example, a request to upload a very large file (via the HTTP PUT method) may encounter a limitation on the upload file size set by the Web server. |
414 – Request-URL Too Long | The web server responds with this error when it is refusing to service the request because the Request-URL is longer than the server is willing or able to interpret.
This rare condition is only likely to occur when a client has improperly converted a POST request to a GET request with long query information when the client has descended into a URL “black hole” of redirection (e.g., a redirected URL prefix that points to a suffix of itself), or when the server is under attack by a client attempting to exploit security holes present in some servers using fixed-length buffers for reading or manipulating the Request-URL. Typically Web servers set fairly generous limits on length for genuine URLs e.g. up to 2048 or 4096 characters. If a long URL is valid and you receive 414 errors, then the Web server may need to be reconfigured to allow such URLs through. |
415 – Unsupported Media Type | The server is refusing to service the request because the entity of the request is in a format not supported by the requested resource for the requested method. |
416 – Requested Range Not Satisfiable | A server usually returns a response with 416 status code if a request included a Range request-header field, and none of the range-specifier values in this field overlap the current extent of the selected resource, and the request did not include an If-Range request-header field. For example, if the resource is an image file and has 1000 bytes, and the Range requested is 500-1500, then it cannot be satisfied. |
417 – Expectation Failed | The web server responds with this status code when the expectation given in an Expect request-header field could not be fulfilled by this server, or if the server is a proxy and has clear evidence that the request could not be fulfilled by the next-hop server. |
How to resolve it: #
Forward the audit findings to your web developer to resolve the issue