Soft 404

Soft 404 is an error that occurs when the web page searched by an Internet user is deleted or not found, but for which the server, failing to indicate to search engines the 404 error code that proves that the page searched is invalid, indicates a success code that assumes the existence of contents on this page. This is not the case

Many of us land on pages that immediately display an error message (404), indicating that they no longer exist or that they have not been found

It is important to note that these errors can mean much more than we imagine

But one thing is certain, webmasters must immediately take into account the soft 404, because it is a category of error that can seriously damage the SEO of their websites.

So

• What is a Soft 404 error?
• How to find Soft 404 errors?
• How to correct them to improve your SEO?

These are all questions that I will have to answer in the rest of this content.

[knock]

Chapter 1: Soft 404, what is it?

In this chapter, I explain what a soft 404 error knows and what can be the source of it.

1.1 What is a Soft 404 error?

To better understand it, let’s remember the course of a request on the web. When a user makes a search, the search engine instantly requests access to the content presented in the URL concerned.

To do this, the browser will first ask the server to give it information about the URL or the page

This information is presented as an HTTP CODE code named Status Codewhich will inform the browser about the status of the page. There are various categories of status code, the most recurrent of which are the following:

• The 200 which indicates the success of the request or simply that there is content on the page;
• The 301 and the 302 which designate respectively a permanent redirection and a temporary redirection;
• The 401 which means that the user is not authorized;
• The 403 means that the user is not authorized ; The 403 : means that the access to the page is denied;
• The 404 : the one that concerns us at the moment and which means Page not found or Not Found ;
• The 500 and the 503 which represent Server Error;
• The 504 which means that the server did not respond.

Thus, in the case of a Soft 404 error, the robot or the search engine asks the server for the code to which it should refer. The server then sends it a 200 code that states the existence of content on the page while the URL of the page clearly shows that there is no obvious content on it, hence the 404

However, it is not uncommon to see some people mixing the meaning of the terms soft 404 error and simple 404 error

It is therefore essential to be able to differentiate between :

• A 404 page and a soft 404;
• A page that seems to be a soft 404 and a soft 404 error itself.

1.1.1. Difference between a simple 404 error and a Soft 404 error

The nuance between the simple 404 error, called soft 404 error, and the soft 404 error, also called 404 software error, is on two levels.

Firstly, in the case of a soft 404 error, the pages cannot be found and the corresponding Http code sent by the server reports a 404 Not found

Whereas in the case of a Soft 404 error, the pages are still not found, but the status or Http code that the server sends is the 200 Page Success instead of the 404, which is confusing

To put it simply, the status code that should be assigned by the server in each case is the 404, but the rule is not followed for the Soft 404 error.

Secondly, in the case of a soft 404 error, the pages are not indexed by search engines and therefore do not appear in the SERP (search engine results page), whereas the pages from a soft 404 error are crawled, indexed by search engines and therefore appear in the SERP

1.1.2. Difference between “a page appears to be a Soft 404” and “a Soft 404

Before Google defines a web page as a soft 404 error, it must first consider it as a page that “appears to be a soft 404”

This is because soft 404 errors are temporary. Therefore, before Google makes a final judgment, it first defines the error as “seems to be a 404“and if the error persists, it is officially classified as a Soft 404.

It is good to know that if the mention “the page seems to be a Soft 404” appears in the ERROR tab, it would mean that Google will explore this page again and can modify its status if necessary

On the other hand, if this same mention appears in the EXCLUSIVE tab, it would mean that Google stops indexing the page because it cannot be found.

1.2. The obvious causes of Soft 404 errors

There are several factors that can cause a soft 404 error. The most regular are:

• Those related to a nearly empty page ;
• Those related to a custom error page;
• Those linked to a lot of redirects.

1.2.1. Factors related to a nearly empty page

The Soft 404 error most often committed by webmasters is the fact of accepting indexing pages that do not have enough content. Let’s consider the example of a category page with contents of an e-commerce site or an advert distribution page that has no element or product to present, apart from a few useless sentences

In this case, it is obvious that Google considers your page as a page not found. To be sure, I suggest you use the Google search console which will be able to explore and examine your site in order to indicate the pages whose contents are now of low quality.

1.2.2. Factors related to a custom error page

The second conclusive cause of a Soft 404 error is the fact of making a bad parameterization by including erroneous htaccess file codes when creating a personalized 404 page creating a custom 404 page

Let’s assume for a moment that the name of your page is “404.HJP”, this is a random name. The error that you must absolutely not make in the file code is the following:

ErrorDocument 404 https://example.com/404.HJP

Here we can notice that the URL of the page starts with https which should not be the case. The correct code in this case is :

ErrorDocument 404 /404.HJP

In any case, the only advice I can give you is to regularly make sure that your server really gives a 404 code and not another code to designate a page or URL that cannot be found.

1.2.3. Factors related to many redirects

The last cause of Soft 404 errors I want to mention is redirects. When you redirect a large amount of pages to another page with different content, Google will see only fire, because it assumes that the action taken is inappropriate
It may then declare this page as a Soft 404 error, no matter how permanent or how good your redirects are.

I also advise you to control the number of your redirects and make them to pages that contain similar content as the basic ones (deleted or not found pages)

Now that we know what a Soft 404 is and what could be the cause of it, let’s find out without further ado how to detect this error and repair it.

Chapter 2: How to detect and repair Soft 404 errors on a website?

In this chapter, I will give you tools to detect a Soft 404 error as well as methods to fix it

But first, we will introduce you to the reasons why it is necessary to repair Soft 404 errors. So here are the problems that can arise from a Soft 404 error.

2.1. The problems of soft 404

When a website issues a status code other than 404 to designate a deleted or not found page, it can dangerously influence the crawl budget budget as well as the natural referencing of your websites

2.1.1. Consequence of the soft 404 on the crawl budget

As a reminder, in a Soft 404, your website indicates to search engines that there is real content on the URL they are trying to access, which is not the case

Therefore, the deleted page will be crawled and indexed wrongly by the search engines thus squandering part of your crawl budgetbudget, commonly called crawl budgeton useless pages.

A crawl budget can be defined as the amount of URLs that Google’s spiders can or want to crawl. It is subdivided into two parts which are:

• The crawl rate which allows Google to regulate the frequency with which it explores the pages, not to explore them too regularly or quickly;
• The Exploration Request designates the criteria according to which Google will want to explore your pages. We can have as criteria the rate of traffic on your pages, etc.;

Thus, Google does not want to spend all its time crawling the same sites indefinitely, which is why it has set a budget for their exploration

Taking into account the crawl budget, if a site contains a large number of soft 404 errors, these pages will obviously be crawled. This crawling mechanism, unnecessarily takes a significant portion of the amount of the crawl budget dedicated to the site.

2.1.2. Consequence on the referencing

If I consider the time that the Google robot or Bingbot spends in exploring 404 soft errors, I can conclude that URLs with real content will have difficulty to be explored. Thus, they will not be able to be regularly indexed,lof which decreases the performance of your SEO strategy.

Finally, when you have a high proportion of soft 404s, it means that you are spending your crawl budget unnecessarily, on top of that, you lose visibility on the pages that are important to you

It is therefore normal to see the performance of a site improve in organic searches, when Soft 404 errors are detected and resolved.

2.2. What to do to find Soft 404 errors?

The most effective method to identify Soft 404 errors as well as simple 404 errors is nothing other than using Google search console, specifically the Coverage Report Tool and the URL Inspection Tool.

I already ask you to register Google Search Console on your website, if you haven’t already done so. This will give you access to different features that can help you improve your SEO.

2.2.1. Coverage report tool

The procedure to find a soft 404 error with the Google search console coverage report is as follows

You need to:

• Connect to Google search console ;
• Then, click on coverage in INDEX to open the coverage report;

Make sure you have selected the ERROR tab, then search for the following items:

• The URL you are looking for looks like a Soft 404 ;
• Soft 404 (to find single 404 pages) ;
• Not found.

When you see the error, click on it and you will get information about the pages involved.

2.2.2. URL inspection tool

The other method used to find Soft 404 errors is the URL inspection tool of Google search. This method consists of finding the Http status code.

All you have to do is:

• Insert a URL in the URL inspection tool;
• Click on Enter ;
• Then, click on View Incoming Page;
• Finally, click on More Information.

You will then see the error in question.

Let’s move on to the solutions.

2.3. 5 solutions to fix Soft 404 errors

In this section I’ll show you the five tips you can use to get rid of soft 404 errors.

2.3.1. Check if the page is really a soft 404 or a false alarm

It is not uncommon to notice that Google search console unfairly designates a page as a soft 404. Therefore, it is necessary to check if Google’s predictions are correct.

You will have to :

• Go to the coverage report and click on “the submitted URL seems to be a soft 404” in order to present the list of the concerned pages;
• Go to a URL and click on “Open in a new tab”.

If the page you are looking for is a valid page on your site, and you want it to show up in the page results, click “VALIDATE FIX”.

By doing so, you will force Google to re-scan the page and restore its status code.

Another method is to test the URL of the page.

• Go to a URL;
• Click on “Inspect URL”.

This action will give you details about the page. You can also force Google to update the coverage report by clicking on “TEST URL LIVE”.

2.3.2. Set up your server to return the correct code

When a server does not really return the right code for a page, the next solution is to configure your server

This will send the correct status code response, i.e. 404. After that, you return the page to Google by clicking on the REQUEST INDEX button found in the URL inspection tool.

The easiest way to get a server to display a 404 error for an invalid page when requested is to remove that page from your website.

2.3.3. Optimize the page

When your page is valid, but Google absolutely wants to mark it as a soft 404, the best solution is to review the page content and improve it

After that, resubmit the page to Google by clicking on the Request for indexing button in the URL inspection tool or by going through VALIDATE FIX. This is a procedure generally used when a page has less content and Google no longer wants to keep it in its index.

By producing quality content, you will demonstrate to browsers that your page is optimized, which will make the 404 soft disappear

The article on 10 tips to create a viral content will be very beneficial to you

2.3.4. Use 301 redirection

The solution I present here to fix your Soft 404 errors is to redirect the affected pages to existing or valid pages

To do this, you just need to attach a 301 redirect to the .htaccess file to show browsers that the page has been moved to a new location.

You can also use Yoast SEO plugin to add a 301 redirect.

2.3.5. Deindex the page and keep it on your website

The last option we propose to correct a soft 404, is to deindex the concerned page by adding a Noindex directive on the header, but keep the page on your website.

Here is how to add the noindex directive on the page header without using Yoast SEO:

By doing this, Google will no longer show your page under the ERROR report. But this page will not appear in the SERPs since it will not be indexed.

In summary

A Soft 404 error can be very harmful for a site that wants to perform well in the SERPs. Repairing a soft 404 error is a natural SEO technique that can help improve the visibility of your pages.

But first, you have to know how to identify it, that’s why I presented you, in this content, a tool you can use to quickly find your 404 soft errors

If there are errors, you also need to correct them, that’s why I decided to present you some tips that you can practice to correct your Soft 404 in order to refer your website well

See you soon!