How to check that your site is well indexed in Google ?

There are currently more than 1.6 billion websites in the world, a number that continues to grow as more than 2 million blogs are published daily . With these numbers ,being present on the first page of Google results is not an easy goal to achieve . But should you give up when you put your website online? Absolutely not! Indeed, the first thing to do is to make sure that your website appears in the results. In technical terms, it is to verify that it has been well indexed by search engines. Usually, this step is automatically done by robots and you often don’t have to do anything. However, sometimes there are problems or errors such as the indexing of useless or obsolete pages that are likely to impact your performance. So in addition to showing you the steps to perform this check, I give you some good practices to follow.

Website indexing

In this video

What is indexing?

Indexing is the process by which a search engine’s program or robot passes over a website, crawls it and indexes its content. Therefore, when we say that your site has been indexed, it means that the search engine robots have visited it and copied its content and stored it in the search engine servers.

Note that the term indexing that I use here refers to Google indexing with its Googlebot

And this, since it is the most used search engine by Internet users with a worldwide share of 90,6 %. Nevertheless, keep in mind that the principle of operation is quite similar to that of other search engines such as Yahoo, Bing and many others.

The functioning of the Google index

The main objective of the crawl or the scanning by the Google robot is to understand the nature, content and quality of a website. This allows it to associate this page with one or more search intentions.

But regarding the ranking of the page according to the queries or search intentions, note that it depends on a large number of criteria defined by the google’s algorithms.

In addition, there are two other types of indexing robots, apart from the search engine robots. You don’t necessarily need to be interested in the latter since the first one can already fully help you to improve your visibility. These are the :

• Web service robots: They explore a particular type of page in order to extract a specific data. It is for example the backlinks for majesticseo.
• Hacking robots: They test the security holes of the sites they browse and try to break into them.

The size of the Google index and the crawl budget

The size of the Google index

In 2016, Google announced that it had passed the 130,000 billion uRLs in terms of the size of its database for database for indexing. This implies that the brand has indexed so many web pages and although there are no statistics for this year, it is easy to guess that this figure has been largely exceeded.

It is important to note that Google only indexes “indexable” pages and not the entire web, especially since the web is far too vast. Moreover, with such impressive statistics, the firm is obliged to manage a daily budget that is dedicated to the exploration of new pages: the crawl budget.

The crawl budget of a website

The crawl budget is the number of pages to index per day for each site and depends on the importance of it. It is very useful to know this budget in order to position the most relevant pages of your website.

To do this, you must use a very powerful tool developed by the firm Google Search Console. If you do not know how to configure the Search Console, you can consult this article which also introduces you to another powerful tool Google Analytics.

After configuring your account, you just have to click on the “Crawl” tab to see the data. However, keep in mind that the crawl budget is constantly changing, but you can still have an average that will be useful in your analysis.

In addition, there are two types of crawl related to the Google indexing of a website:

• The light crawl it is done daily and only takes into account the most important pages such as the homepage. Its verification is therefore superficial and is therefore recognized as the light version;
• The deep crawl it is carried out approximately every month and allocates a much larger budgetthe deep crawl: It is carried out approximately every month and allocates a much larger budget, so it is the heavy version since it also takes into account the new pages.

Apart from these types of crawl, it is important to know that Google has preferences when it comes to indexing web pages.

The Mobile-First index of Google

Google’s indexing process has undergone a real change from the years 2016 and 2017. Indeed, the firm has implemented a new method of indexing called “ mobile-first index “.

This method recommends the use of the mobile version of a page in order to of a page in order to index and position it. And as the company itself indicates, most Internet users access Google via their mobile device.

Therefore, the responsive websites are the ones that currently enjoy a good indexation and thus a good referencing. This implies that you too must have a responsive website that automatically adapts to the screens of mobile devices, if you want to improve your indexation and your referencing.

Why check the indexing of pages in Google?

Knowing the indexing of pages in Google allows you toidentify possible problems in order to quickly bring the adequate solutions.

Identify the pages that should not appear in the SERPS

There are sometimes pages that should not appear in the search results for several reasons. These include pages with private information or completely useless pages since they do not contain any relevant information. In the latter case, we distinguish for example the pages resulting from an import of a demo theme.

Identify unused changes

When you move, rename directories or change permalinks, it can happen that the Google index still points to the old destination. And if you don’t detect these situations quickly and correct them, you risk losing the organic traffic that these pages can generate.

Identify the sources of 404 errors

Sometimes there are pages or media such as videos, images… that remain in the Google index even after you have removed them from your website. These pages or media are likely to generate 404 errors, which it is better to fix quickly.

Check the presentation of your pages in the Google results

Checking the indexing of your pages is quite interesting since you will see what users will find when searching for information about your website.

First of all, you can appreciate how they are displayed in the results by taking into account the titles and meta description tags. In addition, you can click on each of them to make sure you land on the right page and that the changes you may have made recently have been taken into account.

Best practices to improve indexing

In the search results, 75% of Internet users never scroll to the first pageaccording to alioze. This obviously implies that the web pages that appear in the first position are the ones that benefit from most of the traffic.

The idea being to appear in this very limited list, you must configure your site to facilitate its indexation and improve its SEO. I invite you to take into account these different practices that are likely to help you.

A good content structure

This factor will allow your visitors and crawlers to navigate in a fluid and logical way throughout your website. So make sure to build your structure without forgetting to highlight your most relevant pages, your recent articles … in order to lighten the task of robots.

An impeccable internal linkage

Your contents must be accessible in order to be indexed. To this extent, the pages whose links do not appear directly on your home page are likely not to be quickly indexed by the Google robot.

Indeed, Google can sometimes take time to consider your different modifications. But by improving your internal linkage, you will be able to lead it easily to them and I recommend in this case :

• To set up a system of related articles or associated products;
• To create a “Site Map” page that lists all your content;
• To create a module of the most commented posts or front page articles, etc.

Don’t forget also to integrate the links of your old articles on your new articles, especially when it is relevant to do so. This will allow you to bring them back to life especially if they are too buried in your website.

Creation of a sitemap file

It gathers all the URLs of your website and by submitting it to Google, this file can greatly facilitate the indexation of all the pages of your website. Especially since it will know them all and will be able to access them automatically.

Note however that this file does not replace a good structure and a good internal linkage. On the other hand, a good combination of the three elements is likely to be very effective especially when you have a large number of web pages.

Avoid duplication of internal content

At this level, it is about identical content on several pages of your website. Indeed, the duplication of content has a negative impact on the indexing of your different of your different pages, even if it is internal.

In fact, your visitors and Google will find that your contents are redundant and less relevant, which will lead to inefficient indexing. In addition, your internal linking will be quite bad since the most important information is not highlighted.

Avoid the wrong formats for your content

Some formats are not always readable by Google and you must know them in order to use those that the search engine can easily handle. These are for example:

• Flash: which is practically unreadable for Google and which I suggest you avoid;
• Javascript: Which is not always interpreted in an optimal way;
• PDF : This type of format is indexed, but are less effective than a real page with text content;

Good external links

Even if you have respected the previous practices, I invite you to add this one. Indeed, it is to create links to the pages of your website in order to benefit from :

• A better indexation ;
• Credibility;
• Of the popularity;
• Traffic and therefore customers..

After having observed these practices to improve your indexing by Google, it is often necessary to verify that the various pages of your website are correctly indexed.

How to check that your website is well indexed in Google: Steps to do it yourself

In order to show you clearly the steps to follow, I will use the website of my SEO Agency, Twaino.com.

Note that the steps are quite simple and that you can do this check in a few seconds or even minutes if you have several pages.

In addition, my site is several weeks old and it has had time to be crawled and indexed by Google’s robots.

1) Display all the web pages indexed on Google

To check the indexing, go to the search engine Google and type “site: + domain name”.

Mine gives you : “site:twaino.com”.

Then click on the “Search” button and on the results page, you will see all the pages related to your website that have been indexed.

2) Proceed to the analysis of the different links

The first link you will see in the search results is the one from Google Search Console, a powerful tool from Google that allows you to have more precise information about the indexation.

Then you have all the pages that have been indexed by Google and if you want to go further, check each of them.

You can for example check the titles and meta description tags and click on each of them to make sure you land on the right page.

Also, take the opportunity to identify obsolete or useless pages that are not relevant to Internet users, but are nevertheless indexed and appear in search results.

3) Take action on your indexed pages

If none of your pages have any problems, then everything is working properly. But when there is the slightest problem on one of the pages, implement all possible measures, and this, immediately so as not to lose the traffic that this page is likely to generate.

At my level, for example, I have pages that appear in the Google results and that are of no use to me. Indeed, these are pages from the demo theme that I had the opportunity to import. They are among others :

• New – Avada Seo – Twaino ;

• Careers – Avada SEO

Their appearance in the results can bring confusion and to ensure that they no longer appear in the SERPS, simply delete or de-index them.

And that’s it! You have just seen the pages of your website that have been indexed by Google.

But when you do not find your pages, you can perform some actions.

How to get your website indexed by Google when it is not yet indexed?

It can happen that your website is not indexed by Google’s robots because it is brand new. Instead of waiting for your website to be indexed, you can accelerate the process by adopting one or both of the following options:

• Have a link on a site that is already known to Google;
• Inform Google of your existence with Google Search Console.

Link to a site that Google already knows about

Search engine robots work by moving from one site to another or from one page to another thanks to links. You can therefore use this principle to your advantage by having one or more links pointing to your website on sites already known or indexed by Google.

Keep in mind that you have a better chance of being indexed quickly if the sites in question are important and have a good referencing.

Inform Google of your existence

Taking matters into your own hands can sometimes be more helpful than waiting for Google to crawl other sites to land on yours. It’s all about submitting your website to the search engine using one of its powerful tools Google Search Console.

In addition, to ensure that Google will visit the pages of your website, I recommend you to submit also its sitemap. With the plugin Yoast SEO plugin, you can easily generate a sitemap.

After generating it, you just have to paste the address resulting from the operation in the field reserved for this purpose by the Search Console. This online form can also allow you to submit a URL to Google.

Once you have submitted your site to Google via Search Console, you can go back a few days to check if it was indexed correctly this time

On the home page of the toolhomepage, you will have a graph that shows you the number of indexed URLs in relation to those present on the sitemap.

Some factors that can block the indexing of your website

Even if you have made all the necessary configurations, it can happen that your website is still not indexed. Indeed, there are some settings that can prevent search engines from indexing your site or some of your pages.

WordPress settings

Check to see if you are preventing search engines from indexing your website via the WordPress options. To do this, log into your Dashboard and click on the “Settings” section on the left sidebar.

Then select the “Read” option and check if the box ” Ask search engines not to index this site “is checked or not. If it is, uncheck it to allow indexing.

A website under maintenance

Some plugins like the “coming soon” plugins offer options that prevent options that prevent the indexation of your website by search engines when it is under maintenance. And as long as the maintenance continues, the indexing cannot be done

So if you have such plugins, don’t hesitate to disable them or configure them properly

Other factors that prevent indexing

It is also possible that you have manipulated some files such as :

• robots.txt blocks robots access ;
• .htaccess blocks robots access.

If these files are intact, your site should not have any problems with indexing. And you just have to wait a few more days for all the pages of your website to be correctly indexed by search engines

Conclusion

When you create a website to have much more visibilityit is very important that it appears in the first results of search engines. But still it should be able to be effectively present in the results. It is actually its indexation which is ensured by the robots of the search engines in an automatic way. Nevertheless, it is very useful toto make frequent verificationsto detect possible problems or to make sure that only the most relevant pages of your website have been indexed. Moreover, if you have the ambition toto be in the first resultsyou should not neglect anything