Crawler | Sitebulb

Tool : Technical SEO
Plateform : Web
Price : Pay

Short description : Crawler

Sitebulb is a crawler that can crawl your entire website to provide actionable insights and help you improve ranking positions and organic traffic. It’s desktop software for crawling websites, which means it’s a standalone application that you run on your computer.

Long Description : Crawler

Description Sitebulb

Being in constant evolution, the Google algorithm is based on more and more factors to rank a website. This means that despite your efforts to reference your website, you may not be able to meet all the required criteria. To be sure of being in good standing, you must therefore regularly audit your website in order to correct the imbalances that can hinder your ranking. For this, Sitebulb is a powerful exploration tool to check the best practices on a website relevant to Google SEO. Find out in this description how the tool can be useful.

What is Sitebulb ?

Sitebulb is not just a simple website crawler, it runs powerful audits that analyze your website and provide data from an SEO perspective. Not only does the tool guide you through the audit process, but it can detect over 300 issues with your website with just a few clicks. Each index and priority are explained in the audit report followed by some additional advice. While there are a number of crawler tools, Sitebulb stands out for its sleek dashboard and highly actionable insights. Sitebulb mainly aims to illustrate how your website is crawled and tells you how search engines perceive it. Sitebulb therefore helps to ensure that search engines crawl and index content correctly and regularly. ‘s Sitebulb helps you develop an overview of how easily you can make your website more crawlable and indexable.

How to use Sitebulb ?

Using Sitebulb may seem a bit complex, but we’ll show you how to do it in a few steps.

Adding a new project

Like any tool, the preliminary step to using Sitebulb is registration. This is done very quickly in the downloaded application without any complications. Once on your app’s dashboard, you need to add a new project for the site you are going to audit. To set up your project, you need to fill out a form that will ask you to give your project a name, enter your website url and define the type of audit you want to run. Sitebulb will quickly analyze your site’s configuration before letting you set up the audit.

Configure audit settings

Here, the first thing to do is to choose the type of audit that the tool will help to perform. You can choose HTML Crawler or Chrome Crawler. The only difference is that HTML Crawler can run your site audit faster, but without a JavaScript runtime. In contrast, Chrome Crawler will do it much slower and with JavaScript execution. Choose Chrome Crawler to analyze the rendering of the site to run JavaScript and to see if it is fully accessible to search engine crawlers. Then choose additional data types to analyze as part of the audit. At this point, make sure to choose Pagespeed, Mobile Friendly and Front-end to analyze website speed. If your site uses AMP, also check AMP without forgetting the multilingual box to analyze hreflang tags if you are concerned. Check Google Analytics and Google Search Console if you want to match audit results to data from those tools. Under the ”Extraction” section, make sure to check ”Structured data” to analyze and validate the structured data on your site. Next you need to select the URL sources to audit, make sure to check Crawl website and XML Sitemaps to also audit your XML sitemap. You can also access advanced settings to adjust many different and advanced SEO settings for your audit.

Run the exploration

Once the configuration is complete, you can start the audit which displays a report to you in a few seconds. ‘s website Twaino is running. This can take quite a long time depending on the size of the site and the number of resources your website has.

Get Audit Result

Here’s how your audit report can break down:

Indexing Information

The tool’s audit report provides you with a succinct and accurate overview of the indexable and non-indexable content of your website. It highlights any inconsistencies there are about important things like:

Robots.txt;
<head> tag;
And . HTTP header

Canonical tag issues are also checked, making it easier to reduce compromises on your site.

Performance Reports

The overall loading speed and mobile friendliness of your web pages are also two important factors that Sitebulb does not fail to analyze. The ”Site performance” view displays the most critical issues and above all allows you to see how many URLs are affected by each issue.

Internal URLs

This dashboard allows you to view, analyze and export all potential site URLs found during the crawl or in the XML Sitemap. You can also join your Google Analytics and Search Console data with Sitebulb to get a complete dataset. It gives you a breakdown of the content type distribution and gives you confidence that the HTTP or https protocol is consistent.

Site Links

When it comes to website links, the report covers key metrics such as followed links, unfollowed links, broken or redirected links, followed by useful tips that give an idea of how to fix potential issues . Anchor text is particularly useful for spotting pages on which there is overuse or underuse of keywords in internal links. Overall, this data can guide decisions about your website’s internal and external link equity.

On Page

This section is intended to provide all the key data required when performing a site content audit. It summarizes common, but often overlooked, things when it comes to applying on-page best practices for SEO, as well as potential duplicate metadata and on-page content.

Page Resources

Obviously, an audit report is supposed to gather statistics about site page resources such as JS, CSS, video and images only. This section shows us a detailed breakdown of the distribution and behavior of resources on the site. It highlights broken references and tells you exactly where to find them.

Search Traffic

Consolidating Sitebulb crawl data with data from Google Analytics and Search Console is extremely helpful. Not only can you identify the top performing pages, but also the URLs that receive little or no traffic.

AMP

This report highlights any issues with rendering page content on a mobile device. The goal of the AMP standard is to enable publishers to load their websites quickly on mobile devices.

XML

Sitemaps Crawler analysis and interpretation of XML sitemaps, in conjunction with search console data, should help ensure content is accurate and error-free. This feature will undoubtedly prove its worth when dealing with large or multiple XML sitemaps, which can often be quite a pain to parse accurately.

Videos : Crawler

Images : Crawler

Company : Crawler

Sitebulb is an SEO platform created in 2017 and specialized in web referencing. It is a website crawler that performs website audits for SEO consultants and agencies. Sitebulb also offers JavaScript crawling and optimization tips for content creators as well as PDF reports. It also performs audit comparisons in order to see the shortcomings of one website compared to another. The platform is also recognized and appreciated by more than 4000 SEOs around the world and is adopted by many major brands.