Crawler | Netpeak Spider

Description Netpeak Spider

If you have a website online, it would be interesting to browse it very often to see if everything is working correctly. This is precisely the role played by Netpeak Spider, a great tool that helps you crawl your website in just a short time. To understand how Netpeak Spider works, I invite you to read this description carefully.

What is NetpeakSpider ? 

It is an even more essential tool during certain web marketing activities such as:

• The creation and improvement of blog content;
• Creating or improving the link profile;
• The launch of an advertising campaign.

Some of the issues that the tool can help spot on your site include:

• List of incorrect redirects;
• Display of duplicate meta tags;
• Analysis of broken links;
• Collect all pages of a website and check them using 50+ SEO parameters;
• Detection of more than 50 on-page SEO issues and the priority level of each, as well as best practices;
• Analyze all incoming/outgoing internal links;
• Obtain the entire structure of the website;
• The determination of the internal PageRank, that is to say the capacity of each page in terms of authority;
• Professional management and validation of sitemaps;
• A detailed review of HTTP headers and source code;
• Use advanced settings to scrape any website;
• Filter, sort and segment data in the results table;
• Export over 70 reports, including the Optimization Quality Express Audit to PDF;
• Configure HTTP request headers.

From the dashboard, you can notice that it is a tool that offers enough data to analyze your website.  In addition, here is what can be noted from the description of Netpeak Spider. To start using Netpeak Spider, it should be noted that it is a desktop tool that you can easily download from your computer.  As for its evolution, you will not need to download a new version each time, because the tool updates automatically. Once the download is done, you will use some information to register or identify yourself if you already have an account. Log in using the credentials you used to register on the website and download the tool. If you’re not used to it, you’ll find using the tool a little complicated at first, but it’s actually even easier.

How to configure Netpeak Spider ?

To configure the Netpeak Spider explorer, you must go to the ”Parameter” tab of the tool. At this level you will find the many options that you can customize. The tool therefore offers quite specific parameters that allow you to carry out analyzes according to your needs. For example, if you care most about the content of your pages, just select this setting alone and turn off everything else. This makes the crawler’s site crawling process even faster and more efficient. If you are not an SEO newbie, you certainly know that it is important to create custom templates with specific settings to perform suitable tasks. When you set up the tool once, it only takes a few clicks to adjust it to perform other tasks as needed. In the settings, the tool offers a restriction option that lets you set custom issue restrictions. For example, if you are not satisfied with the response time of 2 seconds, you can set this as a problem in the settings. 

How to Configure Netpeak Spider Crawl ?

As in the previous section, the crawler settings are even more customizable. You need to be careful here not to get lost in the huge options you will have to configure. But, the advantage is that each option to configure is accompanied by an information bubble describing what it knows.  The first two settings tabs called ”General” and ”Advanced” contain all the main exploration settings. Here are some important configurations you can do here:

• Crawl speed, it’s actually to set the number of concurrent threads, delay between requests, maximum response delay;
• Customization of JavaScript rendering to make it easier to crawl websites using JS technology;
• Crawl Settings is the option to crawl exact website directory only, crawl all subdomains, crawl external links and check different file types;
• Consider crawling and indexing instructions like robots.txt, meta robots, X-Robots-Tag, etc;
• Crawl URLs from the <link> tag
• Other things like pausing, using cookies, and crawling relative canonical URLs;
• Enabling multi-domain exploration to explore different domains simultaneously.

Some Features of Netpeak Spider

Here are some data you can get when you run the tool:

Dashboard

The dashboard is the first thing you can focus on when using these kinds of software. As for Netpeak Spider, here is what the dashboard has in store for us:

• Number of URLs crawled;
• Number of internal URLs;
• Number of URLs with errors and warnings;
• Number of conforming URLs.

In addition to that, you have an overview of the data in the form of a visual representation. These include:

• Eight blocks with a visual representation of key data;
• The percentage of compliant URLs, 
• The severity of the problem; 
• Status codes;
• The number of URLs per click depth;
• Etc.

You can easily interact with this data on the dashboard without any difficulty.

Categorization of problems

To allow users to have an easier understanding of the data, Netpeak Spider offers three different categories of problems according to their intensity:

•  Errors;
• Warnings;
• Notice.

The idea is to allow even non-professional SEO to understand the priorities on which to put themselves immediately. In addition to giving details on each problem, the tool also provides good practices to remedy them.

Filtering, grouping and segmentation of results

Since the tool provides a significant amount of information, it would be more interesting to have them in an order so as not to unconsciously skip the most important information. For this, the Netpeak Spider report gives you the possibility to sort the data, group the results according to certain parameters and create data segments. Here are some filter manipulations:

• Click on any issue in a sidebar to see only the pages that contain it;
• Click on any parameter of the “Overview” tab to see the pages that correspond to it;
• Click on any part of the website in the “Site Structure” report to see the pages of the corresponding part;
• Click on any value in the dashboard;
• Right-click on any value in the results table and select ‘Filter by value’;
• Define multiple custom filter rules.

When you customize a result, it is automatically displayed in a new tab so you can make a comparison with the main results page.