Trakis

Trakis · Rules

Pages that could not be downloaded at all

Pages the crawl found but could not download at all — timeouts, refused connections, broken responses. What a crawler cannot download does not exist for search. If it happens to our crawler, it may be happening to search engines too, or the server may be treating crawlers differently from people.

WarningCRAWL-007 · Crawlability · checked once for the whole site

Where it usually comes from

Slow or overloaded servers, firewalls and bot protection that refuse unknown clients, or pages that take longer than a crawler waits.

How to fix it

  • Open the listed pages yourself and check that they load quickly.
  • Check the server or firewall logs for blocked or throttled crawler requests.
  • If a security layer (WAF, bot protection) is refusing crawlers, allow the major search engines explicitly.

How to check it yourself

Load the listed pages on a slow connection and check your server or CDN logs for refused or timed-out requests at the time of the audit.

What it looks like in a report

Warning 8 pages could not be downloaded at all. · evidence: timeout after 20 s

The server did not answer in time; if it happens to our crawler, it may happen to search engines on a busy day. (An example shop; the numbers are illustrative.)

How the audit counts it

CRAWL-007 is a warning in the crawlability category, and weighs moderately in that category’s score. It is a fact about the site as a whole, so it is reported once, not once per page.

If the finding is deliberate on your site, it can be accepted as intentional in the report. It then leaves the list and the score, and comes back on its own if what it shows changes.

Other crawlability checks

Find it on your site

A Trakis audit runs this check and the other 65 on your site, and tells you on how many pages each one sits. Or go back to all the rules.