Blocked due to access forbidden (403)

The status sits under Why pages aren’t indexed. Google’s help page explains that a 403 means the visitor sent credentials and was refused, “however, Googlebot never provides credentials, so your server is returning this error incorrectly”. The page will not be indexed.

Google’s status-code page adds the rest: a URL that answers 4xx is not indexed, one already in the index is removed, and, unlike a 5xx, it does not slow the crawl of the rest of the site.

• Bot protection on a CDN. A challenge page is a refusal: Cloudflare’s “Just a moment…” comes back as 403. When our own crawler moved to a cloud server on 12 Oct 2026, one Cloudflare rule on a site we audit answered 40 of its pages that way, to a plain request and to a real browser alike. Rules aimed at “bots” or at cloud addresses catch crawlers unless they are told not to.
• A country block. Google says Googlebot’s default addresses appear to be in the USA, and asks to be treated like any visitor from there. Block the US, and you block Googlebot whenever it comes from a US address.
• A security plugin or server firewall that refuses some user agents, or anyone who asks for many pages quickly.
• The server itself. A folder with no index page, on a server that forbids listing folders, answers 403.
• A login. A page that knows who you are and decides you may not see it answers 403; one that asks for a password first answers 401 (below).

• Inspect the URL in Search Console and run Test live URL. It fetches as Google does, so a 403 there means the block is still in place.
• On Cloudflare, open Security → Analytics → Events and look at the requests from Googlebot. The Service field names the feature that acted: a custom rule, Bot Fight Mode, a managed rule.
• In your server logs, look for Googlebot’s requests with a 403, and check they really came from Google: its verification guide describes the reverse DNS lookup.
• Trakis lists every page that answers with an error, 403 included, with the code each one gave (CRAWL-002). Our crawler is not Googlebot, though: a rule that lets verified crawlers through can still refuse us, so for Google’s view use the live test. When a site refuses our crawler, the report says so instead of scoring the pages as broken.

• Let verified crawlers past your bot rules. Cloudflare keeps a list of verified bots, Googlebot and Bingbot among them, and its own recipe is a custom rule placed first that skips the rest for (cf.client.bot).
• Bot Fight Mode can’t be bypassed. Cloudflare’s free bot protection ignores skip rules; if the events show it refusing Google, turn it off or move to Super Bot Fight Mode, which takes exceptions.
• Lift country blocks for the pages you want in Google, or accept that Googlebot is blocked with the country.
• Open public pages without a login. Google’s alternative is to let Googlebot in without authentication, after checking it really is Googlebot.
• Don’t answer 401 or 403 to slow Google down. Google says not to; the page simply drops out.

Then run the live test again and press Validate fix in the report.

The sibling status: the server asked for a username and password. Google suggests checking it in a private window. On a staging site or a members’ area that is correct, and the fix is to stop linking to it. On a page meant for search, remove the password, or let Googlebot in after verifying it is Google.

Does a 403 hurt the rest of my site? Not its crawling: Google says 4xx answers, except 429, have no effect on the crawl rate. The refused page itself drops out of the index.

Why can I open the page when Google can’t? The rule looks at who is asking: the address, the user agent, the country, how fast they ask. You and Googlebot differ on all four.

Should I let every bot in? No. Let verified crawlers in; keep refusing the rest if you want to.

Is a 403 the same as “blocked by robots.txt”? No. Robots.txt asks Google not to fetch a page, and Google doesn’t; a 403 is the server refusing a fetch Google made. See Blocked by robots.txt.

What about a 429 or a 5xx from the same protection? Google counts those as server errors, a different status with different effects: see Server error (5xx).

The Trakis audit fetches every page and lists the ones that refuse or fail, counted once per problem with the pages it covers. If your protection turns our crawler away, our crawler page says how to let it in. Create a free account and add your site, or see every Search Console status, explained.