Cloudflare SEO: the settings that change what Google sees
Putting a site behind Cloudflare does not, by itself, change how it ranks. What can is a handful of settings that decide what a crawler is given: a challenge instead of the page, a redirect that never ends, a robots.txt you did not write, HTML that Cloudflare has changed on the way. Here they are, each from Cloudflare’s own documentation, and a ten-minute check for yours.
Does Cloudflare hurt SEO?
Not on its own. Cloudflare’s SEO page lists what does not affect it: using Cloudflare’s nameservers, where its servers are, and shared IP addresses, which search engines generally don’t penalise unless several of the sites on them are spam. Caching, it says, only speeds a site up: it does not create duplicate content, rewrite URLs or add subdomains. What it recommends is speed, HTTPS with visitors redirected to it, Crawler Hints, and content worth crawling.
The same page names the risk: security settings can block search engine crawlers by accident. That, a redirect loop from the SSL mode, and a few features that change the page are where sites on Cloudflare lose search traffic.
Settings that can keep Googlebot out
| Setting | What Cloudflare says | What to do |
|---|---|---|
| WAF custom rules, rate limiting rules | They must allow Verified Bots, the crawlers Cloudflare has confirmed, Googlebot and Bingbot among them. Rate limiting should not apply to Google’s crawler. | Add not cf.client.bot to a rule that challenges or blocks, as Cloudflare’s own examples do. |
| Bot Fight Mode (Free) | Challenges requests that match known bot patterns, on the whole domain. It cannot be skipped or given exceptions with custom rules. Its page does not say how it treats verified bots. | If crawl errors start after you turn it on, turn it off first. Exceptions need Super Bot Fight Mode. |
| Super Bot Fight Mode (Pro and up) | Has its own setting for Verified bots, with exceptions through custom rules; Cloudflare’s examples set Verified bots to Allow. | Check that Verified bots is set to Allow. |
| Under Attack mode | Shows every visitor a page with a JavaScript challenge before the site. Cloudflare calls it one of the last resorts during an attack. Its page does not say how crawlers are treated. | Use it during an attack, and turn it off after. |
| IP Access rules, country blocks | Cloudflare’s crawl errors guide says not to block Google’s crawler addresses, and not to block the United States. | Check what your country and IP rules cover. |
| Your server’s own anti-bot module | It can block crawler requests that Cloudflare forwards. | Turn it off, or let Cloudflare’s requests through. |
A rule that challenges a crawler does not show as a 403 in your server’s logs: the request never reaches the server. It shows in Cloudflare, under Security → Analytics → Events, where each challenge names the product that made it in its Service field. In Search Console it shows as pages Google could not fetch.
HTTPS without a redirect loop
Cloudflare’s SSL/TLS encryption mode decides how it talks to your server. In Flexible mode it reaches your server over plain HTTP; if your server then redirects HTTP to HTTPS, the request goes round and round, and browsers and crawlers get ERR_TOO_MANY_REDIRECTS (Flexible, redirect loops). Cloudflare’s fix:
- Put a certificate on your server (Cloudflare’s free Origin CA certificate will do) and set the mode to Full (strict).
- Turn on Always Use HTTPS, which redirects every http:// request at the edge (Always Use HTTPS). Cloudflare recommends not redirecting at your server as well.
- Pick one address, with or without www, and send the other to it with a single 301 redirect rule.
The audit checks each of these: http:// served instead of redirected, the site on two addresses and redirect chains.
robots.txt: the file crawlers read may be Cloudflare's
Cloudflare can write part of your robots.txt, or all of it, from its AI bot settings, and since 15 September 2026 new domains start with some AI bots blocked on pages that show ads. That is told at length in Cloudflare’s AI bot settings, and five things to check. The short version: open https://your-domain/robots.txt, because that is the file crawlers get, and paste it into the robots.txt validator to see who may fetch what.
One line is worth adding on any site behind Cloudflare. Crawlers can wander into /cdn-cgi/, Cloudflare’s own path, and report errors there. Cloudflare says they don’t affect rankings but may show in Search Console, and recommends (the /cdn-cgi/ endpoint):
User-agent: *
Disallow: /cdn-cgi/If you serve resized images through Cloudflare Images, they live under /cdn-cgi/image/: put Allow: /cdn-cgi/image/ before that line, or search engines can’t index them.
Crawler Hints talks to IndexNow, not to Google
Crawler Hints (Caching → Configuration) watches your cache: when a page misses it, which usually means it changed, Cloudflare tells search engines through IndexNow (Crawler Hints). Cloudflare’s SEO page recommends it. The search engines that take IndexNow are Amazon, Bing, Naver, Seznam.cz, Yandex and Yep; Google is not among them (indexnow.org). So it can speed up Bing, and it does nothing for Google, good or bad. A page you keep out of search is still kept out by its noindex.
Features that change your HTML
Search engines index the page Cloudflare serves, not the one your server sent. Cloudflare lists the features that rewrite it on the way (response body changes, features that alter cached content). The ones that matter for search:
- Rocket Loader defers all of a page’s JavaScript until after it renders (Rocket Loader). If your content is drawn by JavaScript, check that Google still sees it; the audit’s content only through JavaScript asks the same of AI crawlers.
- Hotlink Protection refuses images to pages on other sites. Cloudflare says it does not affect crawling, but stops your images showing on Google Images, Pinterest and Facebook (Hotlink Protection). A folder called hotlink-ok, or a configuration rule, exempts images.
- Email Address Obfuscation hides addresses from bots with a script, and Always Online adds a banner to the archived copy it serves when your server is down.
- Markdown for Agents (Pro and up) answers a request that asks for text/markdown with a Markdown copy of the page, and sends HTML to everyone else (Markdown for Agents). Unless your server sets its own, the copy says ai-train=yes, search=yes, ai-input=yes.
After turning one on, look at a page the way Google does: Search Console’s URL Inspection → Test live URL shows the HTML Googlebot received.
Changing titles and descriptions at the edge
The same machinery can fix a page instead of changing it by accident. A Cloudflare Worker on your domain can rewrite the HTML as it is served, with HTMLRewriter: a new <title> or meta description, without touching the site’s code or its CMS. Googlebot gets the rewritten page like any visitor. That is how Trakis’s Fixes work: you write the new title or description for a page the audit found, and a Worker in your own Cloudflare account puts it live. Every request on the Worker’s route runs it and counts toward your Workers plan; pages without a fix go through as your server sent them.
A ten-minute check
- Security → Analytics → Events: look for challenges or blocks of Googlebot and Bingbot, and which product made them.
- Security → Settings, filtered by Bot traffic: Bot Fight Mode on or off; on Pro and up, Verified bots set to Allow.
- SSL/TLS → Overview: Full (strict). Edge Certificates: Always Use HTTPS on.
- Open https://your-domain/robots.txt and paste it into the validator; add Disallow: /cdn-cgi/ if it isn’t there.
- Scrape Shield and Speed settings: note whether Hotlink Protection and Rocket Loader are on, and test a page in URL Inspection.
- Run an audit of the whole site: the Trakis audit reads every page the way a crawler gets it from Cloudflare, with redirects, status codes and robots.txt, and counts each problem once. Create a free account, or read every check it runs.
Common questions
Does Cloudflare slow Google’s crawling? Cloudflare’s crawl errors guide says Google and Bing set their own crawl rates for sites behind a CDN, and that those rates don’t hurt SEO.
Does Cloudflare’s cache create duplicate content? Cloudflare says no: it does not rewrite URLs or create subdomains.
Should I turn on Bot Fight Mode? It stops a lot of scraping, but it can’t be given exceptions, and Cloudflare’s page for it doesn’t say how it treats Googlebot. If Search Console starts reporting pages it couldn’t fetch after you turn it on, turn it off.
Why does Search Console list errors under /cdn-cgi/? That path is Cloudflare’s, not your site’s. Cloudflare says the errors don’t affect rankings; Disallow: /cdn-cgi/ stops them.
Is Crawler Hints bad for SEO? It sends IndexNow notices, which Google does not take. For Google it changes nothing either way.
Sources
Cloudflare’s documentation and indexnow.org, all read on 4 November 2026; the date after each is the page’s own.
- Improve SEO, 20 April 2026: what helps, what does not affect SEO, Verified Bots in security settings.
- Troubleshooting crawl errors, 23 April 2026: Google’s crawler addresses, rate limiting, the United States, /cdn-cgi/, crawl rates.
- Bot Fight Mode and Super Bot Fight Mode, 3 August 2026; Stop malicious bots and Protect forms: verified bots, cf.client.bot.
- Under Attack mode, 20 April 2026.
- Flexible and ERR_TOO_MANY_REDIRECTS, 1 September 2026; Always Use HTTPS, 14 August 2026.
- The /cdn-cgi/ endpoint.
- Crawler Hints, 14 August 2026; IndexNow FAQ.
- Rocket Loader, 14 August 2026; Hotlink Protection, 3 August 2026; Response body changes; Markdown for Agents, 9 October 2026.
- HTMLRewriter.
Last changed 4 November 2026.