Skip to main content
    Insights · SEO

    SEO News Week 33: Cloudflare Can Block Googlebot

    7 min read
    Quick Answer

    Between July 23, 2026, and August 7, 2026, reports revealed that Cloudflare's setting for blocking AI crawlers can also block Googlebot. Cloudflare will change its default on September 15, 2026. Meanwhile, tracking services reported significant ranking fluctuations without a confirmed Google update, and the European Commission imposed a €890 million fine on Google. Your top priority: check crawler access.

    At a Glance

    NewsWhat Is ChangingWho Is AffectedAction NeededSource + Date
    Cloudflare blocks GooglebotThe "AI Training = Block" option returns HTTP 403 to Googlebot; new blocking default starting September 15, 2026All websites behind CloudflareCheck this weekSearch Engine Journal, August 4, 2026
    Ranking fluctuations from August 1, 2026, through August 3, 202613 tracking services report major swings; Google has not confirmed an updateWebsites with a narrow keyword setDocument, don't overhaulSearch Engine Roundtable, August 3, 2026
    AI report in Search Console remains incompleteThe report on AI Overviews and AI Mode appears only when there is enough dataSmall websites without high impression countsWait and adjust expectationsSEO Südwest, August 6, 2026
    €890 million DMA fine against GoogleGoogle must treat all services equally in its rankingsComparison, travel, and local service providersMonitorEuropäische Kommission, July 23, 2026
    Test: login instead of CAPTCHA in SearchGoogle is testing a sign-in requirement for additional results pagesAnyone using rank trackingCheck reports for gapsSearch Engine Land, August 4, 2026

    Cloudflare Can Block Googlebot — Key Date: September 15, 2026

    Cloudflare's "AI Crawlers & Scrapers" option with "AI Training = Block" selected caused Googlebot and Bingbot to receive an HTTP 403 error when fetching a sitemap in a case reported by Search Engine Journal on August 4, 2026. The error disappeared immediately after the option was disabled. SEO Südwest covered the report on August 5, 2026.

    Cloudflare is a service that protects websites against attacks and speeds up their delivery. A large number of websites in Germany use it, often without management knowing, because the agency or hosting provider set up the service.

    The second part of the news is more important: starting September 15, 2026, Cloudflare will, according to the report, block dual-purpose crawlers by default—those that collect data for both search and AI training. Google's John Mueller joined the underlying discussion thread and asked to be contacted directly to investigate.

    What Does This Mean for Your Business?

    If Google can no longer fetch your pages, your site will disappear from the index over the following weeks. This is the most expensive technical error imaginable, and in this case, it stems from a well-intentioned setting. Ask your agency or hosting provider to confirm in writing which bot rules are active in Cloudflare, and test the live URL in Google Search Console. The test takes two minutes and should be completed before September 15, 2026.

    Major Ranking Fluctuations in Early August, but No Confirmed Update

    Significant ranking fluctuations occurred from August 1, 2026, through August 3, 2026, without a confirmed update, as reported by Search Engine Roundtable on August 3, 2026. They were recorded by 13 tracking services, including Sistrix, AccuRanker, Semrush, and Mozcast. In its monthly review on August 4, 2026, Search Engine Roundtable placed this period in a series of other unconfirmed fluctuations, including those on July 24, 2026, as well as July 18, 2026, and July 19, 2026.

    Google has not confirmed an update for any of these periods. The last officially confirmed update remains the June 2026 spam update, which ran from June 14, 2026, through June 26, 2026.

    What Does This Mean for Your Business?

    Websites with only a few important search terms feel these fluctuations more strongly than large portals, because a single position change immediately affects a noticeable share of traffic. Note the period in your reporting and wait two to three weeks. If you rewrite content, merge pages, and change internal links now, you will not be able to distinguish the effects of the fluctuation from those of your own changes later.

    AI Report in Google Search Console: Rollout Still Incomplete

    The Search Console report on visibility in Google's AI features is still not available for all websites, as SEO Südwest reported on August 6, 2026. Google's John Mueller attributed this to a data threshold: the report appears only for websites with enough data, similar to the Discover report. Google announced the feature in June 2026 on the Search Central Blog.

    The report shows which pages receive impressions in AI Overviews and Google AI Mode. AI Overviews are the AI-generated answer blocks above the traditional list of results; Google AI Mode is Google Search's standalone chat interface. The report does not provide click data or search queries.

    What Does This Mean for Your Business?

    For small business websites, the reality is straightforward: the most important free measurement tool for Generative Engine Optimization—optimizing for citations in AI answers—is not available for now. Anyone who wants to demonstrate visibility in AI answers today needs to manually spot-check whether their brand appears in ChatGPT, Perplexity, and Google AI Mode. At Stroe Media, we document a fixed set of test questions for each client and repeat them monthly.

    EU Imposes €890 Million Fine on Google

    The European Commission announced on July 23, 2026, that it was imposing an €890 million fine on Google for violations of the Digital Markets Act. Of that amount, €460 million relates to self-preferencing in Google Search and €430 million to anti-steering rules in the Google Play Store. CNBC reported on the fine the same day.

    The Commission criticizes Google for displaying its own services more prominently than competitors' services, for example higher on the results page and with additional images and filters. Going forward, Google must apply transparent, fair, and nondiscriminatory conditions when ranking all services. The announcement does not specify an implementation date.

    What Does This Mean for Your Business?

    For providers in comparison services, travel, real estate, and local services, this is a potential opportunity. If Google has to scale back its own result boxes, traditional organic listings will move back up. You cannot plan around this yet, because no date has been set. It makes sense to accurately measure visibility for your core terms now so that you can detect any later change to the results page.

    Google Tests a Login Requirement Instead of CAPTCHA in Search Results

    Google is testing a requirement for users to sign in to a Google Account to see additional search results, as Search Engine Land described on August 4, 2026. The message "Sign in to verify you're a human and see more results" appears when users browse beyond the first few results pages. Search Engine Roundtable documented the test on August 3, 2026, and SEO Südwest reported on it on August 4, 2026.

    Google calls this a limited test and has not commented on either its purpose or its rollout.

    What Does This Mean for Your Business?

    Nothing changes directly for your website. Indirectly, however, the reliability of your reports is at stake: rank-tracking tools automatically read search results pages. If lower-ranking results become available only to signed-in users, visibility metrics will become less complete, especially for positions beyond the first page. If you report rankings to management monthly, keep this uncertainty in mind.

    Specific Steps to Take This Week

    1. 1Check whether your website runs through Cloudflare, and ask to see the active rules under "AI Crawlers & Scrapers." If "AI Training = Block" is selected, immediately test a live URL in Google Search Console.
    2. 2Add September 15, 2026, to your calendar as a review date and repeat the live URL test that day, because Cloudflare will then change its blocking default.
    3. 3Open the Google Search Console performance report for July 25, 2026, through August 8, 2026, and highlight August 1, 2026, through August 3, 2026, to check the reported volatility against your own numbers.
    4. 4Create three to five fixed test questions about your offerings and ask them once a month in ChatGPT, Perplexity, and Google AI Mode. Record whether your website is cited as a source.
    5. 5Check Search Console to see whether the generative AI features report is visible for your property, and plan your reporting without it if it is not.
    6. 6Clarify internally who is responsible for server logs, and ask for a record of Googlebot requests over the last 30 days. A drop there is the earliest warning sign of crawling problems.

    Frequently Asked Questions

    With a specific setting enabled, yes. On August 4, 2026, Search Engine Journal reported a case in which Cloudflare's "AI Crawlers & Scrapers" option, with "AI Training = Block" selected, caused Googlebot and Bingbot to receive an HTTP 403 error when fetching the sitemap. The error disappeared immediately after the option was disabled. Only websites that have explicitly enabled this option are affected.

    According to Search Engine Journal's August 4, 2026, report, Cloudflare will change its default setting on that date and begin automatically blocking dual-purpose crawlers that collect data for both search and AI training. If you do nothing, you risk having these bots blocked starting September 15, 2026, without taking any action yourself. Check the setting before then.

    No. On August 3, 2026, Search Engine Roundtable reported significant ranking fluctuations from August 1, 2026, through August 3, 2026, recorded by 13 tracking services, including Sistrix, Semrush, and AccuRanker. Google has not confirmed an update related to these fluctuations. The last officially confirmed update remains the June 2026 spam update.

    Your website probably does not generate enough data. On August 6, 2026, SEO Südwest reported that Google's John Mueller attributed the incomplete rollout to a data threshold: the report appears only for websites with enough impressions, similar to the Discover report. The report shows impressions in AI Overviews and Google AI Mode, but no click data or search queries.

    Nothing in the short term; in the medium term, it could change the appearance of the search results page. On July 23, 2026, the European Commission imposed a €890 million fine and required Google to treat all services transparently, fairly, and without discrimination in its rankings. The announcement does not specify an implementation date. For small businesses, this is more of an opportunity than a risk, because Google's own services could lose their preferential placement.

    Not yet. On August 4, 2026, Search Engine Land described a limited test in which Google requires a login instead of showing a CAPTCHA when users browse beyond the first few results pages. Google explicitly calls this a test. The practical consequences would primarily affect rank-tracking tools, whose data could become less complete.

    Conclusion

    This week brought no ranking update, but it did bring a concrete technical issue to address. The Cloudflare case shows how quickly a protective setting can undermine visibility, and September 15, 2026, makes it a deadline. The ranking fluctuations in early August remain unconfirmed and do not justify rushed action. The EU fine against Google and the login test in Search signal that the results page in Europe continues to evolve.

    If you want to know whether your website is technically accessible to crawlers and how it performs in AI answers, explore our SEO services or contact us directly. Stroe Media GbR is an owner-managed web design and online marketing agency based in Geretsried, Bavaria, south of Munich, Germany. Find out what is changing this week in Google Ads in our Week 33 SEA article. You can find the previous week's recap in SEO News Week 32/2026.

    Sources

    Author: Stroe Media Editorial Team, Updated: August 10, 2026

    Back to overview