On 30 October 2026, Google said it will support a shared robots.txt rule that lets websites opt out of AI training across several providers in one line, aiming to switch it on in 2027. Microsoft Bing announced the same day that it will start honouring new training preferences from 1 February 2027. For a UK small business, the real question is not how to write the rule. It is whether blocking AI training helps you or quietly costs you visibility in AI answers.
This post explains what changed, what the rule does and does not touch, and how to make the call for your own site, with a checklist you can finish this week.
What did Google and Bing announce on 30 October 2026?
user-agent: __usage_ai-training. Google aims to support it in 2027. Bing starts honouring new training and grounding preferences on 1 February 2027.In a Google Search Central blog post by John Mueller dated 30 October 2026, Google said it plans to support a shared AI training control based on work from the IETF's AIPREF working group. Instead of writing separate rules for each AI developer, a site owner sets one broad preference that participating providers read. Google wrote: "We aim to support this new control along with other AI providers in 2027."
Google's common crawlers documentation names the signal as user-agent: __usage_ai-training and says Google intends to support it "in early 2027."
On the same day, Microsoft Bing published its own announcement, written by Krishna Madhavan, Principal Product Manager. Bing gives a working example: User-Agent: __usage_ai-training followed by Disallow: / to block training across a whole site, or Disallow: /example-path/ for one section.
What is the difference between AI training and AI grounding?
This distinction drives every decision in this post, so it is worth being precise.
Training happens in the background. Your text becomes part of the data a model learns from. Nobody is sent to your site as a result, and you are not paid.
Grounding happens at the moment a customer asks a question. The assistant looks up current pages, uses them to answer, and may show a link. If someone asks an AI assistant for "an emergency electrician near Sheffield open on Sundays", grounding is how your site gets named. That is the part that behaves like search marketing.
Bing now treats these as two separate settings. Its blog describes training as a preference you can set in robots.txt or per page, and grounding as a page-level preference with three values: Allow (the default), DisallowExtended, which limits use to Microsoft's own experiences, and Disallow. Bing states plainly that choosing either restrictive grounding option "will reduce brand visibility in AI experiences" and adds that "Neither choice guarantees inclusion, traffic, or payment."
Does blocking AI training hurt your Google rankings?
Google's crawler documentation is direct on this: "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search."
The documentation does not mention AI Overviews in that section, so do not assume either way that this rule changes how your pages appear in AI Overviews. What it does say is the next point, and it matters more.
Why does the Google-Extended rule matter more than it looks?
Google's documentation says Google-Extended governs whether crawled content can be used to train future Gemini models. It also covers grounding in Gemini Apps, where Search index content is supplied to Gemini at the moment someone asks a question.
Now add the line from Google's 30 October post: if a site uses the shared token to opt out of AI training and does not explicitly address Google-Extended, Google plans to treat it as if Google-Extended were disallowed. The same post says sites that specifically allow Google-Extended remain eligible to appear in responses and receive clicks via grounding in Gemini Apps.
Put those together and the trap is clear. An owner who adds the shared training block, thinking it only affects model training, may also be stepping out of Gemini Apps answers. On Bing you can split the two. On Google, in practice, it is one decision.
Which UK small businesses should block AI training, and which should not?
Use this if/then checklist. Pick the first line that describes you.
- If you sell services locally (trades, salons, clinics, solicitors, accountants, estate agents): leave training allowed. Your pages hold facts you want repeated, and there is no payment or ranking reward for blocking.
- If you sell content (paid recipes, online courses, licensed photography, research reports, premium guides): consider blocking training, but decide on grounding first. On Bing you can keep grounding. On Google, an explicit Google-Extended allow keeps Gemini Apps grounding but also keeps training.
- If you run ecommerce: leave training allowed on product and category pages. If you publish original buying guides that cost real money to produce, consider a path-level block on that folder only, using Bing's
Disallow: /example-path/pattern. - If you already use a "block all AI bots" template: audit it now (see below). It may be blocking grounding crawlers you want.
- If your site uses NoArchive or NoCache tags: note Bing's 1 March 2027 cut-off, after which those tags stop counting as AI preferences.
- If you are unsure: change nothing yet. No new rule takes effect before 2027, and an uninformed block is harder to spot later than an informed delay.
For background on how AI answers differ from classic rankings, see this guide to AEO, SEO, GEO and LLMO for UK small businesses.
What does the decision look like in pounds?
These figures are illustrative, to show the method. Swap in your own numbers.
Example 1: a bathroom fitter in Bristol. Suppose AI assistants send 3 enquiries a month, the fitter wins 1 in 3, and the average job is £6,000 at a 25% margin. That is one job a month, or £1,500 of profit. Over a year, £18,000 of profit depends on staying visible in AI answers. If a blanket training block on Google also removed Gemini Apps grounding and halved that flow, the cost would be around £9,000 a year. The benefit of blocking training is £0, because nobody pays for it.
Example 2: an online baking school in the Cotswolds. Suppose it sells £45 recipe courses and makes 80 sales a month, £3,600 in revenue. If people increasingly get its method from an AI answer instead of buying, even a 10% drop is £360 a month, or £4,320 a year. Here, blocking training on Bing while keeping grounding, and making a deliberate choice about Google-Extended, is a reasonable trade.
The method is the same for both: estimate the revenue that comes from AI visibility, estimate the revenue your content generates directly, and protect whichever is larger.
Where does UK copyright law stand on AI training in 2026?
The UK government's Report on Copyright and Artificial Intelligence, published on 18 March 2026, states that "a broad copyright exception with opt-out is no longer the government's preferred way forward." The report says only 3% of consultation respondents supported that option.
The existing UK exception for text and data mining, section 29A of the Copyright, Designs and Patents Act 1988, applies to "non-commercial research only," according to the report. The government also said it does not plan to intervene in the licensing market "at this stage" and will gather further evidence.
The practical meaning for an owner: the new robots.txt token is a voluntary industry signal. It does not create a UK right to payment, and it is not what decides whether a UK use is lawful. Treat it as a marketing and content-control setting, not legal protection.
What are the risks of copying a "block all AI" robots.txt template?
These templates spread through plugin settings, developer starter kits and forum posts. The shared token makes the problem easier to create, because one line now reaches several providers at once.
Three risks stand out. First, you may block grounding or citation crawlers along with training ones, which cuts you out of answers customers are already using. Second, on Google, a shared training block without an explicit Google-Extended line will be treated as a Google-Extended block, per Google's post. Third, nobody notices for months, because organic rankings stay the same and the loss only shows up as fewer AI-referred enquiries, which are already hard to track.
If you want a wider view of how your business shows up across AI assistants, the AI visibility service page sets out what can be measured.
What should you do this week?
- Open yoursite.co.uk/robots.txt in a browser. List every user-agent line. Look for Google-Extended, GPTBot, OAI-SearchBot, ClaudeBot, CCBot,
__usage_ai-trainingor any other AI-sounding name, and note whether each is allowed or disallowed. Leave Googlebot and Bingbot alone: they power normal search results. - Ask whoever built the site whether any plugin or template added AI blocking rules. Get the answer in writing.
- Check for NoArchive or NoCache tags in your page source (search for "noarchive"). If present, note Bing's 1 March 2027 cut-off.
- Pick your line from the if/then checklist above. Service businesses: allow training. Content businesses: decide grounding first, then training.
- Run your own version of the pounds example. Ten minutes with last year's enquiry data is enough.
- Write the decision in one sentence, with today's date, in your business notes. For example: "Allow AI training and grounding on all pages, reviewed 2 November 2026."
- Set a calendar reminder for 1 February 2027, when Bing begins honouring the new preferences, to confirm nothing changed without your knowledge.
The short version: the new control is useful for businesses whose content is the product, and mostly a risk for businesses whose website exists to win work. Decide on purpose, write it down, and check again when the rules go live in 2027.
AI Training Opt — FAQ
What is the new shared AI training control?
It is a single robots.txt rule, written as user-agent: __usage_ai-training, that tells participating AI providers whether your site's content can be used to train their models. It comes from work at the IETF AIPREF working group. Google announced on 30 October 2026 that it aims to support it in 2027, and Microsoft Bing announced the same day that it will begin honouring new training preferences from 1 February 2027. The point is convenience: one line instead of a separate rule for each AI company. Company-specific rules, such as Google-Extended, keep working alongside it, so you can set a broad default and then make exceptions for individual providers.
Will blocking AI training stop my business appearing in Google Search?
No. Google's crawler documentation says Google-Extended does not affect a site's inclusion in Google Search and is not used as a ranking signal. Bing's announcement says the same for traditional Bing results. The catch is elsewhere. Google's documentation says Google-Extended also covers grounding in Gemini Apps, which is how Gemini pulls live web content into answers and links to sources. Google's blog says that if you use the shared token to opt out of training and do not mention Google-Extended, Google will treat that as a Google-Extended block. That could reduce your chances of being cited in Gemini Apps, even though your normal rankings are untouched.
Should a local trades or service business block AI training?
For most UK service businesses, such as plumbers, salons, clinics, accountants and estate agents, the answer is no. Your website exists to win enquiries, and AI assistants are becoming another place customers ask for recommendations. Blocking training brings no payment and no ranking benefit, and on Google it can come bundled with reduced Gemini Apps grounding. The content on a typical service site, such as opening hours, service areas and prices, is information you want repeated accurately. Keep training allowed, make sure the facts on the site are correct, and review the decision if your business model changes.
Who should consider opting out of AI training?
Businesses whose content is the product. That includes paid recipe sites, online course providers, publishers of original research or guides, photographers who license images and anyone selling written expertise. If an AI model can reproduce the value of your pages without sending visitors, training is a real cost. Even then, think about grounding separately. On Bing you can block training and still allow grounding, because they are separate settings. On Google, Google-Extended covers both training and Gemini Apps grounding, so blocking one tends to mean losing the other. Decide which matters more before changing anything.
Do I need to do anything before 2027?
Nothing is mandatory yet. Google says it aims to support the shared control in 2027, and Bing starts honouring the new preferences from 1 February 2027. The one firm deadline that may affect you is Bing's: if your site uses NoArchive or NoCache tags, Bing says those will no longer be read as a training or grounding preference after 1 March 2027. Use the next few weeks to check your robots.txt, find out whether a developer or plugin added AI blocking rules years ago, and write down a deliberate decision. That way you change settings once, on purpose, instead of reacting later.



