Meta Removes 'Train' From New Business Terms but Keeps Broad Rights to Crawl Sites

TL;DR

Meta has rewritten the crawling clause in its new Commercial Terms, which take effect October 30 and cover every business that uses Facebook, Instagram, WhatsApp or Messenger commercially. The first draft let Meta use content scraped from any site "associated with" a business account to "train" Meta products. After Nieman Lab reported it on October 15, Meta deleted "train" and limited the permitted uses to "advertising and business products." The rest of the clause stayed: the list of bots, the right to create test credentials, the broad "associated with" scope, and the rule that you accept or stop using Meta for business. The terms still say nothing about robots.txt. Three publisher trade groups have asked regulators to step in.

What the clause said on Thursday

Meta told business users about the update in early October, through a blog post and a notice in Meta Business Suite. Nieman Lab heard about it from a reader: Stephen Beale, who runs a site called The Steampunk Explorer, saw the notice in his dashboard and emailed the newsroom. Reporters Andrew Deck and Hanaa' Tameez published the clause on Thursday, October 15, with three legal experts reading it as permission to scrape news sites for AI training.

The original text, as archived by the Wayback Machine on October 16 at 16:20 UTC, authorized Meta "(or any entity acting on our behalf)" to use "web scrapers, bots, robots, spiders, crawlers, and user agents" and to "create test credentials" to crawl any domain "associated with your products or services, or that are made available through your or users' use of Meta Company Products." Then it said what Meta could do with the haul: "provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products." Meta's terms elsewhere list "AI at Meta" as one of those products.

There is no opt-out. The update notice reads: "If you do not agree to the updated terms, please stop all access or use of our Products for business or commercial purposes." That applies to a shop with one Facebook Page as much as to a newspaper. As Nieman Lab pointed out, nearly every newsroom has at least one Meta business account, and that includes Nieman Lab.

"This is a remarkable provision to bury in [the] terms of service, given that this would give Meta access to material that would likely cost many millions of dollars in a licensing market," media and IP attorney Matt Topic of Loevy & Loevy told the outlet.

What changed by Friday night

Meta revised the page on Friday afternoon, October 16. The next Wayback snapshot, at 03:19 UTC on October 17, shows the new ending: Meta may crawl "in order to provide, develop, improve, research, and maintain advertising and business products (including business, commercial and related products to help you create and deliver ads and other commercial content, provide users information about your products and services, and connect you with users who may be interested in your products and services) and maintain the integrity and security of Meta Company Products."

Everything before that sentence survived word for word. The effective date still reads October 30, 2026.

Meta's explanation

Meta has called the change housekeeping from the start. A spokesperson told Nieman Lab that crawling language had been "scattered across different terms of use," including the terms for generative AI in ads, and that the update just consolidates it. On October 16, spokesperson Andy Stone posted on X that a summary of the report was false. He said the clause exists so Meta can pull accurate details about products sold on third-party sites for ad and content recommendations, and that it is "not about scraping news sites."

After the rewrite, Meta's statement read: "We've heard you and want to clarify: the crawling terms we consolidated only applied to our ads and business products, and we've clarified that in the terms themselves." A spokesperson added on Saturday that Meta revised the text because "it wasn't clear in the initial consolidation of terms that robots.txt preferences are respected."

The revised clause still does not mention robots.txt. If the problem was that the terms didn't say Meta respects robots.txt, the fix would presumably have been to write that down.

The consolidation story is partly true. Search Engine Journal checked the documents. The current Commercial Terms, dated March 6, 2024, have no crawling clause. The Self-Serve Ad Terms from August 2025 already let Meta crawl the domains of businesses that place ad orders, with "research and training" among the listed uses. What's new is the trigger. Buying an ad used to be the trigger, and from October 30 it is any business use of Meta's products, including a Page that has never run an ad.

Why the lawyers aren't satisfied

Nieman Lab took the rewrite back to its three experts in a follow-up on October 17. None of them was reassured.

• "Develop" covers a lot. Topic: "The revised version of the new terms still allows Meta to use that scraped content to develop any 'business product.' That doesn't seem to be a defined term, and presumably Meta is developing AI business products. I think 'develop' is broad enough to include 'train.'" Meta's own examples of business products are "ads, Meta Business Tools and Meta Business Suite."
• The scope never shrank. James Grimmelmann of Cornell Law School said the "develop...advertising and business products" language "is still broad enough to drive a truck through," and that nothing narrows the "associated with your products or services" wording that "would let Meta scrape entire sites."
• Test credentials are an odd permission. Grimmelmann had already pointed out that they would technically let Meta "sign up as a fake user" to get past registration walls. Meta declined to say whether test credentials could get past paywalls.
• robots.txt is not a contract. Benjamin Sobel of the University of Wisconsin compared robots.txt to a homeowner's sign reading "Don't look at my house." Compliance is voluntary, and Meta now holds a signed contract that does not mention it.

Sobel also compared the setup to "tying" in antitrust law: to use the products, you have to accept the crawling. Nieman Lab's first story also noted that Meta's new Meta One subscription limits how many external links some Facebook and Instagram accounts can post without paying, so Meta is starting to charge some users to post links to sites it can crawl for free under these terms.

The publishers' response

Three trade groups went public on October 16, the day Meta made the edit.

• United States. The News/Media Alliance, the largest US news publishing trade association, called the terms "outrageous and probably illegal." CEO Danielle Coffey: "It cannot be allowed to be implemented on October 30." She asked whether Taylor Swift's or Shutterstock's websites would be fair game if either has a Meta business account.
• European Union. The European Publishers Council said it will petition the European Commission and EU telecom regulators. Executive director Angela Mills Wade singled out the "users' use" wording, which could cover content Meta reaches just because a reader shared a link: "A user sharing a news article cannot confer rights they do not hold."
• United Kingdom. The News Media Association asked ministers to intervene. CEO Theo Bamber called it a "shameless landgrab attempt" and "the worst kind of arrogant tech platform bullying."

What you can actually do before October 30

If you run a site and have a Meta business account (a Page, an Instagram business profile, a WhatsApp channel, or an ad account), the contract will cover your domain. Your practical tool is the same one Meta keeps pointing to. Its crawler documentation separates its bots by job, so you can block training without disappearing from Meta AI answers.

To block the crawler Meta itself labels as the training one:

User-agent: meta-externalagent
Disallow: /

Meta says crawlers may cache robots.txt for up to 24 hours, so a change takes about a day to bite. Sobel's point still applies: that line records a preference and doesn't create an enforceable right. Meta's docs also say Meta-ExternalFetcher "may bypass robots.txt rules" when it fetches pages for users and agents. So also watch your access logs for meta-externalagent after October 30. If it keeps showing up on a disallowed site, you will have a dated record of it.

The other option is the one Grimmelmann described: trust that Meta doesn't mean everything the text allows, or stop using its business products. Beale, the steampunk editor who started all this, told Nieman Lab he is seriously considering shutting his accounts before the deadline. "I know my site has been scraped to death," he said.

Key Takeaways

• Meta deleted "train" from the crawling clause in its October 30 Commercial Terms on October 16, a day after Nieman Lab reported it, and limited permitted uses to "advertising and business products."
• The permission to crawl any site "associated with" a business account, using scrapers and test credentials, is unchanged, and businesses still cannot opt out short of leaving Meta's commercial products.
• Meta says robots.txt preferences are respected, but neither version of the terms mentions robots.txt, and legal experts say "develop...business products" could still cover AI training.
• Meta's earlier ad terms already allowed crawling of advertisers' domains. The new terms apply to any business use, including Pages that have never run an ad.
• The News/Media Alliance, the European Publishers Council and the UK News Media Association have all objected, and the EPC plans to petition the European Commission.
• If you run a site, disallow meta-externalagent in robots.txt and check your logs after October 30.

Sources: Nieman Lab (Oct. 15), Nieman Lab follow-up (Oct. 17), Meta Commercial Terms preview, Wayback snapshot, Oct. 16 16:20 UTC, Wayback snapshot, Oct. 17 03:19 UTC, Search Engine Journal, News/Media Alliance statement, European Publishers Council statement, Meta web crawler documentation