I'd completely missed this but apparently Meta are changing their T&Cs for business users of their platforms to force all companies to allow Meta to scrape and index their content.

[tdp_org@mastodon.social]

I'd completely missed this but apparently Meta are changing their T&Cs for business users of their platforms to force all companies to allow Meta to scrape and index their content. If you disagree, you have to stop using Meta platforms.
https://www.niemanlab.org/2026/10/metas-new-commercial-terms-will-allow-the-scraping-of-news-sites-legal-experts-say/

Scraping starts October 30th apparently.

Yuk.

Anyone know which bot they'll use? Feels likely to be `meta-externalagent` from docs: https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers/

#Meta #AI #Bots #WebDev

[hrbrmstr@mastodon.social]

@tdp_org meta-externalagent and meta-webindexer are heavy hitters from them (partially tracked here: https://observatory.rud.is/rules)

facebookexternalhit and HatchFeed (agent.meta.ai) sometimes from from their attributed v4 and v6 ranges but low volume and seem to be user-initiated (note the deliberately broken x axis)

[hrbrmstr@mastodon.social]

@tdp_org this is a decent resource to keep on the radar https://github.com/ai-robots-txt/ai.robots.txt

[tdp_org@mastodon.social]

@hrbrmstr There's a certain irony to this

[hrbrmstr@mastodon.social]

@tdp_org oh blocking claude to get those notices is def quicker than my immediate "scroll down to to the contributors block in the sidebar". TIL!