https://OpenStreetMap.org has been disrupted today.

[geotux@social.tchncs.de]

@osm_tech thanks for the good work

[isibboi@mastodon.online]

@osm_tech the scrapers don't know that you can torrent the whole OSM for free, without ddosing the webservice

[osm_tech@en.osm.town]

@ISibboI Correct. Not an ounce of brain. It has been going on for months, but it is just getting worse. Waste of their resources/time and ours.

[geospacedman@mastodon.social]

@ISibboI @osm_tech Is there a torrent for the tiles? Or just the world file? Are they scraping tiles?

[osm_tech@en.osm.town]

@geospacedman @ISibboI They are scraping the website pages: /ways/, /nodes/ and /relations/. All this data is published on planet.osm.org already.

[geospacedman@mastodon.social]

@osm_tech @ISibboI Oh that's so much worse than I'd imagined.

[rudirastlos@troet.cafe]

@osm_tech 🫣 thank you for your work!

[justin@mastodon.tacoma.community]

@osm_tech
https://www.bleepingcomputer.com/news/security/new-linux-botnet-sshstalker-uses-old-school-irc-for-c2-comms/

Wonder if related.

[osm_tech@en.osm.town]

@justin No. Background: https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operations-of-the-wikimedia-projects/ and https://cloud.google.com/blog/topics/threat-intelligence/disrupting-largest-residential-proxy-network

[mayalandsman@mastodon.social]

Oof, sorry you are dealing with that load. Beyond blocklists, caching tiles plus aggressive rate limits per ASN, and serving 429s with Retry-After can help. For bulk use, requiring API keys for heavy endpoints can also cut abuse. Thanks for keeping OSM running.

[clickhere@mastodon.ie]

@osm_tech Ugh, I am so sorry to hear this, and thank you to everyone working on this project.

Such a nightmare, and so completely pointless. 🤬

[pesasa@mastodon.social]

@ISibboI @osm_tech I think, that spreading it on 100 000+ IP addresses tells they know they are doing shady thing.

[gme@bofh.social]

@osm_tech@en.osm.town
Time to deploy Anubis in front of all the URLs that the scrapers are hitting.

[tomchadwin@en.osm.town]

@osm_tech Good luck, and thank you as always.

[don_clemente@chaos.social]

@osm_tech wonder if it is collateral damage or intended, to get rid of alternative sources of information?
Anyway, zip-bomb the hell out of them.

[osm_tech@en.osm.town]

@Don_Clemente Just dumb scrapers. They want our data, so maybe we /should/ redirect them to the latest full history planet file, it is only 245GB

[osm_tech@en.osm.town]

@gme Anubis is great, but not ideal for us. We have reasonable technical mitigations in place, but it is unfortunately an arms race (and time/resource blackhole)

[don_clemente@chaos.social]

@osm_tech yes, but that's heavy on your resources and bandwidth. A zip file of 10GB Zeros is much more resource-efficient on your side

[gerardthornley@hachyderm.io]

@pesasa @ISibboI @osm_tech It's a pattern I've seen others relate to the building of AI datasets.

[gerardthornley@hachyderm.io]

@pesasa @ISibboI @osm_tech
If its helpful (if you want to stuff something unpleasant down their throats), I believe @jwz has been making a 'json bomb' for (AIUI) pretty much this purpose:
https://mastodon.social/@jwz/116049057703097965