Skip to content
  • Hjem
  • Seneste
  • Etiketter
  • Populære
  • Verden
  • Bruger
  • Grupper
Temaer
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Kollaps
FARVEL BIG TECH
  1. Forside
  2. Ikke-kategoriseret
  3. https://OpenStreetMap.org has been disrupted today.

https://OpenStreetMap.org has been disrupted today.

Planlagt Fastgjort Låst Flyttet Ikke-kategoriseret
openstreetmapddosscrapers
23 Indlæg 14 Posters 80 Visninger
  • Ældste til nyeste
  • Nyeste til ældste
  • Most Votes
Svar
  • Svar som emne
Login for at svare
Denne tråd er blevet slettet. Kun brugere med emne behandlings privilegier kan se den.
  • isibboi@mastodon.onlineI isibboi@mastodon.online

    @osm_tech the scrapers don't know that you can torrent the whole OSM for free, without ddosing the webservice

    geospacedman@mastodon.socialG This user is from outside of this forum
    geospacedman@mastodon.socialG This user is from outside of this forum
    geospacedman@mastodon.social
    wrote sidst redigeret af
    #7

    @ISibboI @osm_tech Is there a torrent for the tiles? Or just the world file? Are they scraping tiles?

    osm_tech@en.osm.townO 1 Reply Last reply
    0
    • jwcph@helvede.netJ jwcph@helvede.net shared this topic
    • geospacedman@mastodon.socialG geospacedman@mastodon.social

      @ISibboI @osm_tech Is there a torrent for the tiles? Or just the world file? Are they scraping tiles?

      osm_tech@en.osm.townO This user is from outside of this forum
      osm_tech@en.osm.townO This user is from outside of this forum
      osm_tech@en.osm.town
      wrote sidst redigeret af
      #8

      @geospacedman @ISibboI They are scraping the website pages: /ways/, /nodes/ and /relations/. All this data is published on planet.osm.org already.

      geospacedman@mastodon.socialG 1 Reply Last reply
      0
      • osm_tech@en.osm.townO osm_tech@en.osm.town

        @geospacedman @ISibboI They are scraping the website pages: /ways/, /nodes/ and /relations/. All this data is published on planet.osm.org already.

        geospacedman@mastodon.socialG This user is from outside of this forum
        geospacedman@mastodon.socialG This user is from outside of this forum
        geospacedman@mastodon.social
        wrote sidst redigeret af
        #9

        @osm_tech @ISibboI Oh that's so much worse than I'd imagined.

        1 Reply Last reply
        0
        • osm_tech@en.osm.townO osm_tech@en.osm.town

          https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

          rudirastlos@troet.cafeR This user is from outside of this forum
          rudirastlos@troet.cafeR This user is from outside of this forum
          rudirastlos@troet.cafe
          wrote sidst redigeret af
          #10

          @osm_tech 🫣 thank you for your work!

          1 Reply Last reply
          0
          • osm_tech@en.osm.townO osm_tech@en.osm.town

            https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

            justin@mastodon.tacoma.communityJ This user is from outside of this forum
            justin@mastodon.tacoma.communityJ This user is from outside of this forum
            justin@mastodon.tacoma.community
            wrote sidst redigeret af
            #11

            @osm_tech
            https://www.bleepingcomputer.com/news/security/new-linux-botnet-sshstalker-uses-old-school-irc-for-c2-comms/

            Wonder if related.

            osm_tech@en.osm.townO 1 Reply Last reply
            0
            • justin@mastodon.tacoma.communityJ justin@mastodon.tacoma.community

              @osm_tech
              https://www.bleepingcomputer.com/news/security/new-linux-botnet-sshstalker-uses-old-school-irc-for-c2-comms/

              Wonder if related.

              osm_tech@en.osm.townO This user is from outside of this forum
              osm_tech@en.osm.townO This user is from outside of this forum
              osm_tech@en.osm.town
              wrote sidst redigeret af
              #12

              @justin No. Background: https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operations-of-the-wikimedia-projects/ and https://cloud.google.com/blog/topics/threat-intelligence/disrupting-largest-residential-proxy-network

              1 Reply Last reply
              0
              • osm_tech@en.osm.townO osm_tech@en.osm.town

                https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

                mayalandsman@mastodon.socialM This user is from outside of this forum
                mayalandsman@mastodon.socialM This user is from outside of this forum
                mayalandsman@mastodon.social
                wrote sidst redigeret af
                #13

                Oof, sorry you are dealing with that load. Beyond blocklists, caching tiles plus aggressive rate limits per ASN, and serving 429s with Retry-After can help. For bulk use, requiring API keys for heavy endpoints can also cut abuse. Thanks for keeping OSM running.

                1 Reply Last reply
                0
                • osm_tech@en.osm.townO osm_tech@en.osm.town

                  https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

                  clickhere@mastodon.ieC This user is from outside of this forum
                  clickhere@mastodon.ieC This user is from outside of this forum
                  clickhere@mastodon.ie
                  wrote sidst redigeret af
                  #14

                  @osm_tech Ugh, I am so sorry to hear this, and thank you to everyone working on this project.

                  Such a nightmare, and so completely pointless. 🤬

                  1 Reply Last reply
                  0
                  • osm_tech@en.osm.townO osm_tech@en.osm.town

                    @ISibboI Correct. Not an ounce of brain. It has been going on for months, but it is just getting worse. Waste of their resources/time and ours.

                    pesasa@mastodon.socialP This user is from outside of this forum
                    pesasa@mastodon.socialP This user is from outside of this forum
                    pesasa@mastodon.social
                    wrote sidst redigeret af
                    #15

                    @ISibboI @osm_tech I think, that spreading it on 100 000+ IP addresses tells they know they are doing shady thing.

                    G 1 Reply Last reply
                    0
                    • osm_tech@en.osm.townO osm_tech@en.osm.town

                      https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

                      gme@bofh.socialG This user is from outside of this forum
                      gme@bofh.socialG This user is from outside of this forum
                      gme@bofh.social
                      wrote sidst redigeret af
                      #16

                      @osm_tech@en.osm.town
                      Time to deploy Anubis in front of all the URLs that the scrapers are hitting.

                      osm_tech@en.osm.townO 1 Reply Last reply
                      0
                      • osm_tech@en.osm.townO osm_tech@en.osm.town

                        https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

                        tomchadwin@en.osm.townT This user is from outside of this forum
                        tomchadwin@en.osm.townT This user is from outside of this forum
                        tomchadwin@en.osm.town
                        wrote sidst redigeret af
                        #17

                        @osm_tech Good luck, and thank you as always.

                        1 Reply Last reply
                        0
                        • osm_tech@en.osm.townO osm_tech@en.osm.town

                          https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI

                          don_clemente@chaos.socialD This user is from outside of this forum
                          don_clemente@chaos.socialD This user is from outside of this forum
                          don_clemente@chaos.social
                          wrote sidst redigeret af
                          #18

                          @osm_tech wonder if it is collateral damage or intended, to get rid of alternative sources of information?
                          Anyway, zip-bomb the hell out of them.

                          osm_tech@en.osm.townO 1 Reply Last reply
                          0
                          • don_clemente@chaos.socialD don_clemente@chaos.social

                            @osm_tech wonder if it is collateral damage or intended, to get rid of alternative sources of information?
                            Anyway, zip-bomb the hell out of them.

                            osm_tech@en.osm.townO This user is from outside of this forum
                            osm_tech@en.osm.townO This user is from outside of this forum
                            osm_tech@en.osm.town
                            wrote sidst redigeret af
                            #19

                            @Don_Clemente Just dumb scrapers. They want our data, so maybe we /should/ redirect them to the latest full history planet file, it is only 245GB 😉

                            don_clemente@chaos.socialD 1 Reply Last reply
                            0
                            • gme@bofh.socialG gme@bofh.social

                              @osm_tech@en.osm.town
                              Time to deploy Anubis in front of all the URLs that the scrapers are hitting.

                              osm_tech@en.osm.townO This user is from outside of this forum
                              osm_tech@en.osm.townO This user is from outside of this forum
                              osm_tech@en.osm.town
                              wrote sidst redigeret af
                              #20

                              @gme Anubis is great, but not ideal for us. We have reasonable technical mitigations in place, but it is unfortunately an arms race (and time/resource blackhole)

                              1 Reply Last reply
                              0
                              • osm_tech@en.osm.townO osm_tech@en.osm.town

                                @Don_Clemente Just dumb scrapers. They want our data, so maybe we /should/ redirect them to the latest full history planet file, it is only 245GB 😉

                                don_clemente@chaos.socialD This user is from outside of this forum
                                don_clemente@chaos.socialD This user is from outside of this forum
                                don_clemente@chaos.social
                                wrote sidst redigeret af
                                #21

                                @osm_tech yes, but that's heavy on your resources and bandwidth. A zip file of 10GB Zeros is much more resource-efficient on your side 😉

                                1 Reply Last reply
                                0
                                • pesasa@mastodon.socialP pesasa@mastodon.social

                                  @ISibboI @osm_tech I think, that spreading it on 100 000+ IP addresses tells they know they are doing shady thing.

                                  G This user is from outside of this forum
                                  G This user is from outside of this forum
                                  gerardthornley@hachyderm.io
                                  wrote sidst redigeret af
                                  #22

                                  @pesasa @ISibboI @osm_tech It's a pattern I've seen others relate to the building of AI datasets.

                                  G 1 Reply Last reply
                                  0
                                  • G gerardthornley@hachyderm.io

                                    @pesasa @ISibboI @osm_tech It's a pattern I've seen others relate to the building of AI datasets.

                                    G This user is from outside of this forum
                                    G This user is from outside of this forum
                                    gerardthornley@hachyderm.io
                                    wrote sidst redigeret af
                                    #23

                                    @pesasa @ISibboI @osm_tech
                                    If its helpful (if you want to stuff something unpleasant down their throats), I believe @jwz has been making a 'json bomb' for (AIUI) pretty much this purpose:
                                    https://mastodon.social/@jwz/116049057703097965

                                    1 Reply Last reply
                                    0
                                    • anderslund@expressional.socialA anderslund@expressional.social shared this topic
                                    Svar
                                    • Svar som emne
                                    Login for at svare
                                    • Ældste til nyeste
                                    • Nyeste til ældste
                                    • Most Votes


                                    • Log ind

                                    • Har du ikke en konto? Tilmeld

                                    • Login or register to search.
                                    Powered by NodeBB Contributors
                                    Graciously hosted by data.coop
                                    • First post
                                      Last post
                                    0
                                    • Hjem
                                    • Seneste
                                    • Etiketter
                                    • Populære
                                    • Verden
                                    • Bruger
                                    • Grupper