@RnDanger @inthehands well, we don’t know and we will see. My guess are separate scrapers (officially) and a lot of mistrust (are there others?) and masses of unidentified scrapers. Nevertheless, Google can better afford to play by the rules, since hey already own the largest index. Think also of Video etc. Will volume win the war? Or quality and freshness? Etc. Future is difficult.

@inthehands meta noindex it is, definitely. robots disallow can actually hurt the process, since google cannot access the file with the noindex header and therefore won't deindex.
btw, they do indeed respect noindex and robots.txt ATM, since its qute easy to check if pages still get found. Then again, you never know what does not show up in search but is used for training (without giving credit, obv.) anyway. As far as i see, google still remains more standard compliant as e.g. OpenAI.