<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Interesting read on the way LLM bots retrieve pages from a website]]></title><description><![CDATA[<p>Interesting read on the way LLM bots retrieve pages from a website</p><p>Explanations are clear precise and surgical</p><p><a href="https://surfacedby.com/blog/nginx-logs-ai-traffic-vs-referral-traffic" rel="nofollow noopener"><span>https://</span><span>surfacedby.com/blog/nginx-logs</span><span>-ai-traffic-vs-referral-traffic</span></a> </p><p><a href="https://mastodon.bsd.cafe/tags/LLM" rel="tag">#<span>LLM</span></a> <a href="https://mastodon.bsd.cafe/tags/AI" rel="tag">#<span>AI</span></a> <a href="https://mastodon.bsd.cafe/tags/slop" rel="tag">#<span>slop</span></a> <a href="https://mastodon.bsd.cafe/tags/nginx" rel="tag">#<span>nginx</span></a> <a href="https://mastodon.bsd.cafe/tags/traffic" rel="tag">#<span>traffic</span></a> <a href="https://mastodon.bsd.cafe/tags/programming" rel="tag">#<span>programming</span></a> <a href="https://mastodon.bsd.cafe/tags/referral" rel="tag">#<span>referral</span></a> <a href="https://mastodon.bsd.cafe/tags/traffic" rel="tag">#<span>traffic</span></a> <a href="https://mastodon.bsd.cafe/tags/networking" rel="tag">#<span>networking</span></a> <a href="https://mastodon.bsd.cafe/tags/robots" rel="tag">#<span>robots</span></a> <a href="https://mastodon.bsd.cafe/tags/txt" rel="tag">#<span>txt</span></a> <a href="https://mastodon.bsd.cafe/tags/claude" rel="tag">#<span>claude</span></a> <a href="https://mastodon.bsd.cafe/tags/chatgpt" rel="tag">#<span>chatgpt</span></a> <a href="https://mastodon.bsd.cafe/tags/bing" rel="tag">#<span>bing</span></a> <a href="https://mastodon.bsd.cafe/tags/meta" rel="tag">#<span>meta</span></a> <a href="https://mastodon.bsd.cafe/tags/metaAI" rel="tag">#<span>metaAI</span></a></p>]]></description><link>https://forum.fedi.dk/topic/0d06f6f6-3379-4932-83d2-31f5465965e5/interesting-read-on-the-way-llm-bots-retrieve-pages-from-a-website</link><generator>RSS for Node</generator><lastBuildDate>Mon, 27 Apr 2026 13:54:31 GMT</lastBuildDate><atom:link href="https://forum.fedi.dk/topic/0d06f6f6-3379-4932-83d2-31f5465965e5.rss" rel="self" type="application/rss+xml"/><pubDate>Tue, 21 Apr 2026 02:08:31 GMT</pubDate><ttl>60</ttl></channel></rss>