Can the Robot Exclusion Protocol apply to Gopher URLs?

· Boston Conman · Oct. 29, 2026, 3:53 a.m.
Summary
This blog post explores the applicability of the Robot Exclusion Protocol to Gopher URLs, discussing limitations and potential issues with implementing /robots.txt on Gopher servers, particularly related to how Gopher selectors differ from HTTP and Gemini. The author presents personal insights and reflects on the challenges of controlling LLM access to their Gopher content.
AUTHOR