Topics
Follow your own topics →
DIFF.BLOG
New Following Discover Jobs
More
Top Writers Suggest a blog Upvotes plugin
Report bug Contact About
Sign up
Menu
New Following Discover Jobs Top Writers
More
Suggest a blog Upvotes plugin Report bug Contact About
Sign up
The home for great developer writing.
Discover the best posts from developers and engineering teams, all in one place.
Join now → Learn more
TOPICS

Amazonbot is finally respecting robots.txt

107 · · May 14, 2026, 8:23 p.m.
Amazon robots.txt Web Scraping business model
Summary
The author expresses gratitude towards Amazon for enabling a viable business model by respecting the robots.txt protocol, which is relevant for web scraping and SEO practices.
Read full post on xeiaso.net →
MORE POSTS LIKE THIS
Most Googlebots are fake
Digitalseams · Jul 26, 2026
Googlebot Web Scraping
Court Grants SerpApi’s Motion to Dismiss Google Lawsuit
Daring Fireball · Jul 25, 2026
law tech industry
Premium: The Hater’s Guide To Oracle (Part 2)
Ed Zitron · Jul 24, 2026
Oracle Tech Critique
List of New Features Not Available on Older Kindle Scribes
Blog The Ebook Reader · Jul 24, 2026
Amazon Kindle kindle scribe
Dynamic sitemap.xml in Astro SSR
Flaviocopes · Jul 18, 2026
astro SSR
Updated Fire TV interface begins rolling out to the 3rd-gen Fire TV Cube
Elias Saba · Jul 17, 2026
Article 2026 Interface
Discover more posts →
AUTHOR
BLOG POST FEATURED ON

Placeholder image
Hacker News

170 points

Placeholder image
r/hypeurls

1 points

Placeholder image
r/hackernews

1 points

Add this plugin to your blog
RECENT POSTS FROM THE AUTHOR
Choose how you want to continue.
Continue with GitHub Continue with Google