#
DIFF.BLOG
New
Following
Discover
Jobs
More
Top Writers
Suggest a blog
Upvotes plugin
Report bug
Contact
About
Sign up
The home for great developer writing.
Discover the best posts from developers and engineering teams, all in one place.
Join now
→
Learn more
TOPICS
Amazonbot is finally respecting robots.txt
107
·
·
May 14, 2026, 8:23 p.m.
Amazon
robots.txt
Web Scraping
business model
Summary
The author expresses gratitude towards Amazon for enabling a viable business model by respecting the robots.txt protocol, which is relevant for web scraping and SEO practices.
Read full post on xeiaso.net →
MORE POSTS LIKE THIS
Most Googlebots are fake
Digitalseams ·
Jul 26, 2026
Googlebot
Web Scraping
Court Grants SerpApi’s Motion to Dismiss Google Lawsuit
Daring Fireball ·
Jul 25, 2026
law
tech industry
Premium: The Hater’s Guide To Oracle (Part 2)
Ed Zitron ·
Jul 24, 2026
Oracle
Tech Critique
List of New Features Not Available on Older Kindle Scribes
Blog The Ebook Reader ·
Jul 24, 2026
Amazon Kindle
kindle scribe
Dynamic sitemap.xml in Astro SSR
Flaviocopes ·
Jul 18, 2026
astro
SSR
Updated Fire TV interface begins rolling out to the 3rd-gen Fire TV Cube
Elias Saba ·
Jul 17, 2026
Article
2026 Interface
Discover more posts →
AUTHOR
BLOG POST FEATURED ON
Hacker News
170 points
r/hypeurls
1 points
r/hackernews
1 points
Add this plugin to your blog
RECENT POSTS FROM THE AUTHOR
Choose how you want to continue.
Continue with GitHub
Continue with Google