From inconsistent website layouts that break our extraction logic to badly written HTML, web scraping comes with its share of difficulties. Over the last few years, the single most important challenge in web scraping has been to actually get to the data - and not get blocked. This is due to the antibots or the underlying technologies that websites use to protect their data. Proxies are a major component in any scalable web scraping infrastructure. However, not many people understand the tec...