👩💻 Join our community of thousands of amazing developers!
If you are like me, you want to do stuff because you can and why not. One of these things is recursive website downloading. What the junk does that mean? Before I explain, let me explain web crawlers, like Google’s Googlebot. Their job is to search the internet ALL DAY, EVERY DAY! They will go to a website, and check for links, like this. Download the site Scrape it for links If a link is found, download the linked page and repeat. If no, go crawl another link Using the way that lots ...