One of my clients asked me whether there's an easier way to download image files from a website. That is, easier than right clicking the photo and 'save image as' route. I won't share the details but they wanted to crawl an ecommerce website and get the product image files.
I did a few web scraping projects before but I never had to download image files before so I looked into available options.
I found two that looked simple. One was using the Requests library and the other was this thing called Wget https://pypi.org/project/wget/, and I opted for Wget.
Here is the basic setup for downloading an image file from the web.
import wget
url ='http://image_url.jpg'
filename = wget.download(url)
That's it. Use the Requests library and Beautiful Soup to get a list of urls for image files and loop them over and download the files using Wget! Oh make sure you check the website owner's policy on web scraping.
Top comments (0)