DEV Community

Discussion on: Web scraping, efficiently!

Collapse
 
kubistmi profile image
kubistmi • Edited

Hello Navon,
first of all, thank you for a very nice and concise article on python web scraping. It is all that the beginners can ask for when first encountering this topic.

Secondly, it seems to me that there is a minor typo in your code (repeated a few times). In the 'Scraping' part, you firstly use variable req to keep the HTML content but then use variable r for HTML parsing, as follows:

for link in links:
    req = requests.get(link)                  # variable req
    soup = BeautifulSoup(r.text, 'lxml')      # variable r

Cheers,
Michal

Collapse
 
navonf profile image
Navon Francis

Thank you so much for the awesome feedback Michal! Also, I really appreciate your help in finding that typo :) I will update accordingly