DEV Community

Durga Pokharel
Durga Pokharel

Posted on

Day 71 Of 100DaysOfCode: Web Scraping

This is my 71th day of #100daysofcode and #python learning. Like daily today also learned from Datacamp regarding to the topic web scraping. Today I learned about simple xpath, slasher double feature, (At)tribute, content with contains, extracting data from a selector list, inspecting the HTML.

Here is python code to extract data from selector list.

Python code

# Import a scrapy Selector
from scrapy import Selector

# Import requests
import requests

# Create the string html containing the HTML source
html = requests.get( url ).content

# Create the Selector object sel from html
sel = Selector( text = html )

# Print out the number of elements in the HTML document
print( "There are 1020 elements in the HTML document.")
print( "You have found: ", len( sel.xpath('//*') ) )
Enter fullscreen mode Exit fullscreen mode

Day 71 Of #100DaysOfCode and #Python
Web-Scraping From https://t.co/6UM9B22c7fDatacamp#womenintech #100DaysOfCode #CodeNewbie #WomenWhoCode #DEVCommunity pic.twitter.com/A5vd6ID1mE

— Durga Pokharel (@mathdurga) March 9, 2021

Top comments (2)

Collapse
 
otumianempire profile image
Michael Otu • Edited

71 one days.. awesome.. in a funny way you could become a full stack web developer with data science knowledge and experience too. You know html, css, javascript, SQL, numpy, scrapy... That's awesome.. I am cheering for you..

Collapse
 
iamdurga profile image
Durga Pokharel

Thank you for your support.