DEV Community

Cover image for Web Scraping 101 with Javascript and NodeJS

Web Scraping 101 with Javascript and NodeJS

Pierre on June 16, 2020

Javascript has become one of the most popular and widely used languages due to the massive improvements it has seen and the introduction of the run...
Collapse
 
pavelloz profile image
Paweł Kowalski • Edited

Holy moly, i was just about to write about this topic, but i guess now i dont have to :-) Good job!

I had plan to write about:
1) nodejs + jsdom,
2) nodejs + testcafe (because sometimes you just need that real browser APIs for SPAs and i prefer testcafe to puppeteer :-) )

Collapse
 
anantbahuguna profile image
Anant Bahuguna

What do you mean by real browser APIs ?

Collapse
 
pavelloz profile image
Paweł Kowalski

I mean mostly rendering of JS powered SPAs, which is not happening in JSDOM.

Thread Thread
 
anantbahuguna profile image
Anant Bahuguna • Edited

oh k got it !
testcafe can handle that rendering by js ?

Thread Thread
 
pavelloz profile image
Paweł Kowalski

Yeah, basically it runs browser just like puppeteer :)

Thread Thread
 
anantbahuguna profile image
Anant Bahuguna

ok i'll look into it. Thnx 👍

Collapse
 
tcarrio profile image
Tom

Might be worthwhile to look at Playwright. Our team uses Cypress for integration tests but really enjoyed using that for a recent web automation project

Collapse
 
zilti_500 profile image
Daniel Ziltener

...but why? Why do people do that to themselves? Do they like pain?