DEV Community

loading...

Discussion on: I wrote a crawler for the first time.

Collapse
crimsonmed profile image
Médéric Burlet

I used cheerio a while but realized when processing a lot of pages it's very slow.
I'd recommend using regex I cut down processing time by 80%.

Good job though it's a great start!

Collapse
kaylasween profile image
Kayla Sween Author

Good to know! I’m just collecting data from one page for now, but I’ll definitely keep that in mind. Thank you!

Some comments have been hidden by the post's author - find out more