DEV Community

SL
SL

Posted on

2

Web scraping - Data extraction - web scraper google chrome extension

Good afternoon,

I am trying to extract all the products (name, price, image) from a grocery store.

I am using web scraper (google chrome extension). When I start scraping I can see it is running, however it does not return any data.

This is the sitemap I created: {"id":"collectandgo","startUrl":["https://colruyt.collectandgo.be/cogo/nl/home"],"selectors":[{"id":"categories","type":"SelectorLink","parentSelectors":["_root"],"selector":".branch a","multiple":true,"delay":0},{"id":"subcategories","type":"SelectorLink","parentSelectors":["categories"],"selector":".open li a","multiple":true,"delay":0},{"id":"subcategorie3","type":"SelectorLink","parentSelectors":["subcategories"],"selector":".leaf.active a","multiple":true,"delay":0},{"id":"Product","type":"SelectorElement","parentSelectors":["subcategorie3"],"selector":"div.productinner","multiple":true,"delay":0},{"id":"Image","type":"SelectorText","parentSelectors":["Product"],"selector":"a.productimage","multiple":true,"regex":"","delay":0},{"id":"Merk","type":"SelectorText","parentSelectors":["Product"],"selector":"div.productname","multiple":true,"regex":"","delay":0},{"id":"Productomschrijving","type":"SelectorText","parentSelectors":["Product"],"selector":"div.productdescription","multiple":true,"regex":"","delay":0},{"id":"Hoeveelheid","type":"SelectorText","parentSelectors":["Product"],"selector":"div.product_weight","multiple":true,"regex":"","delay":0}]}

web-scraping google-chrome-extension data-extraction

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

Top comments (1)

Collapse
 
sl profile image
SL

I've changed the sitemap as follows:
{"id":"collectandgo","startUrl":["colruyt.collectandgo.be/cogo/nl/ho...branch.branch","multiple":true,"delay":0},{"id":"items","type":"SelectorElement","parentSelectors":["categories"],"selector":"div.productinner","multiple":true,"delay":0},{"id":"productbody","type":"SelectorElement","parentSelectors":["items"],"selector":"div.productbody","multiple":true,"delay":0},{"id":"image","type":"SelectorImage","parentSelectors":["productbody"],"selector":"a.productimage","multiple":false,"delay":0},{"id":"productname","type":"SelectorText","parentSelectors":["productbody"],"selector":"div.productname","multiple":false,"regex":"","delay":0},{"id":"productdescription","type":"SelectorText","parentSelectors":["productbody"],"selector":"div.productdescription","multiple":false,"regex":"","delay":0},{"id":"productweight","type":"SelectorText","parentSelectors":["productbody"],"selector":"div.productweight","multiple":false,"regex":"","delay":0},{"id":"prijs","type":"SelectorText","parentSelectors":["productbody"],"selector":"div.productprice-piece","multiple":false,"regex":"","delay":0},{"id":"eenheidsprijs","type":"SelectorText","parentSelectors":["productbody"],"selector":"div.productprice-unit","multiple":false,"regex":"","delay":0},{"id":"korting-aankoop-hoeveelheid","type":"SelectorText","parentSelectors":["productbody"],"selector":"a.promotion_min-amount","multiple":false,"regex":"","delay":0}]}

When I click on data preview I can see the data. However I keep receiving the message no data scraped.

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs