I have been creating a scraper and need an automation to download some images. I spend hours to finally get it right. So, here I am writing this post, hoping that it would help someone in needs (or even future me finds this from search engine).
There may be a lot of other way, but here's the one that works for me today.
const fs = require('fs');
const fetch = require('node-fetch');
const url = "https://www.something.com/.../image.jpg"
async function download() {
const response = await fetch(url);
const buffer = await response.buffer();
fs.writeFile(`./image.jpg`, buffer, () =>
console.log('finished downloading!'));
}
Please note that fs
is included in the node framework, while node-fetch
may need to be installed first.
You can combine this with any scraper library like puppeteer.
Top comments (5)
Straight forward. Just what I was looking for. Thank you!
With Node version 18, there is a native
fetch
method. However, the response object returned does not have a.buffer
method. If you use the nativefetch
, this minor modification is required:really important due to request been deprecated and still been download a lot
Exactly what i needed, thanks.
I want to simply create a read stream and pass on the image to another api.Is that possible here ? Working with multer gives no help on Heroku.