re: Interacting with the Dev.to Article API VIEW POST

TOP OF THREAD FULL DISCUSSION
re: Cool, another post about the Dev.to API! 🙂 Correct me if I am wrong though but that will go over every single page till it runs out of data right?...
 

The articles API endpoint looks to only return 30 articles per page and seems to cut off around ~460 pages or so. The idea is indeed to grab all public posts as I need a good amount of text for something else I'm toying around with so, in this case, I have a rough idea of how many articles we'll be getting. However, we're only getting 30 articles worth of content in one go which should not cause any issues on the server side or locally. You definitely wouldn't want to be running this repeatedly in a short span or anything though.

 

Yeah as it is still about 13,000 requests - if that ran often, it could easily hammer the site.

The articles endpoint is cached as far as I know so, as is the code shouldn't cause any significant load on Varnish. Heh. But yeah, imagine trying to simply download the articles one at a time, api/articles/1, that what like ~90000 hits. Yuck.

code of conduct - report abuse