DEV Community

Cover image for ES2018. Real-life simple usage of async iteration: get paginated data from REST APIs in 20 lines of code

ES2018. Real-life simple usage of async iteration: get paginated data from REST APIs in 20 lines of code

Carlos Saito on September 24, 2018

The next JavaScript standard, ES2018 is here and it comes with a new big feature: asynchronous iteration. It is a enormously useful feature and I w...
Read full post
Collapse
 
webreflection profile image
Andrea Giammarchi •

why aren't you using the header link returned appositely by GitHub API to know the next page to load instead?

Collapse
 
exacs profile image
Carlos Saito •

It is exactly what I'm doing :)

Collapse
 
webreflection profile image
Andrea Giammarchi • • Edited

you are right, I actually somehow skipped the last url assignment. One possible improvement then, would be to fetch headers once, use URLSearchParams to get/set pages, and load all the pages at once in parallel, returning results as Promise.all(...)

That would be N pages at once, instead of N pages one after the other ;-)

Edit: my suggestion is based on the fact GitHub returns the last page too, but I guess for your article what you are doing is already good enough as example.

Thread Thread
 
exacs profile image
Carlos Saito •

Thanks for the suggestion! Your solution would work perfectly :)

I don't think that my solution is valid for all scenarios but might be good sometimes. For example:

  • We want to fetch pages until we met some condition (for example, search the last 10 commits that says "refactor" in the text) where we don't want to fetch all pages
  • The API doesn't return the "last" page in the header