DEV Community

Discussion on: ES2018. Real-life simple usage of async iteration: get paginated data from REST APIs in 20 lines of code

Collapse
 
webreflection profile image
Andrea Giammarchi

why aren't you using the header link returned appositely by GitHub API to know the next page to load instead?

Collapse
 
exacs profile image
Carlos Saito

It is exactly what I'm doing :)

Collapse
 
webreflection profile image
Andrea Giammarchi • Edited

you are right, I actually somehow skipped the last url assignment. One possible improvement then, would be to fetch headers once, use URLSearchParams to get/set pages, and load all the pages at once in parallel, returning results as Promise.all(...)

That would be N pages at once, instead of N pages one after the other ;-)

Edit: my suggestion is based on the fact GitHub returns the last page too, but I guess for your article what you are doing is already good enough as example.

Thread Thread
 
exacs profile image
Carlos Saito

Thanks for the suggestion! Your solution would work perfectly :)

I don't think that my solution is valid for all scenarios but might be good sometimes. For example:

  • We want to fetch pages until we met some condition (for example, search the last 10 commits that says "refactor" in the text) where we don't want to fetch all pages
  • The API doesn't return the "last" page in the header