re: Iterating Through Elasticsearch Documents Using Scroll and Ruby VIEW POST


We've had a lot of fun with scroll queries (by which I mean we've found every issue along the way!). Here's a couple of other things to do with older versions which I think are worth mentioning in case folks are looking at older code or just seeing older examples:

  • You may see the "search_type=scan" parameter. This was deprecated, and in v5.0 removed. You don't need it.
  • In older versions it used to be that the initial search request with the scroll param did not return the first batch of data. You only got that with the subsequent scroll call (when you pass the scroll_id). So older code would have the loop arranged a bit differently.

It's interesting what you wrote about the memory use, and holding the whole scroll "snapshot" in memory. Wonder if there's any tricks for making it less expensive.

code of conduct - report abuse