Paginate search results properly with Whoosh (offset, total, and "page 3 of 12")

#python #webdev #tutorial #programming

Part of an ongoing series on Whoosh, the pure-Python full-text search library I maintain.

Almost every search UI needs pages: ten hits, a "Next" button, and a little "Showing 21–30 of 214" label. It's easy to hand-roll this with results[20:30], and it's easy to get it subtly wrong — off-by-one offsets, a "Next" button that shows an empty page, or a total count that's actually just the size of the current slice.

Whoosh ships a purpose-built helper for exactly this: Searcher.search_page. Here's how to use it and how it behaves at the edges.

The 10-line version

from whoosh import index

ix = index.open_dir("indexdir")
with ix.searcher() as s:
    from whoosh.qparser import QueryParser
    q = QueryParser("body", ix.schema).parse("python search")

    page = s.search_page(q, pagenum=1, pagelen=10)

    for hit in page:
        print(hit["title"])

    print(f"Page {page.pagenum} of {page.pagecount} — {page.total} total hits")

search_page returns a ResultsPage. You iterate it like a normal result set, but it carries the paging metadata you need for the UI.

The attributes you actually render

page.total — total number of matching documents (not just this page). This is your "of 214".
page.pagenum — the current page number (1-based).
page.pagecount — how many pages exist for this pagelen.
page.pagelen — hits on this page. On the last page this can be less than the pagelen you asked for.
page.offset — zero-based index of the first hit on this page. Your "Showing N–M" label is offset+1 to offset+pagelen.
page.is_last_page() — true when you should hide the "Next" button.

A correct results header, then, is just:

start = page.offset + 1
end = page.offset + page.pagelen
print(f"Showing {start}–{end} of {page.total}")

Two edge cases it handles for you

Asking for a page past the end. If a user tampers with ?page=999, you don't get a crash or an empty list you have to special-case. Whoosh clamps the page number to the last real page:

page = s.search_page(q, pagenum=999, pagelen=10)
# page.pagenum == page.pagecount, and you get the last page's hits

A pagenum of 0 or negative raises ValueError — so treat page numbers as 1-based and validate at the edge of your app. A common pattern:

pagenum = max(1, int(request.args.get("page", 1)))

Why not just slice the Results?

You can slice a Results object — results[20:30] works. But search_page does one important thing for you: it only computes enough of the result set to serve that page. Internally it searches with limit=pagenum * pagelen, so page 1 is cheap. (Deep pagination — page 500 — costs the same as fetching the first 5,000 hits, which is inherent to how ranked search works, not a Whoosh quirk. If you need infinite scroll over huge result sets, prefer "search after the last score/id" style cursors over deep page numbers.)

A Flask endpoint, end to end

from flask import Flask, request, render_template_string

app = Flask(__name__)

TEMPLATE = """
<form><input name="q" value="{{ q }}"><button>Search</button></form>
{% if page %}
  <p>Showing {{ page.offset + 1 }}–{{ page.offset + page.pagelen }}
     of {{ page.total }}</p>
  <ul>{% for hit in page %}<li>{{ hit["title"] }}</li>{% endfor %}</ul>
  {% if page.pagenum > 1 %}<a href="?q={{ q }}&page={{ page.pagenum - 1 }}">Prev</a>{% endif %}
  {% if not page.is_last_page() %}<a href="?q={{ q }}&page={{ page.pagenum + 1 }}">Next</a>{% endif %}
{% endif %}
"""

@app.route("/search")
def search():
    q = request.args.get("q", "").strip()
    pagenum = max(1, int(request.args.get("page", 1)))
    page = None
    if q:
        with ix.searcher() as s:
            query = QueryParser("body", ix.schema).parse(q)
            page = s.search_page(query, pagenum, pagelen=10)
            # render inside the `with` block: hits are tied to the searcher
            return render_template_string(TEMPLATE, q=q, page=page)
    return render_template_string(TEMPLATE, q=q, page=page)

The one gotcha worth repeating: render while the searcher is still open. Hits are lazy views into the searcher, so materialize (or render) inside the with block rather than returning the page object and iterating it later.

Takeaways

Use search_page(q, pagenum, pagelen) instead of manual slicing — you get total, pagecount, offset, and is_last_page() for free.
page.pagelen shrinks on the last page; use it (not the requested pagelen) for your "N–M" label.
Out-of-range pages clamp; page 0/negative raises — validate to 1-based at the edge.
Iterate/render before the searcher closes.

Whoosh is pure Python, pip install whoosh3, no server to run. If you're reviving a project that used the old abandoned Whoosh, the maintained fork lives at github.com/priya-sundaram-dev/whoosh — issues and PRs welcome.

DEV Community