Pagination & Filtering: Managing Large Datasets Efficiently with FastAPI

Why Pagination Matters

Imagine an API returning 10,000 items at once.
That's 10,000 database reads, massive memory usage,
and a slow response every single time.

Pagination solves this by slicing data into
manageable chunks.

The Endpoint

@app.get("/items")
def get_items(
    page: int = Query(default=1, ge=1),
    limit: int = Query(default=5, ge=1, le=20),
    category: Optional[str] = None,
    min_price: Optional[float] = None,
    max_price: Optional[float] = None
):

How Pagination Works

offset = (page - 1) * limit
items = query.offset(offset).limit(limit).all()

• page=1, limit=5 → items 1-5
• page=2, limit=5 → items 6-10
• page=3, limit=5 → items 11-15

The offset tells the database where to start,
limit tells it how many to return.

Filtering

The same endpoint handles filtering:

?category=electronics → only electronics
?min_price=100&max_price=500 → price range filter
?category=electronics&page=2&limit=5 → combined!

The Metadata Response

{
  "metadata": {
    "total": 12,
    "page": 2,
    "limit": 5,
    "total_pages": 3
  },
  "data": [...]
}

Always return metadata with paginated responses,
the client needs to know how many pages exist.

Postman Tests

Page 1 - First 5 items

Page 2 - Next 5 items

Category filter - electronics only

Price range filter

Lessons Learned

Pagination and filtering are not advanced features :
they're basic requirements for any API that handles
real data. Build them in from the start, not as an
afterthought.

Day 13 done. 17 more to go. 🔥

GDGoCBowen30dayChallenge