From timeout to 10 seconds: finding the real bottleneck

There was a screen that, on large accounts, would not open.

It was the listing of document items. On accounts with a lot of volume it hit the nginx timeout, 60 seconds, and for the customer it was simply unreachable. That module was already built. I rebuilt it on my own, frontend and backend.

The backend is PHP with Symfony and CQRS, and the frontend is React. I simplify the code in this post so the idea is clear.

Measure before touching

The temptation is to optimise the first thing that smells bad. Before that, I looked at where the time and the memory were going, and not only the total time: a high total time doesn't tell you whether the problem is the database, the ORM or the frontend. The bottleneck was the ORM's memory consumption.

When an ORM loads many records, it turns each one into objects, and those objects take up memory for as long as they stay alive. With tens of thousands of products, that memory shoots up and the listing never finishes in time.

First change: process in batches

Instead of loading all the items at once, I process them in batches and free memory between one and the next. This is the pattern in the Doctrine batch processing documentation, which calls clear() between batches to free memory.

foreach (array_chunk($ids, $batchSize) as $chunk) {
    $items = $repository->findByIds($chunk);

    yield from $this->present($items);

    $entityManager->clear(); // releases the objects from the previous batch
}

This step still hydrates full entities, but memory stops growing with the size of the account and depends mostly on the size of the batch. That size is a trade-off: bigger performs better, smaller uses less memory, and you choose it by measuring.

Second change: a view instead of the aggregate

Even when processing in batches, loading the full aggregate of each item is expensive. The aggregate exists to protect invariants when writing, and to render a list you don't need it. So I applied a view: the listing no longer goes through the aggregate and reads a read projection, a flat object with only the fields the screen renders.

In CQRS that is a query with its handler, and the handler loads nothing: it asks a read gateway for the data.

final readonly class ListDocumentItemsQuery
{
    public function __construct(
        public string $tenantId,
        public string $documentId,
    ) {}
}

final readonly class ListDocumentItemsQueryHandler
{
    public function __construct(private DocumentItemsReadGateway $items) {}

    /** @return iterable<DocumentItemView> */
    public function __invoke(ListDocumentItemsQuery $query): iterable
    {
        yield from $this->items->stream($query->tenantId, $query->documentId);
    }
}

The handler returns an iterable, not an array. That is the key to everything that comes next: with a generator, only one row is in memory at a time, the one being processed.

The read gateway is the one that talks to the database. It projects only the fields that are needed, sorts in a stable way and hands out rows with yield as the cursor brings them:

final class MongoDocumentItemsReadGateway implements DocumentItemsReadGateway
{
    /** @return \Generator<DocumentItemView> */
    public function stream(string $tenantId, string $documentId): \Generator
    {
        $cursor = $this->collection->find(
            ['tenantId' => $tenantId, 'documentId' => $documentId],
            [
                'projection' => ['sku' => 1, 'name' => 1, 'quantity' => 1, 'price' => 1],
                'sort' => ['_id' => 1], // stable order, with a unique tie-breaker
                'batchSize' => $this->batchSize,
            ],
        );

        foreach ($cursor as $row) {
            yield DocumentItemView::fromRow($row);
        }
    }
}

The heavy fields, the ones only needed on small accounts or when opening the detail, are loaded conditionally depending on the size of the account. Separating reads and writes this way doesn't require a second database or event sourcing: the separation is logical.

Third change: streaming from the controller

With an iterable in hand, the controller can start responding without waiting to have all the rows. In Symfony this is done with a StreamedResponse that writes one line of JSON per item (NDJSON) and flushes:

#[Route('/documents/{documentId}/items', methods: ['GET'])]
final class ListDocumentItemsController
{
    public function __construct(private QueryBus $queryBus) {}

    public function __invoke(string $documentId, TenantContext $tenant): StreamedResponse
    {
        $items = $this->queryBus->ask(
            new ListDocumentItemsQuery($tenant->id(), $documentId),
        );

        $response = new StreamedResponse(function () use ($items): void {
            foreach ($items as $i => $item) {
                echo json_encode($item, JSON_THROW_ON_ERROR) . "\n"; // one row per line

                if ($i % 100 === 0) {
                    flush(); // sent in batches, not row by row
                }

                if (connection_aborted()) {
                    return; // the client has gone, we stop reading
                }
            }
        });

        $response->headers->set('Content-Type', 'application/x-ndjson');
        $response->headers->set('X-Accel-Buffering', 'no'); // nginx must not buffer the response

        return $response;
    }
}

Two details matter. The first, X-Accel-Buffering: no: without it, nginx buffers the response and the client receives nothing until it finishes, and the streaming is pointless. The second, that nginx's read timeout is measured between two reads, not over the whole response: as long as rows keep arriving, it doesn't trigger. Symfony also ships StreamedJsonResponse, which accepts generators; here I use NDJSON because the frontend reads the stream line by line.

Fourth change: React renders with the first results

The other side is making the frontend not wait. To read the stream, an async generator that splits the lines as they arrive:

async function* streamItems(documentId: string, signal: AbortSignal) {
  // In the real frontend this sits behind a repository, with the whole DDD layer; here it goes straight to fetch to keep it simple
  const response = await fetch(`/api/documents/${documentId}/items`, { signal });
  if (!response.ok || !response.body) throw new Error(`HTTP ${response.status}`);

  const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += value;
    const lines = buffer.split('\n');
    buffer = lines.pop() ?? ''; // the last line may arrive half-finished

    for (const line of lines) {
      if (line) yield JSON.parse(line) as DocumentItem;
    }
  }
}

And to hook it up to useQuery, TanStack Query's streamedQuery: it accumulates the chunks that arrive into an array. The query is pending until the first one arrives, moves to success at that moment, and stays fetching until the stream closes.

import { queryOptions, useQuery, experimental_streamedQuery as streamedQuery } from '@tanstack/react-query';

const documentItemsQuery = (documentId: string) =>
  queryOptions({
    queryKey: ['document-items', documentId],
    queryFn: streamedQuery({
      streamFn: ({ signal }) => streamItems(documentId, signal),
    }),
  });

function DocumentItemsList({ documentId }: { documentId: string }) {
  const { data = [], isPending, isFetching } = useQuery(documentItemsQuery(documentId));

  if (isPending) return <ListSkeleton />;

  return (
    <>
      <ItemsTable rows={data} />
      {isFetching && <p>Loading more…</p>}
    </>
  );
}

streamedQuery is still marked as experimental in TanStack Query, so its API may change.

For the person using the screen, the difference is huge: they see the first item almost immediately, instead of having to wait 40 seconds to see it.

What to watch out for with streaming

While rows are arriving the total isn't known yet, so the screen shouldn't show an "of N" or an invented percentage. The order has to be stable, with a unique tie-breaker, or a row can appear twice or be skipped. And it's better to send the rows in batches, not one by one, because with tens of thousands of rows rendering each one separately eats the browser.

Summary of the changes

Change	What it removes
Batches	Memory growing with the size of the account
A view instead of the aggregate	Hydrating the full aggregate of every item
Streaming from the controller	Waiting for all the rows before responding
React with streamedQuery	Waiting for everything to arrive before rendering

Result

From timeout to about 10 seconds, with the screen filling up from the first results. With that you can get into accounts with more than 100,000 products.

Ten seconds is not fast, but before it didn't open.

The same thing, at another scale

In the webhook pipeline, which I cover in Why your webhooks arrive duplicated, I follow the same rule. I use Datadog's APM and traces to see which parts of the flow are really slow, and I aim the optimisation there, not where it seems. The traces are sampled and are useful for performance; it's also worth measuring how long each message waits in the queue, because sometimes the slow part is the wait and not the code.

Before optimising, measure.