Advanced Patterns for Symfony HttpClient: Streaming, Retry, and Resilience

If you’ve worked with Symfony, you’ve used symfony/http-client. You’ve run $client->request(‘GET’, …) and $response->toArray(). This is the bread and butter of API consumption, and it works beautifully for simple use cases.

But modern applications aren’t simple. They’re distributed, asynchronous, and expected to be resilient. What happens when you need to:

• Fetch 100 API endpoints without waiting 30 seconds?
• Consume a 500MB JSON file without hitting your memory limit?
• Handle an API that flakes out and retries automatically?
• Protect your app from a failing downstream service?
• Manage OAuth2 tokens that expire every 60 minutes?

This is where “trivial” usage ends. The HttpClient component is one of the most powerful and layered components in the Symfony ecosystem. It’s designed to solve these exact “non-trivial” problems.

In this article, I’ll move past the basics and into production-grade patterns. I’ll explore high-performance concurrency, memory-safe streaming with new Symfony features, advanced resilience with retries and circuit breakers, automated auth, and dynamic testing.

Let’s level up. 🚀

The Foundation: A Scoped Client

First, let’s set up our project. We’ll use a new Symfony application. The only core package you need to start is symfony/http-client.

composer require symfony/http-client

Throughout this article, we’ll be interacting with a fictional “product API.” The best practice for this is not to use the generic http_client service but to define a scoped client. This gives us a dedicated service instance for that API, pre-configured with its base_uri and default headers.

Let’s define it in config/packages/framework.yaml:

# config/packages/framework.yaml
framework:
    http_client:
        scoped_clients:
            # This creates a new service with the ID 'product_api.client'
            product_api.client:
                base_uri: 'https://api.my-store.com/'
                headers:
                    'Accept': 'application/json'
                    'User-Agent': 'MySymfonyApp/1.0'

Now, we can autowire this specific client in any service using its type-hint and variable name:

// src/Service/ProductService.php
namespace App\Service;

use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Component\DependencyInjection\Attribute\Autowire;

readonly class ProductService
{
    public function __construct(
        #[Autowire(service: 'product_api.client')]
        private HttpClientInterface $client
    ) {
    }

    // ...
}

The Concurrency Trap (And How to Escape It with stream())

Here is the most common performance pitfall I see.

You need to fetch data for multiple items. The junior developer writes a foreach loop.

// The "Slow Way"
public function fetchProductPrices(array $productIds): array
{
    $prices = [];
    foreach ($productIds as $id) {
        // Each request waits for the previous one to finish!
        $response = $this->client->request('GET', "products/{$id}/price");
        $prices[$id] = $response->toArray()['price'];
    }
    return $prices; // If each request takes 300ms, 10 IDs = 3 seconds.
}

This is a serial operation. Each request runs sequentially. Your total execution time is the sum of all request latencies. It’s slow, and it scales terribly.

Use HttpClientInterface::stream().

The stream() method allows you to run multiple requests concurrently. It fires off all requests in parallel (using non-blocking I/O via curl_multi or Amp) and yields responses as they become available.

// The "Fast Way"
public function fetchProductPricesConcurrent(array $productIds): array
{
    $responses = [];
    foreach ($productIds as $id) {
        // This just creates the request object; it doesn't send it.
        $responses[$id] = $this->client->request('GET', "products/{$id}/price");
    }

    $prices = [];
    // This is where the magic happens. All requests are sent in parallel.
    foreach ($this->client->stream($responses) as $response => $chunk) {
        try {
            if ($chunk->isFirst()) {
                // Headers are available, but we wait for the content
            }

            if ($chunk->isLast()) {
                // The full response is now available.
                // We find the original $id by searching the $responses array.
                $id = array_search($response, $responses, true);
                if ($id !== false) {
                    $prices[$id] = $response->toArray()['price'];
                }
            }
        } catch (\Exception $e) {
            // Handle exceptions for individual failed requests
            $id = array_search($response, $responses, true);
            $this->logger->error("Failed to fetch price for {$id}", ['exception' => $e]);
        }
    }

    return $prices; // Total time ≈ the single longest request, not the sum.
}

You can easily prove the difference with the symfony/stopwatch component.

composer require symfony/stopwatch

Then, in a simple console command:

// src/Command/TestConcurrencyCommand.php
use Symfony\Component\Console\Attribute\AsCommand;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Stopwatch\Stopwatch;

#[AsCommand(name: 'app:test-concurrency')]
class TestConcurrencyCommand extends Command
{
    public function __construct(
        private readonly ProductService $productService,
        private readonly Stopwatch $stopwatch
    ) {
        parent::__construct();
    }

    protected function execute(InputInterface $input, OutputInterface $output): int
    {
        $ids = range(1, 10); // 10 product IDs
        $stopwatch = new Stopwatch();

        $stopwatch->start('serial');
        $this->productService->fetchProductPrices($ids);
        $serialEvent = $stopwatch->stop('serial');
        $output->writeln('Serial: ' . $serialEvent->getDuration() . 'ms');

        $stopwatch->start('concurrent');
        $this->productService->fetchProductPricesConcurrent($ids);
        $concurrentEvent = $stopwatch->stop('concurrent');
        $output->writeln('Concurrent: ' . $concurrentEvent->getDuration() . 'ms');

        return Command::SUCCESS;
    }
}

Result: You will consistently see the concurrent method be an order of magnitude faster.

• Serial: ~3000ms
• Concurrent: ~310ms

Taming Large Payloads with symfony/json-streamer (New in 7.3!)

An API returns a large JSON array. GET /products/all returns a 500MB file with 2 million product objects. If you call $response->toArray(), PHP will try to parse 500MB of JSON into a massive array, instantly exhausting your memory limit.

Stream the response. Instead of reading the whole response, we read it chunk by chunk. Even better, with the new symfony/json-streamer (experimental, no BC guarantee) component in Symfony 7.3, we can parse this stream directly into DTOs.

Let’s install it:

composer require symfony/json-streamer

First, create a simple DTO. Note that json-streamer works best with simple, constructor-less classes with public properties.

// src/Dto/ProductDto.php
namespace App\Dto;

use Symfony\Component\JsonStreamer\Attribute\JsonStreamable;

#[JsonStreamable]
class ProductDto
{
    public string $sku;
    public string $name;
    public float $price;
}

Now, let’s create a service to consume a (simulated) large endpoint.

// src/Service/StreamingProductService.php
namespace App\Service;

use App\Dto\ProductDto;
use Symfony\Component\DependencyInjection\Attribute\Autowire;
use Symfony\Component\JsonStreamer\StreamReaderInterface;
use Symfony\Component\TypeInfo\Type;
use Symfony\Contracts\HttpClient\HttpClientInterface;

readonly class StreamingProductService
{
    public function __construct(
        #[Autowire(service: 'product_api.client')]
        private HttpClientInterface $client,
        private StreamReaderInterface $streamReader
    ) {
    }

    public function processAllProducts(): int
    {
        // 1. Make the request, but DON'T read the content yet.
        $response = $this->client->request('GET', 'products/all-stream');

        // 2. Define the expected type. We expect a list of ProductDto objects.
        $type = Type::list(Type::object(ProductDto::class));

        // 3. Use the StreamReader to read directly from the
        //    HttpClient Response object. This is memory-efficient.
        $products = $this->streamReader->read($response, $type);

        $count = 0;

        // 4. $products is a generator. We iterate over it.
        //    Each $product is a fully-formed ProductDto.
        foreach ($products as $product) {
            // $product is an instance of ProductDto
            // Do work here, like saving to a local DB.
            $count++;
        }

        return $count;
    }
}

To test this, we can create a “fake” API endpoint in a controller that streams a large JSON response.

// src/Controller/MockProductApiController.php
namespace App\Controller;

use Symfony\Component\HttpFoundation\StreamedResponse;
use Symfony\Component\Routing\Attribute\Route;

class MockProductApiController
{
    #[Route('/products/all-stream', name: 'mock_api_stream')]
    public function streamAllProducts(): StreamedResponse
    {
        $response = new StreamedResponse();
        $response->headers->set('Content-Type', 'application/json');

        $response->setCallback(function () {
            echo '[';
            // Simulate 500,000 product objects
            for ($i = 0; $i < 500_000; $i++) {
                echo json_encode([
                    'sku' => 'SKU-' . $i,
                    'name' => 'Product ' . $i,
                    'price' => mt_rand(10, 1000)
                ]);
                if ($i < 499_999) {
                    echo ',';
                }
                flush(); // Flush output buffer
            }
            echo ']';
        });

        return $response;
    }
}

If you run your StreamingProductService (e.g., from a command) and monitor memory_get_peak_usage(), you’ll find it stays incredibly low, no matter if you stream 500 or 5 million objects. If you had tried this with $response->toArray(), the script would have crashed.

Building a Bulletproof Client (Retries & a Manual Circuit Breaker)
Resilience is paramount. When a downstream API fails, your app shouldn’t fail with it. HttpClient provides RetryableHttpClient out of the box, but we can go further.

The First Line of Defense (RetryableHttpClient)
This is the easy part. RetryableHttpClient is a decorator that wraps your client and automatically retries requests that fail with specific status codes (like 503, 504) or TransportException.

We just need to update our service definition in config/services.yaml:

# config/services.yaml
services:
    _defaults:
        autowire: true
        autoconfigure: true

    # 1. Define the retry strategy
    App\HttpClient\ProductApiRetryStrategy:
        factory: [Symfony\Component\HttpClient\Retry\GenericRetryStrategy, 'decide']
        arguments:
            - [503, 504] # HTTP codes to retry
            - 1000        # Delay in ms (1s)
            - 2.0         # Multiplier (1s, 2s, 4s)
            - 60000       # Max delay (60s)
            - 0.5         # Jitter (randomness)

    # 2. Decorate our scoped client to make it retryable
    product_api.client.retryable:
        class: Symfony\Component\HttpClient\RetryableHttpClient
        decorates: product_api.client
        arguments:
            - '@.inner' # The decorated service (product_api.client)
            - '@App\HttpClient\ProductApiRetryStrategy'
            - 3 # Max retries

That’s it. Now, any service autowiring #[Autowire(service: ‘product_api.client’)] will actually get the retryable one. If the API returns a 503 Service Unavailable, our client will automatically wait 1s, retry, wait 2s, retry, wait 4s, retry, and only then fail.

Manual Circuit Breaker
Retries are great, but what if the API is hard down? Retrying 3 times for every single request will bog down our own app. We’ll be “hammering a dead service.”

This is the job of the Circuit Breaker pattern. It monitors failures, and if they pass a threshold, it “opens the circuit” — failing instantly for all subsequent requests for a set period, giving the downstream service time to recover.

Symfony does not have a built-in symfony/circuit-breaker **component or CircuitBreakerHttpClient decorator. Many developers assume it does. This provides a perfect opportunity to demonstrate the power of service decoration by building one ourselves using **symfony/cache.

composer require symfony/cache

First, we create our decorator. It must implement HttpClientInterface to be a valid decorator.

// src/HttpClient/CircuitBreakerClient.php
namespace App\HttpClient;

use Psr\Cache\CacheItemPoolInterface;
use Psr\Log\LoggerInterface;
use Symfony\Component\DependencyInjection\Attribute\AsDecorator;
use Symfony\Component\DependencyInjection\Attribute\Autowire;
use Symfony\Component\HttpClient\Exception\TransportException;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;
use Symfony\Contracts\HttpClient\ResponseStreamInterface;

// This attribute automatically configures the service decoration
#[AsDecorator(decorates: 'product_api.client.retryable', priority: 10)]
readonly class CircuitBreakerClient implements HttpClientInterface
{
    private const STATE_CLOSED = 'closed';
    private const STATE_OPEN = 'open';
    private const FAILURE_THRESHOLD = 5; // Open circuit after 5 failures
    private const OPEN_TTL = 60; // Stay open for 60 seconds

    public function __construct(
        // #[AutowireDecorated] is not needed because we specify the ID above
        #[Autowire(service: '.inner')]
        private HttpClientInterface $inner,
        #[Autowire(service: 'cache.app')]
        private CacheItemPoolInterface $cache,
        private LoggerInterface $logger
    ) {
    }

    public function request(string $method, string $url, array $options = []): ResponseInterface
    {
        if ($this->isOpen()) {
            $this->logger->warning('Circuit breaker is OPEN for product_api');
            throw new TransportException('Circuit breaker is open', 0);
        }

        try {
            $response = $this->inner->request($method, $url, $options);

            // This is a lazy check. We must get the status code to trigger potential exceptions.
            $response->getStatusCode(); 

            // Request was successful, reset failure count
            $this->resetFailures();

            return $response;
        } catch (\Exception $e) {
            // Request failed, record it
            $this->recordFailure();
            throw $e;
        }
    }

    private function isOpen(): bool
    {
        $state = $this->cache->getItem('product_api.circuit.state');
        return $state->isHit() && $state->get() === self::STATE_OPEN;
    }

    private function recordFailure(): void
    {
        $failuresItem = $this->cache->getItem('product_api.circuit.failures');
        $failures = $failuresItem->isHit() ? $failuresItem->get() : 0;
        $failures++;

        if ($failures >= self::FAILURE_THRESHOLD) {
            // Open the circuit!
            $stateItem = $this->cache->getItem('product_api.circuit.state');
            $stateItem->set(self::STATE_OPEN);
            $stateItem->expiresAfter(self::OPEN_TTL);
            $this->cache->save($stateItem);

            // Clear the failure count
            $this->cache->deleteItem('product_api.circuit.failures');
            $this->logger->critical('Circuit breaker OPENED for product_api');
        } else {
            // Just save the new failure count
            $failuresItem->set($failures);
            $this->cache->save($failuresItem);
        }
    }

    private function resetFailures(): void
    {
        $this->cache->deleteItem('product_api.circuit.failures');
    }

    // --- Must implement all other interface methods ---

    public function stream(ResponseInterface|iterable $responses, ?float $timeout = null): ResponseStreamInterface
    {
        // For brevity, we don't add circuit breaker logic to stream()
        // In a real app, you would.
        return $this->inner->stream($responses, $timeout);
    }

    public function withOptions(array $options): static
    {
        $clone = clone $this;
        $clone->inner = $this->inner->withOptions($options);
        return $clone;
    }
}

(Note: This is a simple implementation. A production-grade one would also include a HALF-OPEN state.)

Because we used the #[AsDecorator] attribute, we don’t even need to touch services.yaml! Our decoration chain is now:

ProductService -> CircuitBreakerClient -> RetryableHttpClient -> NativeHttpClient

If the API fails 5 times (after all retries), the CircuitBreakerClient will open the circuit and fail-fast for 60 seconds, protecting our app.

Automated OAuth2 (Managing Tokens with AccessTokenHttpClient)

You’re consuming an OAuth2-protected API. You have a token, but it expires in one hour. Your code is littered with this:

$token = $this->cache->get('api_token');
if ($token->isExpired()) {
    $token = $this->fetchNewToken();
    $this->cache->save($token);
}
$this->client->request('GET', '/data', [
    'auth_bearer' => $token->getValue()
]);

This is manual, repetitive, and error-prone.

Use AccessTokenHttpClient. This is another built-in decorator that takes a callable responsible for providing a valid token. It doesn’t know how you get the token, it just knows who to ask.

We’ll create a dedicated service to manage token fetching and caching.

// src/Service/OAuthTokenProvider.php
namespace App\Service;

use Psr\Cache\CacheItemPoolInterface;
use Symfony\Component\DependencyInjection\Attribute\Autowire;
use Symfony\Contracts\HttpClient\HttpClientInterface;

readonly class OAuthTokenProvider
{
    public function __construct(
        // We need a *different* client for auth, one that isn't
        // decorated with auth itself, or we'll get an infinite loop!
        #[Autowire(service: 'product_api.auth_client')]
        private HttpClientInterface $authClient,
        #[Autowire(service: 'cache.app')]
        private CacheItemPoolInterface $cache
    ) {
    }

    public function getToken(): string
    {
        // $cache->get() handles checking for existence and expiration
        return $this->cache->get('product_api.oauth_token', function ($item) {
            $this->logger->info('Fetching new OAuth2 token...');

            // Set TTL, e.g., 55 minutes for a 1-hour token
            $item->expiresAfter(3300); 

            $response = $this->authClient->request('POST', '/token', [
                'json' => [
                    'client_id' => '%env(CLIENT_ID)%',
                    'client_secret' => '%env(CLIENT_SECRET)%',
                    'grant_type' => 'client_credentials'
                ]
            ]);

            return $response->toArray()['access_token'];
        });
    }
}

Now, we wire it all up in config/services.yaml. This time, we can’t use attributes because the configuration is too complex.

# config/services.yaml
services:
    _defaults:
        autowire: true
        autoconfigure: true

    # ... other services ...

    # The token provider service
    App\Service\OAuthTokenProvider: ~

    # 1. The unauthenticated client used *only* for getting the token
    product_api.auth_client:
        parent: 'http_client.abstract'
        arguments:
            - base_uri: 'https://auth.my-store.com/' # Note: different host!

    # 2. Our main 'product_api.client'
    #    (This is the service ID from framework.yaml)
    product_api.client: ~

    # 3. The retryable decorator (from before)
    product_api.client.retryable:
        class: Symfony\Component\HttpClient\RetryableHttpClient
        decorates: product_api.client
        arguments:
            - '@.inner'
            - '@App\HttpClient\ProductApiRetryStrategy'
            - 3

    # 4. The NEW AccessTokenHttpClient
    #    It decorates the *retryable* client
    product_api.client.authed:
        class: Symfony\Component\HttpClient\AccessTokenHttpClient
        decorates: product_api.client.retryable # Decorates the decorator!
        arguments:
            $client: '@.inner'
            # This is the magic: we pass our service's method as a callable
            $getToken: '[@App\Service\OAuthTokenProvider, "getToken"]'
            # We must also define the auth strategy (e.g., Bearer header)
            $strategy: !php/object:Symfony\Component\HttpClient\Header\HeaderStrategy {
                type: 'Bearer'
            }

Now, any service that uses the product_api.client will actually get product_api.client.authed. When it makes its first request, AccessTokenHttpClient will call OAuthTokenProvider::getToken(). This will fetch and cache the token. All subsequent requests will use the cached token until the cache expires, at which point it will automatically fetch a new one.

Your application code becomes blissfully simple: $this->client->request(‘GET’, ‘/data’); It has no idea the complex token management happening under the hood.

Advanced Testing (Dynamic & Sequential Mocking)

You need to test a service that makes HTTP calls. The standard MockHttpClient is fine for a single response, but what if your service:

• Makes a GET request first.
• Then makes a POST request using data from the GET.

You need to assert the sequence of calls and validate the body of the POST.

Use a Generator or callable as the MockResponse factory.

// tests/Service/ProductServiceTest.php
namespace App\Tests\Service;

use App\Service\ProductService;
use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;

class ProductServiceTest extends KernelTestCase
{
    public function testProductUpdateFlow(): void
    {
        // 1. Define the response factory as a generator
        $responseFactory = function (): \Generator {

            // 1st Yield: The GET /products/1 response
            yield new MockResponse(json_encode([
                'id' => 1,
                'name' => 'Old Name',
                'price' => 10.0
            ]));

            // 2nd Yield: A callable to validate the 2nd request (the POST)
            yield function (string $method, string $url, array $options): MockResponse {
                // Assert the request *itself* is correct
                self::assertSame('POST', $method);
                self::assertSame('https://api.my-store.com/products/1/update', $url);

                self::assertJsonStringEqualsJsonString(
                    '{"name":"New Name","price":12.5}',
                    $options['body']
                );

                // Return the response for the POST
                return new MockResponse(
                    json_encode(['status' => 'success', 'id' => 1]),
                    ['http_code' => 200]
                );
            };
        };

        // 2. Create the MockHttpClient with our generator
        $mockClient = new MockHttpClient($responseFactory);

        // 3. Get the real service from the container and inject the mock
        //    (Or just instantiate it manually)
        self::bootKernel();
        $container = static::getContainer();

        // You could use service decoration in test env,
        // but for unit tests, manual instantiation is clearer.
        $productService = new ProductService($mockClient);

        // 4. Run the service method that makes both calls
        $result = $productService->updateProductName(1, 'New Name', 12.5);

        self::assertTrue($result);

        // 5. Assert that all expected mock responses were used
        self::assertSame(0, $mockClient->getRequestsCount());
    }
}

This test now provides 100% confidence. It confirms that your service not only makes the calls, but makes them in the right order, with the right data, and handles the responses correctly — all without ever touching a real network.

Conclusion

The Symfony HttpClient is far more than a simple wrapper around curl. It’s a sophisticated, extensible, and production-ready toolkit for building modern, distributed applications.

We’ve seen how to break out of the serial “foreach” trap with concurrent streaming, how to handle massive files with the new symfony/json-streamer, and how to build a truly resilient client with retries and a manual circuit breaker. We’ve automated complex OAuth2 token management and written powerful, dynamic unit tests to verify it all.

By moving beyond the regular request() call, you can leverage the full power of this component to build applications that are not just functional, but fast, memory-efficient, and bulletproof.

I’d love to hear your thoughts in comments!

Stay tuned — and let’s keep the conversation going.

Comments

[roshan sharma]

That’s incredible! You’ve gone from zero public projects to creating polished, open-source tools that are fast, elegant, and user-focused. LeedPDF and Glucose show serious skill, creativity, and thoughtfulness, what you’ve built is truly impressive!