What Happens When a Backend Service Makes 10,000 Concurrent Network Requests?

ABHIRAMI RAJEEV — Wed, 12 Aug 2026 15:14:50 +0000

When we talk about Go, one of the first things we hear is:

"Go can handle thousands of concurrent operations."

That's true—but it can also create a dangerous misconception.

If a backend service needs to make 10,000 network requests concurrently, is creating 10,000 goroutines enough?

Not really.

The interesting part starts when we look underneath the goroutine.

A network request involves much more than application-level concurrency. We also have sockets, file descriptors, TCP connections, ephemeral ports, connection pools, DNS, timeouts, operating-system limits, and the capacity of the downstream service.

So what actually happens when a Go service tries to make 10,000 network requests at the same time?

1. Start with the obvious approach

Imagine a service that needs to call another API 10,000 times.

A simple Go implementation might look like this:

var wg sync.WaitGroup

for i := 0; i < 10000; i++ {
    wg.Add(1)

    go func(i int) {
        defer wg.Done()

        resp, err := http.Get("https://example.com/api")
        if err != nil {
            return
        }
        defer resp.Body.Close()

        // Process response
    }(i)
}

wg.Wait()

At first glance, this looks completely reasonable.

Go can create thousands of goroutines, and goroutines are lightweight compared with operating-system threads.

But there is an important distinction:

10,000 goroutines do not automatically mean 10,000 network connections.

And understanding that distinction is where things get interesting.

2. Goroutines are not network connections

A goroutine is a unit of execution managed by the Go runtime.

A network connection is an operating-system resource represented by a socket.

They are related, but they are not the same thing.

You could have:

10,000 goroutines
        |
        +---- 100 TCP connections
        |
        +---- 500 TCP connections
        |
        +---- 10,000 TCP connections

The actual number depends on how the application uses the HTTP client and how connections are reused.

For example, HTTP keep-alive allows multiple requests to reuse an existing TCP connection.

HTTP/2 goes even further: multiple concurrent requests can be multiplexed over the same TCP connection.

So the first question should not be:

"How many goroutines do I have?"

It should be:

"How many network connections and other resources does this workload actually require?"

3. What happens underneath an HTTP request?

A simplified request looks like this:

Application
     |
     v
HTTP Client
     |
     v
DNS Resolution
     |
     v
Socket
     |
     v
TCP Connection
     |
     v
TLS (for HTTPS)
     |
     v
HTTP Request
     |
     v
Downstream Service

If a new TCP connection is required, the operating system needs to allocate resources for it.

That means our 10,000 concurrent requests can start interacting with limits outside our Go code.

4. File descriptors become relevant

On Unix-like systems, sockets are represented by file descriptors.

A process has a limit on how many file descriptors it can have open.

You can inspect the limit on many Linux systems with:

ulimit -n

Suppose your process is allowed only a few thousand open file descriptors.

If your application tries to establish thousands of simultaneous connections, you can hit that limit.

At that point, the problem isn't:

"Go cannot create enough goroutines."

The problem is:

"the process cannot obtain enough OS resources."

This is an important distinction when debugging high-concurrency services.

5. What about ephemeral ports?

This is another interesting limit.

When your machine initiates an outbound TCP connection, it needs a source port.

Conceptually, a connection looks like:

Source IP : Source Port
        |
        | TCP
        |
Destination IP : Destination Port

The source port is generally selected from the system's ephemeral-port range.

So if a service creates a very large number of outbound connections, available ephemeral ports can become a constraint.

But there is an important nuance:

You cannot simply say "10,000 requests = 10,000 ports."

Connections can be reused, and the relevant TCP connection identity includes the source/destination addresses and ports.

HTTP connection reuse therefore becomes extremely important.

6. Connection pooling changes the picture

This is one of the reasons HTTP clients are more sophisticated than they initially appear.

Instead of creating a brand-new TCP connection for every request:

Request 1 → TCP connection 1
Request 2 → TCP connection 2
Request 3 → TCP connection 3
...

a client can reuse connections:

                 +--> Request 1
                 |
TCP Connection --+--> Request 2
                 |
                 +--> Request 3

This avoids repeatedly paying the cost of:

TCP connection establishment
TLS handshakes
socket creation
port allocation
connection teardown

Go's net/http client uses a Transport to manage connection reuse and pooling.

That means how you configure and reuse your HTTP client matters.

For example, repeatedly creating clients or transports inside a hot path can prevent you from getting the connection reuse you actually want.

7. HTTP/2 changes the model again

With HTTP/1.1, multiple requests can reuse connections, but concurrency across requests is still constrained by the connection model and client/server behavior.

HTTP/2 introduces multiplexing.

Conceptually:

                 Request 1
                    |
                 Request 2
                    |
                 Request 3
                    |
                    v
              One TCP Connection

Multiple HTTP/2 streams can share the same connection.

So you might have:

10,000 concurrent requests

        ↓

far fewer TCP connections

        ↓

many HTTP/2 streams

This is one reason the phrase "10,000 concurrent requests" is not enough information to understand the actual network load.

8. Then comes the downstream service

Even if our own service can technically create 10,000 concurrent requests, that doesn't mean we should.

Imagine:

                Backend Service
                 /    |    \
                /     |     \
             10,000 concurrent requests
                       |
                       v
                Downstream API

If the downstream service can comfortably process only 1,000 concurrent requests, sending 10,000 requests at once may simply overwhelm it.

We can end up with:

More concurrency
       ↓
More load
       ↓
Higher latency
       ↓
Timeouts
       ↓
Retries
       ↓
Even more load

This is where a local concurrency decision becomes a distributed-systems problem.

9. Unlimited concurrency is not always better

A common approach is:

for _, item := range items {
    go process(item)
}

But if items contains 1 million elements, we've just created a potentially huge amount of concurrent work.

A better approach is to introduce a limit.

For example, a semaphore:

sem := make(chan struct{}, 100)

for _, item := range items {
    sem <- struct{}{}

    go func(item Item) {
        defer func() {
            <-sem
        }()

        process(item)
    }(item)
}

Now the application can have at most approximately 100 operations in the controlled section at once.

The exact number should not be chosen randomly.

It depends on things such as:

downstream capacity
request latency
CPU usage
memory
connection limits
database capacity
rate limits
expected traffic

10. Backpressure becomes important

Suppose requests arrive faster than the downstream system can process them.

Without backpressure:

Incoming work
     ↓
More goroutines
     ↓
More requests
     ↓
More memory
     ↓
More connections
     ↓
More timeouts

With controlled concurrency:

Incoming work
     ↓
Bounded queue
     ↓
Worker pool
     ↓
Controlled concurrency
     ↓
Downstream service

Instead of allowing the system to consume unlimited resources, we deliberately control how much work is in flight.

This is one of the most important principles in building reliable distributed systems:

A system should control the amount of work it allows into an overloaded component.

11. Timeouts are just as important as concurrency

Consider a request that normally takes 100 ms.

Now imagine the downstream service becomes unhealthy and requests start taking 30 seconds.

If we have 10,000 goroutines waiting:

10,000 goroutines
       |
       v
Waiting on slow network calls
       |
       v
Resources remain occupied

This can quickly turn into resource exhaustion.

That's why network operations should have explicit deadlines or timeouts.

For example:

client := &http.Client{
    Timeout: 5 * time.Second,
}

Timeouts prevent a slow dependency from holding resources indefinitely.

But timeouts also need careful design.

A timeout that is too short creates unnecessary failures.

A timeout that is too long allows resources to remain occupied for too long.

12. What happens when requests fail?

Now imagine 10,000 requests are made and the downstream service starts returning errors.

The natural reaction might be:

Request failed
      ↓
Retry

But if all 10,000 requests retry immediately:

10,000 requests
      ↓
failure
      ↓
10,000 retries
      ↓
more load
      ↓
more failures

This is how a small dependency failure can become a much larger distributed-system failure.

Retries should therefore generally be combined with mechanisms such as:

exponential backoff
jitter
retry limits
deadlines
idempotency
circuit breakers
rate limiting

Concurrency, networking, and reliability are deeply connected.

13. So how many concurrent requests should we allow?

There isn't a universal number.

The correct concurrency limit depends on the system.

For example:

                   Concurrency Limit
                         |
        +----------------+----------------+
        |                |                |
   Downstream        Network          Application
    capacity          limits           resources
        |                |                |
    Rate limits     Connections        CPU/Memory

You need to measure the system rather than simply choosing a large number.

A good starting point is to load-test the service and observe:

latency
throughput
error rate
CPU
memory
connection count
file descriptors
downstream saturation

Then increase concurrency gradually until you find the point where additional concurrency stops improving throughput or begins degrading reliability.

14. The bigger lesson

When I started thinking about "10,000 concurrent requests", I initially thought the interesting question was:

Can Go handle 10,000 goroutines?

But that's actually the easy part.

The more interesting questions are:

How many connections are required?

Can the OS provide the required resources?

Are connections being reused?

How many file descriptors are being consumed?

What happens to ephemeral ports?

Can the downstream service handle the traffic?

What happens when latency increases?

What happens when requests start timing out?

What happens when retries begin?

Where should backpressure be applied?

This is why backend concurrency is not simply about creating more goroutines.

Concurrency is a system-design decision.

The Go runtime is only one part of the system.

The operating system, network stack, HTTP client, downstream services, databases, and failure-handling mechanisms all participate in determining how much concurrency the system can safely handle.

And that's the part I find most interesting about backend engineering: a seemingly simple line of code can eventually lead all the way down to sockets, TCP, operating-system limits, and distributed-system failure modes.

DEV Community: ABHIRAMI RAJEEV