How I Built an API Latency Tester in 80 Lines of Python (And What It Surprised Me About)

Last week I needed to compare latency across three different API providers for the same endpoint. curl + time worked for one-off tests, but I wanted concurrency, percentiles, and side-by-side comparisons.

So I wrote a tiny CLI tool. 80 lines. Here's how it works — and what the numbers actually taught me.

---

The Problem

I had three endpoints returning the same JSON structure:

• Provider A: US East, 200ms baseline
• Provider B: Europe, claimed "sub-100ms"
• Provider C: Asia Pacific, no SLA

I needed to know: which one is actually fastest from my region, under real concurrency?

curl -w "@curl-format.txt" gives you total time. ab (ApacheBench) gives you concurrency. Neither gives you percentile latency with concurrent connections in a single readable output.

---

The Tool

Here's the full thing. I called it apiping:

#!/usr/bin/env python3
"""apiping — concurrent API latency tester with percentile output"""
import asyncio, aiohttp, time, statistics, argparse, json
from dataclasses import dataclass

@dataclass
class Result:
    url: str
    latencies: list  # in milliseconds
    errors: int = 0

    @property
    def avg(self): return statistics.mean(self.latencies) if self.latencies else 0
    @property
    def p50(self): return self._percentile(50)
    @property
    def p95(self): return self._percentile(95)
    @property
    def p99(self): return self._percentile(99)
    @property
    def success_rate(self):
        total = len(self.latencies) + self.errors
        return (len(self.latencies) / total * 100) if total else 0

    def _percentile(self, p):
        if not self.latencies:
            return 0
        idx = int(len(self.latencies) * p / 100)
        return sorted(self.latencies)[min(idx, len(self.latencies)-1)]

async def ping(session, url, method="GET", headers=None, body=None):
    start = time.monotonic()
    try:
        async with session.request(method, url, headers=headers, json=body,
                                    timeout=aiohttp.ClientTimeout(total=10)) as resp:
            await resp.read()
            return (time.monotonic() - start) * 1000  # ms
    except Exception:
        return None

async def run_test(url, concurrency, count):
    async with aiohttp.ClientSession() as session:
        tasks = [ping(session, url) for _ in range(count)]
        results = []
        # Process in chunks to limit concurrency
        for i in range(0, len(tasks), concurrency):
            chunk = tasks[i:i+concurrency]
            results.extend(await asyncio.gather(*chunk))
    latencies = [r for r in results if r is not None]
    return Result(url=url, latencies=latencies,
                  errors=len(results) - len(latencies))

async def main():
    parser = argparse.ArgumentParser(description="Concurrent API latency tester")
    parser.add_argument("urls", nargs="+", help="URLs to test")
    parser.add_argument("-c", "--concurrency", type=int, default=10)
    parser.add_argument("-n", "--count", type=int, default=100)
    parser.add_argument("-H", "--header", action="append",
                        help="Headers in 'Key: Value' format")
    args = parser.parse_args()

    headers = {}
    if args.header:
        for h in args.header:
            k, v = h.split(":", 1)
            headers[k.strip()] = v.strip()

    print(f"Testing {len(args.urls)} endpoint(s), "
          f"{args.count} requests each, concurrency={args.concurrency}\n")

    results = []
    for url in args.urls:
        r = await run_test(url, args.concurrency, args.count)
        results.append(r)
        print(f"  {url}")
        print(f"    avg={r.avg:.1f}ms  p50={r.p50:.1f}ms  "
              f"p95={r.p95:.1f}ms  p99={r.p99:.1f}ms  "
              f"success={r.success_rate:.1f}%")

    # Comparison summary
    if len(results) > 1:
        print("\n--- Comparison ---")
        best = min(results, key=lambda r: r.p95)
        print(f"  Lowest p95: {best.url} ({best.p95:.1f}ms)")

if __name__ == "__main__":
    asyncio.run(main())

Usage:

pip install aiohttp
python apiping.py https://api-a.com/health https://api-b.com/health -c 20 -n 200

That's it. No dependencies beyond aiohttp (and statistics from stdlib).

---

What Surprised Me

Here's what the numbers actually showed — and two things I did NOT expect:

1. Average is a liar

Provider B had 85ms average. Great, right?

p95 was 1,240ms. Fourteen times the average. One in twenty requests took over a second.

The pattern was clear: every ~30 seconds, a cold-start spike hit. Their "sub-100ms" claim was technically true for the median — which is exactly why they quoted it.

Lesson: Always ask for p95 or p99. If a provider only gives you "average latency," assume the real number is 5-10x worse.

2. Concurrency changes everything

Provider A (US East) at concurrency=1: 180ms avg, consistent.
At concurrency=50: 2,400ms avg, 8% error rate.

They had a connection pool capped at 32. Once I exceeded it, every extra request queued up behind that limit. The latency curve was flat until 32, then went vertical.

Lesson: Test at your actual production concurrency. A single-request benchmark tells you nothing about what happens under load.

Provider C (Asia Pacific) won the p95 comparison because their infrastructure was newer and over-provisioned — higher baseline latency (220ms) but almost zero variance. For my use case (batch processing), consistent 220ms beat "sometimes 80ms, sometimes 1.2s."

---

What I'd Add Next

• Warm-up phase: Discard first N requests (cold TCP/TLS handshake noise)
• Streaming support: Test SSE/WebSocket endpoints
• Latency histogram: ASCII chart in terminal output
• JSON output mode: For piping into jq or Grafana

---

The Real Takeaway

You don't need k6, locust, or artillery to get useful latency data. For API selection decisions (not full load testing), 80 lines of Python plus aiohttp gives you percentiles, concurrency, and error rates — enough to catch providers that cherry-pick their numbers.

The full code is up on GitHub if you want it. Drop a comment if you've run into similar surprises benchmarking APIs — I'm collecting stories.