DEV Community

pathvector-dev
pathvector-dev

Posted on • Originally published at blog.pathvector.dev

Traffic Shaping with a Token Bucket: Cap a 56 Gbit/s Link at 10 Mbit/s Using tc

#qos #network #containerlab #linux

Originally published at https://blog.pathvector.dev/protocol-lab-qos-28/ — part of the free Protocol Lab series.

This post is part of Protocol Lab, a free, hands-on series for learning networking protocols by building and breaking them in a container lab. All the lab material — topologies, configs, and scripts — lives in the repo: github.com/pathvector-studio/protocol-lab.

In Lab 08, tc netem added delay and loss to a link. This lab uses tc for the other classic job: traffic shaping — deliberately capping how fast a link may send. You attach a token-bucket queueing discipline (tbf) to the sender's egress and measure the effect with iperf3.

Reading guide: rfc-notes/qos-traffic-shaping.md

Prerequisite: TCP Lab 08: Loss, Retransmission, and the Window

Expected time: 40–55 minutes.

The Goal

  • First, iperf3 runs at the veth's native speed (tens of Gbit/s inside a container),
  • then a tc tbf shaper caps the client's egress at 10 Mbit/s,
  • iperf3 now measures ~10 Mbit/s — the token bucket meters the traffic to the configured rate.

By the end, you should be able to explain this table:

State iperf3 throughput
no shaping tens of Gbit/s (native veth)
tbf rate 10mbit ~10 Mbit/s (metered to the rate)

What You Will Learn

  • What traffic shaping is, and how it differs from netem's delay/loss (Lab 08).
  • What a queueing discipline (qdisc) is in Linux tc.
  • How a token bucket (tbf) meters traffic: tokens accumulate at the rate, and a packet needs tokens to leave.
  • Why burst and latency matter (and why a too-small burst throttles almost to zero).
  • Where shaping is used (rate plans, fair-sharing, protecting a slow uplink).

This lab does not cover:

  • Classful shaping (HTB) with multiple classes and priorities.
  • Marking/policing (DiffServ DSCP) or ingress shaping.
  • Fair queueing (fq, fq_codel) and bufferbloat in depth.

Where to Read in the RFCs

Traffic shaping is a Linux-implementation (tc) topic, so there is no single defining RFC. These references cover the related concepts:

Reference What to focus on
tc-tbf(8) man page The token bucket filter's parameters (rate, burst, latency)
RFC 2475 The DiffServ architecture (where shaping/policing fit in)
RFC 2697 Single Rate Three Color Marker (the general form of a token bucket)
RFC 5737 Confirming the addresses used here are documentation-only

The Big Picture

Two nodes: a client and a server. The server runs iperf3 -s.

client (10.0.0.1) ------ eth1/eth1 ------ server (10.0.0.2, iperf3 -s)
  tc tbf on egress                        measure throughput with iperf3
Enter fullscreen mode Exit fullscreen mode

Attach a token bucket to the client's egress (eth1), and everything leaving through it is metered down to the configured rate.

flowchart LR
  app["client app<br/>(iperf3)"] --> q["tbf qdisc<br/>rate 10mbit"]
  q --> link["eth1 (native speed)"]
  link --> S["server"]
  note["tokens fill at 10 Mbit/s.<br/>a packet leaves only when<br/>enough tokens exist."]
Enter fullscreen mode Exit fullscreen mode

Note: 10.0.0.0/24 is a local, closed range, so nothing in this lab touches the real internet.

What You Need

Recommended environment:

  • Linux / WSL2 / a Linux VM
  • Docker
  • containerlab

Images used:

  • nicolaka/netshoot:latest — bundles both tc and iperf3.

No additional images are required.

Running the Lab

The quick path, which deploys, verifies, and tears down for you:

./scripts/labctl.sh run qos-28
Enter fullscreen mode Exit fullscreen mode

Or step through it manually:

1. Move into the working directory 

cd protocol-lab/examples/qos-28
Enter fullscreen mode Exit fullscreen mode

2. Deploy and start the iperf3 server 

sudo containerlab deploy -t qos-28.clab.yml
docker exec -d clab-qos-28-server iperf3 -s
Enter fullscreen mode Exit fullscreen mode

3. Measure the unshaped baseline 

docker exec clab-qos-28-client iperf3 -c 10.0.0.2 -t 4
Enter fullscreen mode Exit fullscreen mode

This is a veth pair, so expect tens of Gbit/s (the exact figure depends on your machine).

4. Cap it at 10 Mbit/s with a token bucket 

docker exec clab-qos-28-client tc qdisc add dev eth1 root tbf rate 10mbit burst 32kb latency 100ms
docker exec clab-qos-28-client tc -s qdisc show dev eth1
docker exec clab-qos-28-client iperf3 -c 10.0.0.2 -t 4
Enter fullscreen mode Exit fullscreen mode

Throughput drops to roughly 10 Mbit/s.

Warning: burst is in bytes: 32kb = 32 kilobytes. If you accidentally specify it in bits — 32kbit, which is only 4 KB — the shaper throttles the link almost to zero.

5. Remove the shaper and restore full speed 

docker exec clab-qos-28-client tc qdisc del dev eth1 root
docker exec clab-qos-28-client iperf3 -c 10.0.0.2 -t 4   # fast again
Enter fullscreen mode Exit fullscreen mode

Expected Output

  • Unshaped iperf3: tens of Gbit/s (fast, environment-dependent).
  • With tbf rate 10mbit: about 10 Mbit/s.
  • After deleting the qdisc: fast again.

Why It Works

QoS (Quality of Service) work with tc comes in two flavors: "add delay and loss" (Lab 08's netem) and "cap the speed" (shaping). This lab is the latter. Shaping means putting an artificial ceiling on how fast a link may send.

  • qdisc (queueing discipline). Linux hangs a qdisc on each interface's egress; it decides how packets are sent out — ordering, timing, and drops. tc is the tool that configures it. The default is a simple FIFO-style qdisc.
  • Token bucket (tbf). "Tokens" accumulate in a bucket at the configured rate (here 10 Mbit/s). Sending a packet costs tokens. If tokens are available, the packet leaves immediately; if not, it waits. Over the long run traffic is held to the rate; momentarily, it may burst up to however many tokens have been saved. That is how the average speed is metered to the rate.
    • burst is the bucket size — how many tokens it can hold, i.e. how much can leave in one go. Too small, and only a trickle can be sent at each instant, dragging effective throughput far below the rate (in this lab, 32kbit = 4 KB dropped it to about 0.5 Mbit/s). So you need a burst, in bytes, sized to match the rate.
    • latency is the maximum time a packet may wait in the queue. Exceed it and the packet is dropped — effectively a cap on queue depth.
  • Shaping vs. policing. Shaping delays traffic to smooth it (queue packets and pace them to the rate). Policing drops or marks anything over the limit immediately. tbf is shaping.
  • Where it's used. Enforcing contracted bandwidth (ISP rate plans), fair-sharing so one user can't monopolize a line, and pacing a sender to protect a slow upstream link (a bufferbloat countermeasure).

The key insight: control the egress with a qdisc, and let the token bucket meter the average speed to the configured rate. Alongside netem (delay/loss), this is tc's other major job.

Common Pitfalls

  • Confusing shaping with netem. netem adds delay and loss; shaping caps speed. Both live in tc, but they serve different purposes.
  • The burst unit. tc's burst is in bytes: 32kb = 32 KB, while 32kbit = 4 KB (far too small). Size it to match the rate.
  • It only works on egress. tbf shapes transmission (egress). Shaping ingress requires a different mechanism (e.g. ifb).
  • veth is very fast. Container veth pairs push tens of Gbit/s, so the unshaped baseline varies a lot by environment.
  • Shaping vs. policing. Queue-and-smooth (shaping) or drop-the-excess (policing)? tbf is the former.
  • Measurement noise. iperf3 results wobble a bit. Judge success by whether the result lands near the configured rate.

Cleanup 

sudo containerlab destroy -t qos-28.clab.yml --cleanup
Enter fullscreen mode Exit fullscreen mode

If you used labctl.sh run qos-28, the script runs destroy for you at the end.

Check Your Understanding

  1. What is traffic shaping? How does it differ from Lab 08's netem (delay/loss)?
  2. What is a qdisc in Linux? What does tc configure?
  3. How does a token bucket (tbf) meter the average speed to the rate?
  4. What does burst represent? What happens when it's too small? What unit is it in?
  5. What's the difference between shaping and policing? Which one is tbf?
  6. Name two situations where shaping is used.

References

Verified Run Log (2026-07-07)

This lab has been confirmed reproducible on real hardware.

Environment:

  • Ubuntu 26.04 LTS (kernel 7.0.0-27-generic, x86_64)
  • Docker 29.1.3
  • containerlab 0.77.0
  • client / server: nicolaka/netshoot:latest (tc, iperf3 bundled)

Running PATH="/tmp/pl-shim:$PATH" ./scripts/labctl.sh run qos-28 performed deploy → verify → destroy, and verification.json returned "status": "verified".

Native speed → 10 Mbit/s under the token bucket 

[protocol-lab][qos-28] baseline: 56551 Mbit/s
[protocol-lab][qos-28] + tc qdisc add dev eth1 root tbf rate 10mbit burst 32kb latency 100ms
[protocol-lab][qos-28] shaped: 9 Mbit/s
Enter fullscreen mode Exit fullscreen mode

Before shaping, the veth ran at its native 56551 Mbit/s (about 56 Gbit/s). After attaching tbf rate 10mbit to the client's egress, iperf3 throughput fell to 9 Mbit/s — right around the configured rate. The token bucket is metering the average speed to the rate.

(The burst unit really is the crux: with 32kb = 32 KB the link settles at about 10 Mbit/s, but a too-small 32kbit = 4 KB over-throttles it to roughly 0.5 Mbit/s.)

Cleanup 

containerlab destroy -t qos-28.clab.yml --cleanup
Enter fullscreen mode Exit fullscreen mode

That's traffic shaping: hang a token-bucket qdisc on the egress, and the link's average speed obediently settles at whatever rate you configured — no cooperation from the application required.

Explore the full Protocol Lab series here: github.com/pathvector-studio/protocol-lab. If these labs are useful to you, please ⭐ star the repo on GitHub — it genuinely helps others find the project.

Next up, we'll go beyond a single rate cap and look at classful shaping — carving one link into multiple prioritized classes with HTB.

Top comments (0)