DEV Community

SMSRoute Dev
SMSRoute Dev

Posted on

Your SMPP Bind Is Idle 90% of the Time: Window Size, RTT, and the Throughput Math

The complaint: "we bought 100 TPS and we're getting 12"

An operator provisions a bind rated at 100 messages/second. Production measures 12. Nobody has touched the carrier config, and the provider's dashboard shows no throttling errors — no ESME_RTHROTTLED, no queue backlog on their side.

The bug is not the carrier. It is a client that sends one submit_sm and waits for the matching submit_sm_resp before sending the next.

That design has a hard ceiling that has nothing to do with what you purchased:

max_throughput = window_size / round_trip_time
Enter fullscreen mode Exit fullscreen mode

With a window of 1 and an 80 ms RTT to the SMSC:

1 / 0.080s = 12.5 messages/second
Enter fullscreen mode Exit fullscreen mode

Twelve. Exactly what was measured. The bind spends roughly 90% of every second waiting on network latency with zero bytes in flight. You cannot buy your way out of this; 1000 TPS of entitlement divided by a window of 1 is still 12.5.

What "window" actually means in SMPP

SMPP v3.4 is an asynchronous protocol running over a single TCP connection. Every PDU carries a sequence_number, and responses reference the sequence number of the request they answer. Nothing in the specification requires you to wait for a response before sending the next request.

The window is simply how many PDUs you allow to be outstanding — sent, not yet acknowledged — at any moment. It is a client-side choice, enforced by your own bookkeeping. Most naive clients set it to 1 implicitly by writing synchronous code:

# implicit window = 1: the ceiling is 1/RTT, forever
for msg in queue:
    sock.send(encode_submit_sm(msg, seq=next_seq()))
    resp = read_pdu(sock)          # <-- blocks here, link goes idle
    handle(resp)
Enter fullscreen mode Exit fullscreen mode

The asynchronous form decouples sending from acknowledgement:

# window = N: up to N PDUs in flight
inflight = {}                       # sequence_number -> (msg, sent_at)
while queue or inflight:
    while queue and len(inflight) < WINDOW:
        msg = queue.popleft()
        seq = next_seq()
        inflight[seq] = (msg, now())
        sock.send(encode_submit_sm(msg, seq=seq))
    pdu = read_pdu(sock)            # responses arrive out of order
    if pdu.command_id == SUBMIT_SM_RESP:
        msg, sent_at = inflight.pop(pdu.sequence_number, (None, None))
        if msg is None:
            log.warning("resp for unknown seq %d", pdu.sequence_number)
        else:
            observe_rtt(now() - sent_at)
            handle(msg, pdu)
Enter fullscreen mode Exit fullscreen mode

With a window of 20 and the same 80 ms RTT:

20 / 0.080s = 250 messages/second
Enter fullscreen mode Exit fullscreen mode

Now the carrier's 100 TPS limit is the binding constraint, which is what you paid for.

Choosing a window size without guessing

The useful heuristic is Little's Law. To sustain a target rate you need enough messages in flight to cover one round trip:

window >= target_tps * rtt_seconds
Enter fullscreen mode Exit fullscreen mode

For 100 TPS at 80 ms:

100 * 0.080 = 8      # minimum
Enter fullscreen mode Exit fullscreen mode

Set it somewhat above the minimum to absorb RTT jitter — a window of 16 to 32 is a reasonable operating range for most binds. Going far higher does not help: once the carrier's rate limit is the bottleneck, extra window depth only converts into queueing delay and larger loss on reconnect.

Two constraints bound it from above:

The SMSC has its own window. Many SMSCs cap outstanding PDUs per bind, commonly in the 10 to 100 range. Exceeding it does not fail cleanly — some implementations stop reading, and TCP backpressure stalls your writes. Ask the carrier for the number rather than probing for it in production.

Everything in flight is at risk on disconnect. A window of 500 means 500 messages whose fate is unknown when the socket drops. Each one is a message you must either re-submit, and possibly duplicate, or drop. Window size is therefore also a decision about how much ambiguity you are willing to resolve after a failure.

The sequence_number bug that silently drops messages

sequence_number is a 4-byte field, but SMPP restricts valid values to 0x00000001 through 0x7FFFFFFF. Two errors are common and both are quiet.

The first is starting at zero or letting the counter run past the maximum into negative territory in languages with signed integers. The second is reusing a number that is still in flight.

Reuse is the dangerous one. If sequence 42 is outstanding and you assign 42 to a second message, the next submit_sm_resp with sequence 42 is ambiguous. Your correlation map holds one entry, so one message gets the response and the other is orphaned. It is never acknowledged, never retried, and never reported as failed. It simply vanishes, and the only symptom is a delivery rate slightly below what your own counters claim.

Correct generation wraps within the legal range and skips values still in use:

MAX_SEQ = 0x7FFFFFFF

def next_seq(self):
    for _ in range(MAX_SEQ):
        self._seq = self._seq + 1 if self._seq < MAX_SEQ else 1
        if self._seq not in self.inflight:
            return self._seq
    raise RuntimeError("sequence space exhausted: inflight window is too large")
Enter fullscreen mode Exit fullscreen mode

If that loop ever exhausts, your window is larger than the sequence space, which means it is far larger than it should be.

enquire_link: the keepalive that decides how fast you notice death

A TCP connection to an SMSC can be dead for minutes without either side noticing. NAT devices and stateful firewalls drop idle mappings silently; no FIN, no RST. Your socket remains ESTABLISHED in the kernel while writes disappear.

enquire_link is SMPP's application-level keepalive. Send it on an interval, expect enquire_link_resp within a timeout, and tear the bind down when the response does not arrive.

ENQUIRE_INTERVAL = 30      # seconds between keepalives
ENQUIRE_TIMEOUT  = 10      # give up on a response after this

if now() - last_pdu_sent > ENQUIRE_INTERVAL:
    seq = next_seq()
    send_enquire_link(seq)
    pending_enquire[seq] = now()

for seq, sent_at in list(pending_enquire.items()):
    if now() - sent_at > ENQUIRE_TIMEOUT:
        log.error("enquire_link %d timed out; rebinding", seq)
        reconnect()
        break
Enter fullscreen mode Exit fullscreen mode

The interval is a tradeoff against NAT idle timeouts, which frequently sit between 60 and 300 seconds. Thirty seconds is comfortably inside typical windows. Pushing to 300 seconds risks the mapping expiring before your keepalive refreshes it, at which point you discover the dead link only when a real message fails.

Note the ordering that matters: detection time is ENQUIRE_INTERVAL + ENQUIRE_TIMEOUT in the worst case, and every message submitted during that window is in doubt.

Reconnect: the part that creates duplicates

When a bind drops with a non-empty window, each in-flight message is in one of three states, and the client cannot distinguish them:

  1. Never reached the SMSC. Safe to resubmit.
  2. Reached the SMSC and was accepted; the response was lost. Resubmitting sends the message twice.
  3. Reached the SMSC and was rejected; the response was lost. Resubmitting is correct.

There is no SMPP mechanism that resolves this ambiguity for you. The protocol gives you no way to ask "did you already accept sequence 42?" — and after a rebind, sequence numbers restart anyway.

The practical resolution is to move the deduplication key up a layer. Attach your own message identifier to each send, keep a short-lived record of identifiers submitted in the last few minutes, and check it before resubmitting after a reconnect. If your platform exposes an idempotency key on send, that mechanism is what it exists for. Providers differ in whether they offer one, so this is worth confirming against your gateway's API documentation before you need it — the SMS API docs for whichever gateway you use should state whether repeated submissions with the same key collapse to one message.

Without a dedup layer, the safe options are both bad: resubmit and accept duplicate OTPs, or drop and accept silent loss. Pick deliberately rather than by default.

Measuring the right thing

Three metrics tell you whether the bind is healthy, and none of them is "messages sent".

Window occupancy — the average size of your in-flight map. If it sits near zero, your window is not the constraint and something upstream is starving the sender. If it pins at the maximum, the window is the constraint and you should raise it or accept the ceiling.

RTT percentiles — measured from send to matching response, not from your queue. The p50 tells you what window size you need; the p99 tells you how much jitter headroom to add. A p99 that is many multiples of p50 usually indicates SMSC-side queueing, and raising your window will not fix it.

Unmatched responses — responses arriving for sequence numbers not in your map. This should be exactly zero. Any nonzero value means either sequence reuse or a correlation bug, and both cause silent message loss.

A bind that reports high throughput and a nonzero unmatched-response count is not delivering what its counters claim.

The short version

  • One submit_sm at a time caps throughput at 1/RTT, independent of what the carrier sold you.
  • Size the window as target_tps * rtt, then add headroom for jitter; confirm the SMSC's own cap rather than probing it.
  • Generate sequence_number within 1..0x7FFFFFFF and never reuse a value still in flight, or messages disappear without error.
  • Send enquire_link well inside NAT idle timeouts; worst-case detection is interval plus timeout.
  • Everything in the window is ambiguous when the link drops. Deduplicate above SMPP or choose which failure mode you prefer.

Top comments (0)