A floating (concurrent) license sells a number of seats — say ten — that a team shares. Anyone can grab a seat when they start the app, and the eleventh person to try is told to wait. The whole model rests on one server-side number: how many seats are in use right now? Get that number wrong in the customer's favour and you give away seats; get it wrong in your favour and you lock out people who paid.
The naive implementation gets it wrong almost immediately. You increment a counter on checkout and decrement it on checkin, and it works beautifully in a demo where every app exits cleanly. Then the first real crash happens — a StackOverflowException that takes the process down without unwinding, a VM that the ops team pauses and throws away, a developer who closes the lid and drives home — and that seat is never returned. The counter never comes back down. Do that a few dozen times over a quarter and a ten-seat license is quietly stranded at two usable seats, and the support ticket says "we bought ten, why can only two of us run it?"
This post is about the fix: stop treating checkout and checkin as a matched pair you can rely on, and treat every seat as a lease with a time-to-live that the client has to keep alive with heartbeats. A seat comes back either when the client releases it or when its lease expires — whichever happens first. The expiry is what makes the count self-healing when a client dies the wrong way.
The lease is a signed claim with an expiry
A Keyright lease is not a boolean "you have a seat." It is a small set of claims the issuing service signs — the same signature scheme that protects every other Keyright token — so the client can read it, cache it, and prove it, but cannot alter it. The fields that matter for concurrency are the license id, a server-assigned seat id, the holder (so a UI can show who has the other nine seats), and two timestamps: when the lease was issued and when it expires.
public sealed record SeatLease
{
public required string LicenseId { get; init; }
public required string SeatId { get; init; } // server-assigned, unique per live seat
public required string Holder { get; init; } // user or machine label, for display
public required DateTimeOffset IssuedAt { get; init; }
public required DateTimeOffset ExpiresAt { get; init; } // IssuedAt + policy.Ttl
// Signed by the issuing service; verified on the client against the embedded public key.
public required string Signature { get; init; }
public bool IsLive(DateTimeOffset now, TimeSpan skew) => now <= ExpiresAt + skew;
}
The client never computes ExpiresAt itself and never trusts its own clock to extend it — the expiry is whatever the server signed. The only local clock use is the honest direction: deciding the lease has lapsed so the client stops acting on it. We will come back to clock skew, because it is the one place this design can bite you.
Checkout: hand out a seat only if one is free
Checkout is the one operation that must be serialized per license, because it is where two clients can race for the last seat. The server counts the leases that are still live, refuses if the pool is full, otherwise mints a fresh lease with a new seat id and returns it signed.
public sealed class SeatService
{
private readonly ILeaseStore _store; // persistence is an implementation detail
private readonly ILeaseSigner _signer; // holds the private key; server-only
private readonly TimeProvider _clock;
public async Task<CheckoutResult> CheckoutAsync(
string licenseId, string holder, SeatPolicy policy, CancellationToken ct)
{
// Serialize per license so two callers cannot both see "one seat free".
await using var _ = await _store.LockLicenseAsync(licenseId, ct);
var now = _clock.GetUtcNow();
var live = await _store.CountLiveLeasesAsync(licenseId, now, policy.Skew, ct);
if (live >= policy.SeatCount)
return CheckoutResult.NoSeatsAvailable(policy.SeatCount);
var lease = new SeatLease
{
LicenseId = licenseId,
SeatId = Guid.NewGuid().ToString("N"),
Holder = holder,
IssuedAt = now,
ExpiresAt = now + policy.Ttl,
Signature = "" // filled by the signer below
};
var signed = _signer.Sign(lease);
await _store.UpsertAsync(signed, ct);
return CheckoutResult.Granted(signed);
}
}
Two details decide whether the count stays honest. First, CountLiveLeasesAsync must count by expiry, not by a status flag — a lease is live if now <= ExpiresAt + skew, full stop. If you keep a separate "active" column and forget to clear it, you are back to the leaked-counter problem. Second, the per-license lock is mandatory. Without it, two clients checking out the tenth seat at the same millisecond both read live == 9, both pass the check, and you have eleven seats out against a ten-seat license. The lock scope is one license, held for microseconds, so it does not serialize your whole service.
Heartbeat: the client renews its own lease
Once a client holds a lease, its only job is to come back before the expiry and ask the server to push the expiry forward. The server re-checks that the seat still exists (it might have been revoked or reaped) and, if so, issues a new lease for the same seat id with a later expiry.
public async Task<HeartbeatResult> HeartbeatAsync(
string licenseId, string seatId, SeatPolicy policy, CancellationToken ct)
{
await using var _ = await _store.LockLicenseAsync(licenseId, ct);
var now = _clock.GetUtcNow();
var existing = await _store.FindAsync(licenseId, seatId, ct);
// Seat was revoked or already reaped — the client must check out again.
if (existing is null || !existing.IsLive(now, policy.Skew))
return HeartbeatResult.SeatLost();
var renewed = _signer.Sign(existing with
{
IssuedAt = now,
ExpiresAt = now + policy.Ttl
});
await _store.UpsertAsync(renewed, ct);
return HeartbeatResult.Renewed(renewed);
}
Note what heartbeat does not do: it never increments the seat count. It only ever renews a seat the client already holds, so there is no race to serialize against the pool size — the lock here is just to avoid a renew colliding with a reap of the same row. A heartbeat for a seat that has already been reaped returns SeatLost, and the client's correct response is to run checkout again, which will either get it a fresh seat or tell it the pool is full.
On the client, the heartbeat is a background loop that fires at a fraction of the TTL so a single dropped request is not fatal. A third of the TTL gives you two or three attempts before the lease lapses. Add a little jitter so a thousand clients that all launched after a deploy do not heartbeat in lockstep.
public sealed class LeaseKeeper : BackgroundService
{
private readonly KeyrightClient _client;
private readonly LeaseState _state; // holds the current signed lease for enforcement
private readonly TimeProvider _clock;
protected override async Task ExecuteAsync(CancellationToken stop)
{
while (!stop.IsCancellationRequested)
{
var lease = _state.Current;
// Beat at ~1/3 TTL, with jitter, measured from this lease's own window.
var window = lease.ExpiresAt - lease.IssuedAt;
var baseDelay = window / 3;
var jitter = TimeSpan.FromMilliseconds(Random.Shared.Next(0, 2_000));
await Task.Delay(baseDelay + jitter, stop);
try
{
var result = await _client.HeartbeatAsync(lease.LicenseId, lease.SeatId, stop);
if (result.Renewed)
_state.Replace(result.Lease); // new expiry, enforcement continues
else
await ReacquireOrDegradeAsync(stop); // SeatLost: try checkout again
}
catch (HttpRequestException)
{
// Transient: do nothing. The lease is still valid until ExpiresAt; we will
// retry on the next tick. Only a *lapsed* lease should degrade the app.
}
}
}
}
The catch is the whole point of the TTL. A failed heartbeat is not an eviction — the lease the client already holds is valid until its signed ExpiresAt, so a thirty-second network blip is invisible. The client only degrades (read-only mode, a "seat lost" banner, whatever your product does) when the lease it holds has actually lapsed and a re-acquire failed. That separation — transient failure versus genuine expiry — is what keeps a flaky café Wi-Fi from kicking a paying user out mid-edit.
The reaper: reclaim what the clients did not
Expiry on the lease is necessary but not sufficient. A crashed client stops heartbeating, and its lease will read as expired to anyone who checks — but nothing checks until the next checkout happens. If the pool is quiet, a dead seat can sit in the store looking occupied for a long time, and CountLiveLeasesAsync already excludes it (it counts by expiry), so correctness is fine. What you lose without a reaper is tidiness and observability: stale rows pile up, and a dashboard that lists "current holders" shows ghosts.
The reaper is a periodic sweep that deletes leases whose expiry (plus skew, plus an optional grace) is well in the past.
public sealed class LeaseReaper(ILeaseStore store, TimeProvider clock) : BackgroundService
{
private static readonly TimeSpan SweepInterval = TimeSpan.FromSeconds(30);
protected override async Task ExecuteAsync(CancellationToken stop)
{
while (!stop.IsCancellationRequested)
{
var now = clock.GetUtcNow();
// Reap only leases that are past expiry by a margin, so a client whose heartbeat
// is a few seconds late is never reaped out from under itself.
var cutoff = now - LeasePolicy.ReapGrace; // e.g. expiry + 15s
var reaped = await store.DeleteExpiredAsync(before: cutoff, stop);
if (reaped > 0)
Log.SeatsReclaimed(reaped);
await Task.Delay(SweepInterval, stop);
}
}
}
The reaper and CountLiveLeasesAsync must agree on the arithmetic, or you get the nastiest bug in this whole design: a seat that the counter treats as free but the reaper has not yet deleted, or vice versa. Keep the rule in one place — a lease is live iff now <= ExpiresAt + skew — and have both the counter and the reaper call the same predicate. The reaper's ReapGrace is purely a safety margin on deletion; it must be larger than the skew the counter allows, so the reaper never deletes a row the counter would still count. Delete too eagerly and a client whose heartbeat is three seconds late finds its seat gone and has to re-checkout for no reason.
Clock skew is the one thing that will bite you
Every timestamp here is the server's. The client reads ExpiresAt to know when to stop, but it compares that against its own clock, and client clocks are wrong — sometimes by minutes, occasionally by hours, and a determined user can set theirs to anything. Two failure modes follow.
If the client's clock runs slow (behind the server), it thinks the lease is still live after the server considers it expired. Harmless for counting — the server reaps the seat on schedule regardless of what the client believes — but it means a client can briefly act on a lease the server has already reclaimed. Keep the enforcement window tight and this is a sub-TTL effect that the next heartbeat corrects.
If the client's clock runs fast (ahead of the server), it thinks the lease expired early and degrades or re-acquires too soon. Annoying but not a licensing hole — the user loses nothing they paid for; they just generate an extra checkout.
The skew allowance (policy.Skew) exists to absorb the normal few seconds of drift so that neither side flaps. What it must not do is become a backdoor: never let the client's clock extend a lease. The expiry is server-signed, the server reaps on its own clock, and the client's clock only ever decides to give a seat up early. That asymmetry — client can release early, only the server can extend — is what stops "set your clock back" from being a seat-hoarding exploit. It is the same discipline that defeats trial-clock rollback, applied to concurrency.
What you actually ship
The pieces are small, and most of the correctness lives in two rules you can state in a sentence each. A seat is live iff now <= ExpiresAt + skew, and both the counter and the reaper obey that one predicate. The client may release a seat early but may never extend one — only the server's signature does that. Get those two right and the rest is plumbing: a signed lease record, a checkout that locks per license and counts by expiry, a heartbeat that renews without touching the count, a background keeper on the client that distinguishes a dropped request from a lapsed lease, and a reaper that tidies up on a margin. The payoff is a concurrent-seat count that heals itself: a customer who buys ten seats can always run ten, a crashed client costs them one seat for at most one TTL, and nobody has to file a ticket because the licensing quietly ate their seats.
Top comments (0)