Implementing 3 Python API Spend Limits with Capability Routing Preferences

#python #api #architecture

Short answer: use one account-level API spend limit as the hard stop, then shape cost per capability with routing preferences and application-owned quotas, so a logistics platform can keep accepting essential events without turning features off.

The bill is made of request volume multiplied by the selected route's unit cost, plus whatever the platform retains for replay and audit. Start with the dominant term. In an illustrative month with 12 million tracking events, assume address normalization runs on every event while proof-of-delivery enrichment runs on 2 million. If normalization is the expensive route, tuning the smaller enrichment stream first is theater; changing the preferred route for the 12 million-call capability moves the bill, while a hard account cap merely defines where all work eventually stops. These are workload inputs, not benchmark claims. Replace them with invoice exports before setting policy.

Retention belongs in the same calculation because an outage turns yesterday's “temporary” payload into tomorrow's replay source. Keep the immutable event envelope, policy decision, selected route class, attempt number, and result category. Deliberately stop keeping full third-party response bodies after the investigation window, and never retain credentials in an audit record. That reduces stored bytes and secret exposure, but the loss is real: after the response expires, an investigator can prove which policy and route were used, yet may be unable to reconstruct every byte returned by the dependency.

That is the trade.

How should capability API spend limits and routing preferences shape cost?

Treat the controls as three layers with different jobs. The account limit is the last financial boundary. A capability budget says how much of that boundary address checks, route estimation, notification, or document extraction may consume. A routing preference decides which eligible provider class receives the next request. Conflating the layers creates a bad failure mode: the global cap becomes the first control anyone notices, then every capability goes dark at once.

The application needs its own quota per endpoint because an account-level cap cannot express that tracking ingestion is essential while a richer delivery summary is deferrable. “Per endpoint” should mean a stable internal capability identifier, not a raw URL string; URL versions and retry paths change, while the business action remains the same. Charge the budget when work is admitted, attach the reservation to the event ID, and make retries reuse that reservation. Otherwise a dependency outage causes retries to spend the same logical event three or four times.

Do not turn the feature off when its preferred route is exhausted. Move through an explicit ladder: preferred route, lower-cost eligible route, delayed execution, then a minimal result that preserves the event and exposes its pending state. The ladder must be defined per capability because a delayed proof-of-delivery thumbnail and a delayed hazardous-materials validation do not carry the same operational risk. It's tempting to hide this distinction behind one “degraded mode” flag, but that flag is too coarse to audit.

Control	Decision it owns	Evidence to retain	Failure if misplaced
Account hard limit	Maximum aggregate spend	Limit version and remaining band	All capabilities stop together
Capability quota	Admission for one business action	Capability, reservation, event ID	A noisy endpoint consumes the account
Routing preference	Order of eligible route classes	Policy version and chosen class	Savings cannot be attributed
Retention rule	Replay and investigation horizon	Object hash and expiry class	Audits either leak data or lack evidence

Routing changes should pass a test call before production traffic relies on the projected cost. A successful configuration write proves only that the policy was accepted; it does not prove that a representative request selects an eligible route or produces an acceptable result. Record the test input class, policy version, chosen route class, and outcome, but redact tokens and user payload fields. OWASP's secrets-management guidance is useful here because the audit trail and the secret lifecycle must remain separate.

Build the quota decision as an auditable state machine

The policy state machine can stay small. It accepts an event, checks an idempotent reservation, chooses the first allowed route under the capability's soft budget, and records the decision before dispatch. If no paid route remains, it queues the work or emits a minimal local result according to the capability policy. The event is still accepted.

Below is a runnable Python model of that decision boundary. The numbers are test fixtures expressed as abstract cost units; they are not vendor prices. The hash chain makes accidental editing detectable within the exported sequence, although it is not a substitute for access-controlled, immutable storage.

from __future__ import annotations

from dataclasses import asdict, dataclass
from decimal import Decimal
from hashlib import sha256
import json
from typing import Literal


Disposition = Literal["dispatch", "defer", "minimal"]


@dataclass(frozen=True)
class Route:
    name: str
    cost_units: Decimal


@dataclass(frozen=True)
class Decision:
    event_id: str
    capability: str
    disposition: Disposition
    route: str | None
    reserved_units: str
    policy_version: str
    previous_hash: str


class BudgetPolicy:
    def __init__(
        self,
        account_limit: Decimal,
        capability_limits: dict[str, Decimal],
        preferences: dict[str, list[Route]],
        fallbacks: dict[str, Disposition],
        policy_version: str,
    ) -> None:
        self.account_limit = account_limit
        self.capability_limits = capability_limits
        self.preferences = preferences
        self.fallbacks = fallbacks
        self.policy_version = policy_version
        self.account_reserved = Decimal("0")
        self.capability_reserved = {
            capability: Decimal("0") for capability in capability_limits
        }
        self.reservations: dict[str, Decision] = {}
        self.audit: list[dict[str, str | None]] = []

    def admit(self, event_id: str, capability: str) -> Decision:
        if event_id in self.reservations:
            return self.reservations[event_id]

        selected: Route | None = None
        for route in self.preferences[capability]:
            account_ok = self.account_reserved + route.cost_units <= self.account_limit
            capability_ok = (
                self.capability_reserved[capability] + route.cost_units
                <= self.capability_limits[capability]
            )
            if account_ok and capability_ok:
                selected = route
                break

        previous_hash = self.audit[-1]["entry_hash"] if self.audit else "GENESIS"
        disposition: Disposition = "dispatch" if selected else self.fallbacks[capability]
        decision = Decision(
            event_id=event_id,
            capability=capability,
            disposition=disposition,
            route=selected.name if selected else None,
            reserved_units=str(selected.cost_units if selected else Decimal("0")),
            policy_version=self.policy_version,
            previous_hash=str(previous_hash),
        )

        if selected:
            self.account_reserved += selected.cost_units
            self.capability_reserved[capability] += selected.cost_units

        entry = asdict(decision)
        canonical = json.dumps(entry, sort_keys=True, separators=(",", ":"))
        entry["entry_hash"] = sha256(canonical.encode("utf-8")).hexdigest()
        self.audit.append(entry)
        self.reservations[event_id] = decision
        return decision


policy = BudgetPolicy(
    account_limit=Decimal("20"),
    capability_limits={"tracking": Decimal("12"), "delivery_summary": Decimal("4")},
    preferences={
        "tracking": [Route("standard", Decimal("2")), Route("economy", Decimal("1"))],
        "delivery_summary": [Route("rich", Decimal("4")), Route("compact", Decimal("1"))],
    },
    fallbacks={"tracking": "minimal", "delivery_summary": "defer"},
    policy_version="quota-3",
)

print(policy.admit("evt-1001", "tracking"))
print(policy.admit("evt-1001", "tracking"))  # Same reservation on retry.
print(policy.admit("evt-1002", "delivery_summary"))

There is an important limitation in this compact example: an in-memory reservation is not durable across processes or restarts. Production admission needs a transactional store that atomically creates the event-ID reservation and increments both counters, or concurrent workers can oversubscribe the same budget. The audit export also needs independent retention and access controls. Don't put authorization headers, API keys, addresses, or full shipment payloads into the decision row; store a stable event reference and a content hash when later correlation is required.

Keep policy publication separate from request admission. A policy version should be immutable once used, and a new version should become active at a recorded boundary. During an outage, responders can then answer a precise question — “why was event evt-1001 sent by the economy class?” — using the event ID and policy version instead of trying to infer yesterday's configuration from today's dashboard.

Survive an outage without spending twice

Accept the logistics event into a durable local boundary before calling an optional external capability. A transactional outbox is one valid shape: commit the shipment state and an outbound work item together, then let a dispatcher apply the quota policy. The dispatcher must distinguish a new logical event from another delivery attempt. Idempotency covers the first; bounded retry and backoff cover the second.

This is where cost control and availability collide. If a route is unavailable, immediately moving every queued event to the next eligible route may protect latency while destroying the budget forecast; waiting may protect spend while missing an operational deadline. Encode the choice by event class and deadline. A warehouse-arrival event might emit a minimal accepted state and defer enrichment, while a safety-critical validation may reserve budget from a protected capability pool. I'm not sure a universal threshold exists because the correct delay depends on the logistics contract and the consequence of stale data. The policy needs that business deadline as input, not a guess buried in worker code.

Retries are noisy.

Observe reservations, not raw attempts, as the main spend signal. Keep attempts for reliability analysis, but graph logical events admitted, units reserved, route-class distribution, deferred age, and account headroom. Alert before the hard limit by using bands rather than a prediction presented as certainty. A forecast is still useful, especially when route mix changes, but it should never be the enforcement mechanism.

Reconciliation closes the loop. Compare reserved units with settled usage at a fixed cadence, flag unmatched event IDs, and release or correct reservations according to a documented rule. The model above reserves the listed route cost before dispatch; a real integration needs to define what happens when the final charge differs, a request is cancelled, or the outcome arrives after the accounting window. Without reconciliation, conservative reservations accumulate and eventually imitate a spend outage even though the account has room.

Test the policy before it owns production traffic

Unit tests should cover boundary equality, one unit over the capability quota, one unit over the account limit, duplicate event IDs, preference reordering, and each fallback disposition. Then run a replay with a redacted distribution of real capability labels and payload-size bands. The goal is not a flattering average. Look for the largest route-mix change, the oldest deferred essential event, and any capability that reaches its quota while the account retains substantial headroom.

Deployment needs two approvals: the routing policy and the budget policy. Roll them out with one shared change identifier, test the effective routing with a representative call, and retain the result beside the policy version. If they are changed independently, an operator can select a more expensive route under an old quota or tighten a quota before the intended lower-cost route becomes eligible. A dry-run mode can calculate decisions without dispatching work, but its output must pass through the same state-machine code or it proves little.

The catch is operational weight. Application-owned quotas require transactional counters, reconciliation, policy versioning, replay tests, and an on-call view of deferred work. They are not suitable when a small service has one homogeneous capability and no meaningful routing choice; stick with the account hard limit and a straightforward queue in that case. At the other extreme, if legal or safety rules forbid a lower-fidelity result, do not use graceful degradation for that capability. Reserve protected capacity, fail admission visibly when it is exhausted, and preserve the original event for audit.

Your mileage may vary on retention. Longer decision history improves investigation and route-cost attribution, while shorter retention reduces storage and the blast radius of sensitive metadata. Set separate horizons for the immutable event reference, the policy decision, and any dependency response; “keep all logs” isn't an architecture.

The final acceptance test is blunt: exhaust one capability's application quota while the account still has headroom. Essential event ingestion must continue, the exhausted capability must follow its declared fallback, another capability must still route normally, and every outcome must name the same immutable policy version. Then approach the account hard limit and confirm that the last boundary is enforced across capabilities. Those two tests demonstrate that routing shapes cost, local quotas isolate features, and the global cap remains a cap rather than the everyday control plane.

DEV Community

Implementing 3 Python API Spend Limits with Capability Routing Preferences

How should capability API spend limits and routing preferences shape cost?

Build the quota decision as an auditable state machine

Survive an outage without spending twice

Test the policy before it owns production traffic

References

Further reading

Top comments (0)