Oluwasegun Adedigba for AWS Community Builders

Posted on Nov 4 AI-assisted

Non-functional Application Requirements: Performance

#aws #architecture #ai #performance

In 2021, Vodafone published a test every team arguing about performance should know. Two landing pages looked and worked the same, with paid traffic split 50/50, about 34,000 visits a day each. One version rendered its critical HTML on the server, moved a widget off the client and resized its hero image. Its Largest Contentful Paint was 31% better in field data (5.7 s against 8.3 s).

That page sold 8% more, with 15% more leads and 11% more carts per visit (web.dev).

Here's the part I like: 5.7 seconds is still "poor" by Google's thresholds (over 4). The winning page wasn't fast, just less slow, and people still bought more.

So when someone says performance can wait until after launch, show them this.

TL;DR

Budget on percentiles (p95/p99), and say whether you mean latency or throughput. Averages hide the users you lose.

Cache in layers (edge, application, database), picked by how fresh the data must be.

Lambda cold starts hit under 1% of invocations (AWS), which is exactly your tail.

Load-test with an open model and fail the build when p95 goes over budget.

For LLM features, budget time-to-first-token, output tokens/sec and end-to-end p95 separately. Streaming and prompt caching are your biggest levers.

What performance actually means

Performance is how fast, and how much, work your system does for a user under a stated load. Two numbers make it testable.

Latency is the time one request takes between a defined start and end, so latency at the load balancer and in the browser are different numbers. Throughput is work completed per unit of time (requests, bytes, tokens) at an acceptable latency. Without a latency target it's meaningless, since any queue accepts work if nobody cares when it finishes.

Percentiles describe latency's shape: p50 is the median, p95 the value 95% of requests beat, p99 the slowest 1%. Averages describe no real user: if 98 requests take 50 ms and 2 take 3 seconds, the average is 109 ms and the p99 is 3 seconds.

That high end is tail latency, and it compounds. If one server is slow on 1 request in 100, a request fanning out to 100 such servers is slow 63% of the time (The Tail at Scale). Microservices, RAG pipelines and agent tool calls all fan out.

For frontends, Google's Core Web Vitals are judged at p75 of page loads. LCP (loading) is good at 2.5 s or less and poor over 4.0 s. INP (responsiveness, which replaced FID in 2024) is good at 200 ms or less and poor over 500 ms. CLS (visual stability) is good at 0.1 or less and poor over 0.25.

A testable requirement reads: "Product page LCP ≤ 2.5 s at p75 on mobile. GET /products/{id} p95 ≤ 150 ms and p99 ≤ 400 ms at 200 requests/second."

Why it's worth the effort

It isn't a one-off. A Google-commissioned Deloitte and 55 study of 37 European and US brands and more than 30 million sessions found a 0.1-second improvement in mobile site speed lifted retail conversions by 8.4% and travel conversions by 10.1% (Milliseconds Make Millions, 2020).

Distance matters too. AWS's first CloudFront edge location in Lagos (June 2023) brought up to a 20% average latency improvement for content it served. If your users are in West Africa and your origin is in eu-west-1, physics is part of your budget.

When to think about it

Design: write the budget per user journey and split it across hops. Of the page's 2.5 seconds, how much goes to the API, and how much to the database?
Build: emit latency with a cache hit or miss dimension, and profile before you optimize.
Test: load-test at the target arrival rate before every release, on production-shaped data, with thresholds that fail the pipeline.
Operate: alarm on p95 and p99, not the average, and watch CloudWatch RUM beside synthetic tests.

Where the time goes, and what to use

Most performance mistakes are the right tool on the wrong job. First, the layers.

flowchart LR
    U["User"] -->|"TLS ends at a nearby edge"| CF["CloudFront<br/>static assets and cacheable GETs"]
    CF -->|"miss or dynamic request"| AG["API Gateway"]
    AG --> L["Lambda<br/>connections reused when warm"]
    L -->|"check cache first"| V["ElastiCache for Valkey"]
    V -->|"hit: return"| L
    L -->|"on miss: GetItem"| D["DynamoDB"]
    D --> L
    L -->|"SET with jittered TTL"| V

Each hop to the right adds latency and load, and each cache exists to stop the request going further.

Edge caching with CloudFront

CloudFront's speed comes from its hit ratio, which comes from the cache key. Every header, cookie or query string in it splits the cache. Two cookies with three values each can mean nine origin requests per object.

flowchart LR
    R["Requests for one object"] --> T["Tight cache key<br/>only what the origin uses"]
    R --> W["Cache key with two cookies<br/>three values each"]
    T --> T1["1 cached copy<br/>1 origin request"]
    W --> W1["3 x 3 = 9 cached copies<br/>9 origin requests"]
    T1 --> H["Higher hit ratio"]
    W1 --> M["Lower hit ratio"]

Every value you add to the cache key multiplies the copies CloudFront has to fetch from your origin.

Pick the policy by content:

CachingOptimized (24-hour default TTL) suits versioned static assets. Keep it off changing HTML and API responses, because its 1-second minimum TTL overrides an origin's no-cache.
UseOriginCacheControlHeaders works when your origin sends correct Cache-Control.
CachingDisabled is for personalized or write paths, not public content.
Origin Shield helps with global viewers and expensive origins. It adds a per-request charge, so skip it for dynamic or rarely requested content.

For edge logic, CloudFront Functions do sub-millisecond cache-key and header work, with no network access and 10 KB of code. Lambda@Edge gets network access and up to 30 seconds, but caps at 10,000 requests per second per Region.

Add stale-while-revalidate and stale-if-error to Cache-Control so users get a cached answer while CloudFront refreshes. The same goes for Cloudflare or Fastly. The key matters more than the CDN.

Compute: Lambda cold starts

AWS says cold starts "typically occur in under 1% of invocations" in production (AWS Compute Blog, 2025). That 1% is your p99. Since 1 August 2025 INIT is billed in all configurations, so slow init costs money.

Your four options solve different problems:

Right-size memory first. CPU scales with memory, and 1,769 MB is one vCPU. AWS Lambda Power Tuning (open source) finds the sweet spot, and shorter duration often offsets the higher GB-second price.
SnapStart restores a snapshot of an initialized environment, for Java 11+, Python 3.12+ and .NET 8+ with heavy init. It can't be used with provisioned concurrency, EFS and ephemeral storage over 512 MB. It's free for Java, while Python and .NET pay cache and restore charges.
Provisioned concurrency keeps N environments initialized, for strict latency on spiky but predictable traffic. It bills continuously, used or not.
Lambda Managed Instances run Lambda on EC2 in your account, many invocations per environment, no cold starts. They fit steady, high-volume traffic at the EC2 price plus a 15% management fee, and code must be thread-safe.

flowchart TB
    A["Is p99 dominated by Init Duration?"] -->|"no"| Z["Fix the hot path first:<br/>memory size, client reuse, slow downstreams"]
    A -->|"yes"| B["Java 11+, Python 3.12+ or .NET 8+?"]
    B -->|"yes"| C["Try SnapStart<br/>check EFS and storage limits"]
    B -->|"no"| D["What does traffic look like?"]
    C -->|"still over budget"| D
    D -->|"steady and high volume"| E["Lambda Managed Instances<br/>or Fargate"]
    D -->|"spiky with known peaks"| F["Provisioned concurrency<br/>with scheduled scaling"]
    D -->|"spiky and rare"| G["Accept cold starts<br/>trim package and init code"]

The first question is a measurement, because many "cold start problems" are slow code running during INIT.

On an Application Load Balancer with uneven request cost, set the target group to least_outstanding_requests, since round robin queues behind busy targets. You lose slow start.

Data: DynamoDB vs DAX vs ElastiCache

Part 1 promised this comparison. ElastiCache runs Valkey (the open-source Linux Foundation fork of Redis), Redis OSS or Memcached, serverless or node-based.

	DynamoDB alone	DynamoDB + DAX	ElastiCache for Valkey
Typical read latency	Single-digit milliseconds	Microseconds for cached eventually consistent reads	In-memory. Measure your own p99
Code change	None	Swap in the DAX client	You write the cache logic
Consistency	Strong or eventual, per read	Eventual only. Strongly consistent reads pass through uncached	Whatever you build: TTLs, invalidation, write-through
Staleness trap	None	Query/Scan cache is not invalidated by item writes, and writes made directly to DynamoDB bypass DAX	Forgotten invalidation. Stampedes when hot keys expire together
Choose when	Latency already meets budget	Read-heavy DynamoDB with hit rate above 90%	Multiple data sources, computed results, or semantic caching for AI
Watch out	Hot partitions	VPC-only. Unbounded attribute names exhaust memory	Serverless caches need TLS. Vector search needs node-based Valkey 8.2

The staleness row catches teams out, so read the DAX consistency docs first.

Start with DynamoDB alone and measure. Before you add a cache, I'd check the access pattern (a scan, a hot partition key), because fixing key design is cheaper than running a cluster.

Load testing tools

The load model matters more than the tool. k6 (open source) gives you an open model via constant-arrival-rate, with thresholds that turn p95 budgets into CI pass or fail, though big tests need distributed runners. Locust (open source) suits Python teams and complex flows, with scripts in Python only. Apache JMeter (GUI and XML plans) covers the most protocols but is heavy to maintain as code. Distributed Load Testing on AWS (v4.3.2) runs JMeter, k6 or Locust on Fargate across Regions, though you pay for the stack and still design the test.

Use an open model. In a closed model each virtual user waits for its last response, so a slowing system receives less load and the test hides the problem. That's coordinated omission (k6 docs).

Decide: the right fix, and what it costs elsewhere

Match the symptom to the fix before buying capacity.

If your problem is…	Reach for	Not
Static assets slow far from the origin	CloudFront with a tight cache key	A bigger origin
Repeated identical reads on DynamoDB	DAX	A hand-written cache
Expensive computed results or mixed sources	ElastiCache for Valkey, cache-aside	DAX (it caches items, not your computations)
Init Duration in your p99	SnapStart, then provisioned concurrency	More memory alone
Steady high-volume compute	Lambda Managed Instances or Fargate	Provisioned concurrency sized for peak all day
LLM responses feel slow	Streaming, prompt caching, smaller model	A bigger instance of anything

Each fix touches the other NFRs. Some effects help. Each cache hit is load the database never sees (scalability), stale-if-error serves cached content while the origin is down (availability), fast INP helps low-end devices (accessibility), and cache hits and smaller models cost less (cost) and use less compute per request (sustainability).

Others cost you:

Security: a cache key missing the user ID serves one user's page to another.
Compliance: latency-optimized and cross-Region inference process prompts outside your home Region.
Accessibility: streaming text floods screen readers.
Availability: a cold cache after failover stampedes the database.
Scalability: a cache sized for today's keys evicts hot ones at tomorrow's volume.
Maintainability: invalidation logic is a classic source of bugs.
Cost: provisioned concurrency, the Priority tier and idle nodes bill regardless.
Sustainability: always-warm capacity burns energy at idle.

None of that means skip the work. Just write down what you traded.

Hands-on: cache-aside in front of DynamoDB, with a p95 gate

Let's build the middle of the first diagram: a Lambda read path for GET /products/{id} with ElastiCache Serverless for Valkey in front of DynamoDB, plus a k6 gate. The cache can speed requests up but never break them, TTLs carry jitter so hot keys don't expire together, and latency is emitted per path.

sequenceDiagram
    participant G as API Gateway
    participant L as Lambda
    participant V as Valkey
    participant D as DynamoDB
    participant CW as CloudWatch
    G->>L: GET /products/id
    L->>V: GET product key, 50 ms timeout
    alt cache hit
        V-->>L: cached product
    else miss, timeout or cache error
        L->>D: GetItem, eventually consistent
        D-->>L: item
        L->>V: SET with TTL of 300 s, plus or minus 10%
        Note over L,V: a failed SET is ignored
    end
    L->>CW: EMF log line, ProductReadLatency by CacheResult
    L-->>G: 200 with the product, or 404

Every cache failure falls through to DynamoDB, and the jittered TTL spreads out when hot keys expire.

Checked against the valkey-py docs, boto3 DynamoDB get_item, ElastiCache in-transit encryption and CloudWatch Embedded Metric Format on Oct 28, 2026. Not deployed, so try it in a sandbox account first.

# handler.py — cache-aside read path. Python 3.12 Lambda in the cache's VPC,
# with a DynamoDB gateway VPC endpoint so it can reach the table without NAT.
# Package dependency: valkey (pip install valkey). boto3 is in the Lambda runtime.
import json, os, random, time
import boto3
from boto3.dynamodb.types import TypeDeserializer
from valkey.cluster import ValkeyCluster

TABLE = os.environ["TABLE_NAME"]
TTL_SECONDS = int(os.environ.get("CACHE_TTL_SECONDS", "300"))

# Created once per execution environment, outside the handler, so warm invocations reuse them.
ddb = boto3.client("dynamodb")
deser = TypeDeserializer()
_cache = None

def cache():
    # Lazy: if the cache is unreachable at cold start, the function still serves from DynamoDB.
    global _cache
    if _cache is None:
        _cache = ValkeyCluster(
            host=os.environ["CACHE_ENDPOINT"],  # serverless cache endpoint
            port=6379,
            ssl=True,             # serverless caches have in-transit encryption enabled
            socket_timeout=0.05,  # 50 ms: a slow cache must not be slower than the database
            decode_responses=True,
        )
    return _cache

def emit_latency(ms, cache_result):
    # Embedded Metric Format: this log line becomes a CloudWatch metric, no PutMetricData call.
    print(json.dumps({
        "_aws": {
            "Timestamp": int(time.time() * 1000),
            "CloudWatchMetrics": [{
                "Namespace": "Nfr/Performance",
                "Dimensions": [["CacheResult"]],
                "Metrics": [{"Name": "ProductReadLatency", "Unit": "Milliseconds"}],
            }],
        },
        "CacheResult": cache_result,
        "ProductReadLatency": ms,
    }))

def get_product(product_id):
    key = f"product:{product_id}"
    try:
        cached = cache().get(key)
        if cached is not None:
            return json.loads(cached), "hit"
    except Exception:
        pass  # cache trouble degrades to a database read; it never fails the request
    # Eventually consistent read (the default): half the cost of a strongly consistent one.
    item = ddb.get_item(TableName=TABLE, Key={"pk": {"S": product_id}}).get("Item")
    if item is None:
        return None, "miss"  # not cached: consider caching misses briefly if bots probe random IDs
    product = {k: deser.deserialize(v) for k, v in item.items()}
    try:
        ttl = int(TTL_SECONDS * random.uniform(0.9, 1.1))  # +/-10% jitter spreads expiry
        cache().set(key, json.dumps(product, default=str), ex=ttl)
    except Exception:
        pass
    return product, "miss"

def handler(event, context):
    start = time.perf_counter()
    product, result = get_product(event["pathParameters"]["id"])
    emit_latency((time.perf_counter() - start) * 1000, result)
    if product is None:
        return {"statusCode": 404, "body": "{}"}
    return {"statusCode": 200, "headers": {"Content-Type": "application/json"},
            "body": json.dumps(product, default=str)}

Graph ProductReadLatency p99 by CacheResult. If hit and miss sit close, the cache isn't helping. Delete it.

Now the gate. This k6 test holds 200 requests per second for ten minutes and exits non-zero if p95 or p99 breaks budget.

Checked against the k6 thresholds and constant-arrival-rate docs on Oct 28, 2026. Not deployed, so try it in a sandbox account first.

// load-test.js — run: k6 run -e BASE_URL=https://your-api.example.com load-test.js
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  scenarios: {
    steady_reads: {
      executor: 'constant-arrival-rate', // open model: load does not drop when the API slows
      rate: 200,                         // 200 iterations...
      timeUnit: '1s',                    // ...per second
      duration: '10m',
      preAllocatedVUs: 50,
      maxVUs: 400,                       // headroom to hold the rate as latency rises
    },
  },
  thresholds: {
    'http_req_duration{endpoint:product}': ['p(95)<150', 'p(99)<400'],
    http_req_failed: [{ threshold: 'rate<0.01', abortOnFail: true, delayAbortEval: '30s' }],
    checks: ['rate>0.99'],
  },
};

const ids = ['p-001', 'p-002', 'p-003', 'p-004', 'p-005'];

export default function () {
  // Skewed traffic: 80% of reads hit one hot item, as in most real catalogs.
  const id = Math.random() < 0.8 ? ids[0] : ids[Math.floor(Math.random() * ids.length)];
  const res = http.get(`${__ENV.BASE_URL}/products/${id}`, { tags: { endpoint: 'product' } });
  check(res, { 'status is 200': (r) => r.status === 200 });
}

For a real run, load thousands of IDs with SharedArray so the hit ratio looks like production.

Performance in the age of AI

LLM features need three budgets

An LLM call spends most of its time generating output, but users judge it by when the first words appear. Give it three budgets.

Time to first token (TTFT) covers queueing and prompt processing. Prompt length and caching, model and tier move it, and Bedrock added a TimeToFirstToken CloudWatch metric for streaming APIs in March 2026. Output tokens per second (OTPS) is generation speed, set by the model, the tier and latency-optimized inference. You derive it as OutputTokenCount / (InvocationLatency − TimeToFirstToken). End-to-end p95 is everything, retrieval and tools included. maxTokens, agent steps and retrieval move it, and your trace spans show it.

sequenceDiagram
    participant U as Browser
    participant G as API Gateway
    participant L as Lambda
    participant R as Retriever
    participant B as Bedrock
    U->>G: POST /chat
    G->>L: invoke in STREAM mode
    Note over L: cold start only if no warm environment
    L->>R: embed question and vector search
    R-->>L: top-k chunks
    L->>B: ConverseStream with cached prefix
    Note over B: queue and prefill, a cache hit shortens this
    B-->>L: first token
    L-->>U: first chunk, TTFT ends here
    loop decode at OTPS
        B-->>L: tokens
        L-->>U: chunks
    end
    B-->>L: metadata with usage and latencyMs
    L-->>U: stream closes, end-to-end ends here

Everything before the first chunk (cold start, retrieval, prefill) is dead air for the user, so optimize there first.

Streaming, and the plumbing that limits it

Streaming is the biggest perceived-performance lever, but the plumbing has limits. API Gateway REST integrations time out at 29 seconds. Regional and private APIs can raise it, possibly lowering your throttle quota. HTTP APIs stop at a fixed 30 seconds. Since November 2025, REST APIs can stream proxy integrations for up to 15 minutes (5-minute idle timeout on Regional endpoints), without endpoint caching or VTL.

Lambda streams natively on Node.js, while Python needs a custom runtime or the Lambda Web Adapter. Streams reach 200 MB, capped at 2 MBps after the first 6 MB.

The Bedrock levers

Once you stream, these are Bedrock's knobs.

Lever	Effect	Use when	Caveat (as of Oct 2026)
Prompt caching	Up to 85% lower latency, 90% lower cost per AWS	Long, stable system prompts, documents, tool schemas	512–4,096-token minimum by model. 5-minute default TTL. Not for batch
Priority tier	Up to 25% better OTPS than Standard on most supported models, at a premium	Customer-facing paths at peak	Limited model list. Shares your on-demand quota
Flex tier	Discount for slower processing	Evals, summarization, background agents	Never on an interactive path
Latency-optimized inference	`performanceConfig: {latency: "optimized"}`	Nova Pro, Claude 3.5 Haiku, Llama 3.1 70B/405B	Preview. US Regions via cross-Region inference only. Falls back to standard past its quota
Distilled model	Up to 500% faster, 75% cheaper, under 2% accuracy loss for RAG, per AWS	High-volume, narrow tasks	You need evals to prove quality held
Intelligent Prompt Routing	Routes each prompt to a cheaper or stronger model in one family	Mixed easy and hard traffic	English-optimized. Same-family models only

Start with prompt caching. The launch figures are AWS maximums, so measure your own.

Exact vs semantic caching

Why not cache answers too? An exact cache, keyed on a hash of the normalized prompt, is safe but rarely hits on free text. A semantic cache answers when a past question's embedding is close. ElastiCache for Valkey 8.2 on node-based clusters has offered vector search since October 2025 at no extra charge, with up to 99% recall per AWS.

The risk is a confident wrong hit, since "cancel my order" and "cancel my subscription" sit close in embedding space. Scope keys by tenant and intent, use a strict threshold and short TTLs, and never semantically cache user-specific answers.

flowchart LR
    Q["New question<br/>cancel my subscription"] --> E["Embed it and search<br/>past questions"]
    E --> N["Closest past question<br/>cancel my order"]
    N -->|"loose threshold, unscoped keys"| W["Confident wrong hit<br/>returns the order answer"]
    N -->|"strict threshold, keys scoped by tenant and intent"| M["Treated as a miss<br/>the model answers"]

Two questions can sit close in embedding space and still need different answers, so the threshold and key scope decide what counts as a hit.

Retrieval latency

Retrieval lands in TTFT. In AWS's September 2026 benchmarks, OpenSearch measured p50 31 ms and p99 52 ms (1.2M vectors), Aurora PostgreSQL with pgvector about 32 ms and 89–94 ms, and S3 Vectors 294 ms and 506 ms (1M vectors). The workloads differed, so compare shape, not ranking (AWS ML Blog, 2026). S3 Vectors holds 2 billion vectors per index at up to 90% lower cost (GA December 2025). It's a good full-corpus store behind a faster hot tier, and a poor fit for a 1-second TTFT budget.

Agent loops multiply everything

Plan, three sequential tool calls, answer: at least five round-trips, each paying TTFT again. Parallelize independent tools, cap iterations and trace each step. CloudWatch generative AI observability (GA October 2025) traces model calls through Amazon Bedrock AgentCore operations.

Measure before you tune

This prints all three budgets and the prompt cache result.

Checked against boto3 converse_stream and Bedrock prompt caching on Oct 28, 2026. Not deployed, so try it in a sandbox account first.

# ttft.py — TTFT, output tokens/sec and end-to-end latency for streamed Bedrock calls
import os, time, boto3

client = boto3.client("bedrock-runtime")
MODEL_ID = os.environ["MODEL_ID"]                  # a model or inference profile you can invoke
SYSTEM_PROMPT = open("system_prompt.txt").read()   # long, stable prefix: eligible for caching

def measure(question):
    start, first = time.perf_counter(), None
    usage = {}
    resp = client.converse_stream(
        modelId=MODEL_ID,
        system=[{"text": SYSTEM_PROMPT}, {"cachePoint": {"type": "default"}}],
        messages=[{"role": "user", "content": [{"text": question}]}],
        inferenceConfig={"maxTokens": 400},        # an output cap is also a latency cap
        serviceTier={"type": os.environ.get("SERVICE_TIER", "default")},  # or "priority"
    )
    for event in resp["stream"]:
        if "contentBlockDelta" in event and first is None:
            first = time.perf_counter()
        if "metadata" in event:
            usage = event["metadata"]["usage"]
    end = time.perf_counter()
    first = first or end
    print({
        "ttft_ms": round((first - start) * 1000),
        "e2e_ms": round((end - start) * 1000),
        "otps": round(usage.get("outputTokens", 0) / max(end - first, 1e-6), 1),
        "cache_read": usage.get("cacheReadInputTokens", 0),
        "cache_write": usage.get("cacheWriteInputTokens", 0),
    })

for q in ["What is the refund window?", "Do you ship to Ghana?"]:
    measure(q)  # the second call should report cache_read > 0 if the prefix met the minimum

If Bedrock's TimeToFirstToken is low but your client-side TTFT is high, look at your own path.

AI as an engineering aid

AI tools help with the tedious parts: turning a trace into a hypothesis, drafting a k6 scenario, explaining a flame graph. CloudWatch investigations (GA June 2025, no additional cost) proposes root-cause hypotheses from your metrics, logs and traces.

They also mislead predictably: caches with no invalidation story, closed-model load tests, numbers from nowhere. Three rules help. Every proposed optimization ships with a before-and-after measurement from your environment. Generated load tests use arrival-rate executors and thresholds. Any number an assistant offers needs a link before it reaches a design doc.

Who owns what

Performance slips when nobody owns it. The product owner is accountable for latency budgets per journey, which the architect writes. Developers own p95/p99 and TTFT instrumentation. SRE or platform is accountable for CI load tests (developers write them) and owns alarms and regression response.

The AWS line moves with compute. On EC2 it handles hardware and hypervisor, and you own instance type, OS tuning, scaling and connection pools. ECS on Fargate takes hosts and OS off you, leaving task sizing, scaling and app code. Lambda runs environments and scaling mechanics, and you own memory, init code and concurrency settings. Bedrock handles model serving and tier capacity, and you own model choice, prompt and output length, streaming, caching and quotas.

Put simply, the provider owns how fast a model generates. You own how many tokens you send and request, and how often you call.

Your Monday checklist

[ ] Write one budget sentence (percentile, endpoint, load) for your most important user journey.
[ ] Replace every average-latency alarm with p95 and p99 alarms.
[ ] Check LCP, INP and CLS at p75 on mobile from field data.
[ ] Remove every header, cookie and query string from CloudFront cache keys that the origin doesn't use.
[ ] Compare Init Duration in Lambda logs with your p99, and run Lambda Power Tuning on your busiest function.
[ ] Add a k6 constant-arrival-rate test with p(95) thresholds to your pipeline.
[ ] Chart TimeToFirstToken and OTPS for every LLM feature, and confirm responses stream.
[ ] Move stable instructions to the front of prompts, add a cachePoint, and check cacheReadInputTokens.

Conclusion

Comparisons need a ruler. A cache, tier or runtime option is only better against a budget, at a percentile, under a load. With one, the options here become decisions. DAX when DynamoDB reads repeat. Valkey when results are computed. SnapStart when INIT owns your tail. Streaming and prompt caching when an LLM is the slowest hop. Without it, you're tuning by feel.

Vodafone's faster page was still slow, and it sold 8% more. You don't need to be fast in the abstract, just measurably faster where your user is waiting.

Pick the percentile first. The architecture follows from it.

Series: Non-functional Application Requirements

Non-functional Application Requirements: An Introduction
Non-functional Application Requirements: Security
Non-functional Application Requirements: Compliance
Accessibility
Performance
Availability (coming soon)
Scalability (coming soon)
Maintainability (coming soon)
Cost (coming soon)
Sustainability (coming soon)

Cover image: AWS re:Invent 2023 – Keynote with Dr. Werner Vogels

Next up, Availability: a fast system that's down is infinitely slow, so we'll look at composite SLOs, static stability, and what happens when your model provider throttles.

DEV Community