DEV Community: Carlos Saldaña

Debugging PostgreSQL Performance

Carlos Saldaña — Tue, 02 Jun 2026 15:11:22 +0000

Let's chat about performance, optimization, and debugging a PostgreSQL database. Can you read EXPLAIN ANALYZE output fluently? Do you know how to find which tenant is causing a slow query, how to use pg_stat_statements, how to spot lock contention? If you have a hard time answering those, don't worry. Today we're going to cover all of them and practice how to get better at it.

The goal here: you should be able to look at a slow API endpoint and trace it down to the exact query, the exact tenant, the exact missing index, in under 10 minutes. That's the bar.

EXPLAIN ANALYZE: read the plan, not the vibes

Let's start with EXPLAIN ANALYZE because everything else builds on it.

So why does this exist?

The query planner makes a cost-based guess about the best way to execute your SQL query. EXPLAIN ANALYZE actually runs the query and shows you whether that guess was correct.

Always use this form:

EXPLAIN (ANALYZE, BUFFERS) SELECT ...;

ANALYZE → Runs the query and reports what actually happened.

BUFFERS → Shows cache hits vs. disk reads. Without it, performance analysis is incomplete.

⚠️ ANALYZE executes the query for real, including writes. Use BEGIN and ROLLBACK to test without committing changes.

The single most important skill: estimated vs actual
Every node prints two row counts. When rows (estimate) and actual rows diverge by more than ~10x, the planner is working from bad information.

How to read the tree

The plan is a tree. Read the most-indented node first. That's where execution starts. Costs and times are cumulative: a parent node's time includes its children.

actual time=X..Y → X is time to first row, Y is time to last row, per loop. If loops=5000, the real total for that node is Y × 5000. This is how a "0.05 ms" node secretly costs 4 seconds.

The best resource if you're starting out is explain.depesz.com. You can paste any EXPLAIN output and it visualizes the slow nodes. Use it on your own queries until you can predict what it'll say before you paste.

Index Design

I'm not going to go deep into index design here. Instead, read Use The Index, Luke! by Markus Winand.

It's free, easy to read, and probably the best resource available on SQL indexes. Many senior engineers use indexes every day, but haven't taken the time to read it from start to finish.

If you read one resource on indexing, make it this one.

If you've ever thought indexes were some kind of database magic, you're not alone. Most of the confusion disappears once you realize they're just a clever way of keeping data sorted.

The key idea is that a B-tree index is a sorted copy of one or more columns, plus pointers to the corresponding table rows.

The fact that it's sorted is what gives it its power. PostgreSQL can quickly find the rows it needs instead of reading the entire table.

The rule that matters most: leftmost prefix

An index on (tenant_id, created_at, status) is sorted by tenant_id, then within
each tenant by created_at, then by status. So it can serve:

✅ WHERE tenant_id = ?
✅ WHERE tenant_id = ? AND created_at > ?
✅ WHERE tenant_id = ? AND created_at > ? AND status = ?
✅ ORDER BY created_at      (within a fixed tenant_id)
❌ WHERE created_at > ?     (skips the leading column, can't seek)
❌ WHERE status = ?         (skips two leading columns)

You can only use a contiguous prefix, starting from the left. This is why column
order in a composite index is a design decision, not an afterthought.

Things that quietly break index usage

Using a function on the indexed column

WHERE lower(email) = ? can't use a regular index on email.

Fix: create an expression index:

CREATE INDEX ON users (lower(email));
Modifying the column in the filter

WHERE created_at + interval '1 day' > now() makes it harder to use the index.

Instead, apply the calculation to the constant side of the comparison.
Leading wildcards

LIKE '%foo' can't efficiently use a B-tree index.

LIKE 'foo%' can.

For substring searches, use a trigram index (pg_trgm).
Type mismatches

Implicit casts on the column side can prevent index usage. Make sure the types match.
Low-selectivity columns

An index on a boolean column or a status column with only a few possible values is often not useful by itself. PostgreSQL may choose a sequential scan instead. Consider a composite index or a partial index.

Index features worth knowing

Covering indexes (INCLUDE)

Add extra columns to the index so PostgreSQL can answer the query without reading the table. This can enable an Index Only Scan, which is often faster.
```
CREATE INDEX ON orders (tenant_id, created_at)
INCLUDE (total, status);
```
Partial indexes

Index only the rows you actually query. The index is smaller and cheaper to maintain. Often faster, too.
```
CREATE INDEX ON jobs (created_at)
WHERE status = 'pending';
```
B-tree is the one you'll use 95% of the time. Learn it first. The others exist for specialized data types and query patterns.

The trade-off: indexes aren't free

Indexes make reads faster, but they make writes more expensive.

Every INSERT, UPDATE, and DELETE must update the table and all of its indexes. More indexes mean more disk, more memory, more work on every write.

Regularly check for unused indexes (pg_stat_user_indexes.idx_scan = 0) and remove them when appropriate.

The goal isn't to have more indexes. The goal is to have the fewest indexes needed to support your important queries.

`pg_stat_statements`: where the server actually spends its life

When someone tells me "the database is slow," this is usually the first place I look. Most performance problems aren't caused by one terrible query. They're caused by a handful of queries quietly consuming time all day long.

A production application can execute thousands of different queries. You can't run EXPLAIN ANALYZE on all of them.

pg_stat_statements solves this problem by collecting execution statistics for every normalized query. Queries that differ only by parameter values are grouped together, making it easy to see where the database is spending its time.

To enable it, add the following to postgresql.conf:

shared_preload_libraries = 'pg_stat_statements'
pg_stat_statements.track = all

Restart PostgreSQL, then create the extension:

CREATE EXTENSION pg_stat_statements;

Learn these columns:

calls → How many times the query ran.
total_exec_time → Total time spent executing the query.
mean_exec_time → Average execution time per call.
rows → Total rows returned or affected.
shared_blks_hit vs shared_blks_read → Cache hits vs disk reads.

The most useful question you can ask in production is:

Where is my database spending most of its time?

This query answers it:

SELECT
  substring(query, 1, 80) AS query,
  calls,
  round(total_exec_time) AS total_ms,
  round(mean_exec_time, 2) AS mean_ms,
  rows,
  round(
    100.0 * shared_blks_hit /
    nullif(shared_blks_hit + shared_blks_read, 0),
    1
  ) AS hit_pct
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 20;

This tells you:

which queries are globally expensive,
slowest average queries,
most frequently executed queries.

This is often where real tuning starts in production.

Sort by total time, not average time

Many engineers focus on mean_exec_time, but total_exec_time is usually the more important metric.

A query that takes 3 ms and runs 2 million times per hour can consume far more resources than a report that takes 2 seconds and runs twice a day.

Think about it this way:

Sort by total execution time to find where the server is spending most of its effort.
Sort by average execution time to find individually slow queries.

Both views are useful, but total_exec_time is where you'll usually find the biggest wins.

Resetting the stats

When you want to start a fresh measurement window:

SELECT pg_stat_statements_reset();

Running the "Top 20 by total execution time" report every week is one of the simplest ways to catch performance problems before they become outages.

`auto_explain`: catch the slow query you can't reproduce

pg_stat_statements tells you which queries are slow overall. What it can't tell you is why a query was slow at 2 AM during an incident.

Execution plans can change as data grows, statistics become outdated, or different parameter values are used. A query that's normally fast can suddenly take a very different path.

That's where auto_explain helps.

It automatically logs the full execution plan for queries that exceed a time threshold, capturing the plan that was actually used when the slowdown happened.

Configure it in postgresql.conf:

session_preload_libraries = 'auto_explain'

auto_explain.log_min_duration = '500ms'
auto_explain.log_analyze = on
auto_explain.log_buffers = on
auto_explain.log_nested_statements = on
auto_explain.log_format = 'text'

Use it carefully in production

log_analyze = on measures actual execution times, which adds overhead to every logged query.

A safe approach is:

Start with a relatively high threshold (for example, 500 ms or 1 second).
Leave log_analyze off initially.
Enable more detailed logging only when investigating a performance issue.

Why it matters

Think of the tools as solving different problems:

pg_stat_statements answers: "Which queries are costing us the most?"
auto_explain answers: "What plan did PostgreSQL actually use when this query became slow?"

Together they form one of the most valuable performance debugging tool sets in PostgreSQL.

pg_stat_statements helps you find the problem.

auto_explain helps you understand it.

When you have both, performance investigations become much less about guessing and much more about reading the evidence.

`pg_locks`: who is blocking whom

When a query suddenly stops making progress, the problem is often not the query itself. It's waiting for a lock.

pg_locks shows one row for every lock currently held or being waited on by PostgreSQL sessions.

The most important column is:

granted = true → the lock has been acquired.
granted = false → the session is waiting for a lock held by someone else.

Joining it with pg_stat_activity gives you the lock plus the SQL behind it:

SELECT
  l.pid,
  l.locktype,
  l.mode,
  l.granted,
  left(a.query, 60) AS query
FROM pg_locks l
JOIN pg_stat_activity a ON a.pid = l.pid
ORDER BY l.granted, l.pid;

A waiting query is often a symptom of lock contention somewhere else in the system.

Deadlocks: not a database failure, a concurrency bug

A deadlock happens when two transactions are waiting on each other:

Transaction A holds lock 1 and wants lock 2.
Transaction B holds lock 2 and wants lock 1.

Neither transaction can continue, creating a cycle.

PostgreSQL detects this situation automatically (after deadlock_timeout, which defaults to 1 second) and resolves it by aborting one of the transactions with a deadlock detected error.

Your application should be prepared to catch this error and retry the transaction.

One lesson that's easy to forget: a deadlock isn't a database failure. It's a concurrency bug. PostgreSQL is doing exactly what it should, so the system can move forward.

For lock contention specifically, the canonical resource is the Postgres wiki page "Lock Monitoring." Search for it, bookmark it; the query at the bottom that shows blocking chains is one you'll run many times in your career.

Use it with `pg_stat_activity`

On its own, pg_locks is hard to read: raw pids and lock modes, no context.

That's why the query above joins it with pg_stat_activity. Together they show you:

Which queries are currently waiting.
Which sessions are blocking them.
How long they've been waiting.
The SQL being executed by both sides.

When debugging database issues, think of these views as a pair:

pg_stat_activity → What is everyone doing?
pg_locks → Who is blocking whom?

Together they help you diagnose lock contention, blocked queries, and deadlocks before they turn into production incidents.

Once you've worked through all of this, the natural next step is MVCC and VACUUM.

That's where you'll learn why idle in transaction causes bloat, why dead rows affect query performance, and why a perfectly reasonable index can still be slow.

In my experience, MVCC is the point where PostgreSQL starts to make sense as a system instead of a collection of features. Many performance problems that look mysterious become obvious once you understand how PostgreSQL stores and cleans up row versions.

A prompt is not a conversation. It's a component contract.

Carlos Saldaña — Mon, 25 May 2026 21:48:25 +0000

Most of us use LLMs by trial and error. This post gives you a structure: the building blocks of an LLM, and a reusable template for writing production prompts.

What is an LLM?

Foundation models are very large models pretrained on internet-data; that's what builds Generative AI. With a foundation model, you can adapt one pretrained model to many tasks.

A Large Language Model (LLM) is a foundation model for text, and at its core, the same FM can be used for many tasks: summarisation, classification, translation, code generation.

So what an LLM does is predict the next word (next token) in a sequence. At each step it checks the surrounding context of what it has seen so far and then produces a probability distribution over the possible next tokens. Running this in a loop produces fluent, coherent new content. So basically, an LLM gets text and returns text. The input is called the prompt and the output is called completion.

What goes into a prompt

A prompt is any input given to a generative model to produce a desired output. Prompt engineering is the practice of designing and refining those prompts to get the best possible results from the model. Refining a prompt means experimenting with the factors that influence the model's output. A vague prompt can lead to many reasonable responses; every constraint you add reduces the number of possible responses.

A well-structured prompt is usually assembled from four parts.

Building block	Role
Instruction	The task you want performed.
Context	Relevant background that frames the situation for the model.
Input data	The specific content the task should operate on.
Output indicator	A description (or example) of the form the response should take.

The instruction and context here are two of the template slots, Task and Context, we'll pull together at the end.

And best practice for writing prompts can be organised into four dimensions. Strong prompts attend to all four.

Dimension	Practice
Clarity	Use simple, direct language. Avoid ambiguous or overly complex terminology so the prompt is easily understood.
Context	Provide relevant background and specific details to guide the model's understanding of the situation.
Precision	Clearly state the type of response you want, and use examples to illustrate the expected output.
Role-play / Persona	Write the prompt from the perspective of a specific character or expert, with enough detail for the model to assume that role effectively.

Think of a prompt like a search beam. Vague = wide beam, the model lands somewhere in a large valid region. Each constraint narrows the beam. Specificity isn't politeness toward the model; it's aiming.

Who reads the output: code or a person

The output is the more interesting part for us. An LLM's output has two possible audiences, a parser and a person, and you write a contract for each. Even when a person reads the output, "looks fine" isn't the same as "matches what I needed." When an LLM's output is read by code instead of eyes, the output is an API response and the prompt is its schema. A human forgives a messy answer; json.loads() doesn't. It either succeeds or throws. Without an explicit spec, the model decides format, length, tone, and depth, and it picks something plausible but generic. Controlling output here means moving that decision from the model to you.

Two kinds of control:

Format control: the shape of the output (JSON keys and types; or prose vs. bullets vs. table, headings, sections).
Behavior control: what the model may and may not do (e.g. "valid Terraform only, no comments"; or "concise executive tone, no technical jargon").

Each audience fails differently, and one of them fails silently:

	Parser (output read by code)	Person (output read by a human)
What breaks	Markdown fences, chatty preamble, invented/renamed keys, numbers as strings	Too long, wrong tone, missing a section, pitched at the wrong level
How it breaks	Loud: `json.loads()` throws, the pipeline stops; you notice immediately	Quiet: output looks fluent and complete; the gap only shows on a second read, and nobody flags it

A loud failure is annoying; a quiet failure is more dangerous because a subtly off-target answer can go unnoticed.

There are some mitigations you can apply for better results:

For the parser:

Specify the exact schema.
Forbid the noise.
Give one example of the exact shape.
Set temperature: 0.
Use native structured outputs / tool-calling.

For the person:

State the structure explicitly.
Give a length budget.
Name the audience.
List the required elements.
For a recurring format, show one example of the desired output.

So the point here is that prompt-only format control is a request; decode-time constraints are a guarantee. Treat the model's output as an API response, and the prompt as its schema.

Constraints and Output are template slots too: the rules that prune behavior, and the exact shape you contract for.

The system and user split

We already saw the role as one of the four dimensions of an effective prompt. It's the part lots of people forget about.

A prompt can include three types of messages: system, user, and assistant.

System message: defines the model's behavior, rules, and overall role.
User message: contains the request or input for the current task.
Assistant message: previous responses from the model, used as conversational context or examples.

The "type" is usually set through the role field of each message.

The system message is the component's configuration, while the user message is the input for a specific call.

The system prompt defines persistent behavior and rules. The user message provides the variable data for that particular request. The system sets how the component behaves in general; the user message tells it what to do right now.

Why do roles work? It's not magic.

When you say, "Act as a senior security engineer," the model shifts its output toward patterns statistically associated with that kind of writing in its training data.

Likewise, "Explain this to a junior developer" pushes the model toward simpler, more educational, and more heavily explained responses.

A role doesn't give the model a real personality. It changes the probability distribution of the kind of text the model is likely to generate. This is the Role slot, the first line of the template.

Why the system/user split matters in production:

Caching. The system message is usually stable and reused across requests, which makes it ideal for prompt caching. Keep the system prompt consistent, and place most of the changing data in the user message.
Testability. The system prompt is one of the highest-leverage parts of an AI application. Treat it like code: version it, compare changes, and test it carefully.
Security. Trusted instructions should live in the system message. Untrusted content (user input, retrieved documents, tool outputs) should stay in the user message. When untrusted text lands somewhere the model treats as instructions, you get prompt injection. Clean separation is the first line of defense.

Showing the model examples

One good prompt gets you far. A few techniques get you further. The most useful one: showing the model examples.

In few-shot prompting, the prompt includes a few worked demonstrations. The model uses in-context learning to infer the pattern and apply it to the new input.

The few-shot examples are unit tests that double as a spec.

Why it works: the model is a pattern continuator. A few input→output pairs establish a strong, low-ambiguity pattern, and the model continues it.

When to use which:

Zero-shot: simple, common tasks the model has clearly seen many times.

[
  {
    "role": "user",
    "content": "Summarize this article in 3 bullet points."
  }
]

One-shot: when you mainly need to pin the format.

[
  {
    "role": "user",
    "content": "Convert countries to JSON.\n\nExample:\nFrance -> {\"country\": \"France\"}\n\nNow convert:\nBrazil"
  }
]

Few-shot: classification, structured output, code style, anything with a specific schema or edge cases the model wouldn't guess.

Those worked demonstrations are the Examples slot.

In the example below, the last message is the new input; the model produces the assistant turn.

[
   {"role": "user",      "content": "Today the weather is fantastic"},
   {"role": "assistant", "content": "positive"},
   {"role": "user",      "content": "I don't like your attitude"},
   {"role": "assistant", "content": "negative"},
   {"role": "user",      "content": "That shot selection was awful"},
]

Tokens are what you pay for

LLMs are not free, and the token is the meter. For an LLM, it's the unit of both cost and latency. For most of us the mental model is familiar: tokens are network payload, and each LLM call is a billable, latency-bearing API request.

Output tokens dominate latency. The model generates text one token at a time, and each new token depends on all the previous ones, so generation has to happen sequentially.

Input works differently: the prompt is processed in a single parallel "prefill" pass, which is relatively fast (even though you still pay for those tokens).

Four levers do most of the work:

Compress the input

Instead of sending entire documents, summarize them first or extract only the relevant parts. Most prompts ship context the model never reads.
Limit and structure the output

Set max_tokens, prefer structured formats like JSON or arrays instead of long prose, and ask for concise summaries such as "keep under 120 tokens."
Use the right model for the task

Small models are faster and cheaper for simple work like classification, routing, or extraction. Save the stronger models for tasks that actually require reasoning or synthesis.
Use caching

There are two different kinds:

Prompt/prefix caching

Reuses stable prompt sections like system prompts, examples, or reference documents. Since the provider caches this server-side, you avoid recomputing the expensive input processing step.

Practical implication: put stable content first and variable content last to maximize cache hits.

Response/semantic caching

Your own infrastructure stores previous answers and reuses them when the same or a very similar request appears again. This caches outputs, not prompts.

A Reusable Prompt Template

All that we've seen here gives us the structure to build a template to use when working on a production prompt.

The template R-T-C-C-E-O:

[ROLE]        Who the model acts as. Sets the output distribution.
[TASK]        The one thing to do, stated unambiguously.
[CONTEXT]     Inputs, background, data; clearly delimited from instructions.
[CONSTRAINTS] Rules that prune the space. Each maps to a real failure mode.
[EXAMPLES]    1–3 representative input→output pairs (for structured/edge-case tasks).
[OUTPUT]      The exact shape of the response. Schema + example if machine-consumed.

Not every prompt needs all six, but every prompt should be a deliberate subset, not an accident.

Finally, let's make the production checklist, a pre-flight pass before a prompt ships:

Role set and matched to the task?
Task: could a competent stranger misread it?
Constraints: does each map to a failure mode you've actually seen? Drop the decorative ones.
Output contract: explicit; schema + example if consumed by code?
Format guarantee: decode-time constraint (structured output / tool call), not just a worded request?
Examples: present for classification/structured work; diverse, consistent, minimal?
Sampling: temperature matched to the task?
Token budget: max_tokens capped; output format compact?
Model: right-sized, not just "the strong one"?
Cache-friendliness: stable content first, variable content last?
Injection safety: instructions in the system message, data in the user message?
Versioned & tested: in source control with a few regression cases?

So we covered six slots: Role, Task, Context, Constraints, Examples, Output, plus a pre-flight checklist. Run every production prompt through both, and you've turned a hopeful request into a tested component. A prompt is not a conversation. It's a component contract.

The Protocol Behind Python's Containers

Carlos Saldaña — Tue, 19 May 2026 14:35:05 +0000

Lists, tuples, dicts, and sets look like four different things. Underneath, they share the same idea: they all implement a small set of protocols, and once you see that, the rest stops feeling arbitrary.

What is a protocol?

A protocol in Python is an informal contract defined by behavior, not by declaration. Anything that behaves like a sequence is a sequence, as far as Python cares. This style of typing has a name, it’s called structural typing, or as most of us know it, duck typing.("if it walks like a duck...").

Think of this like the USB port. A USB port doesn't ask "are you a keyboard?" or "are you a hard drive?" It exposes a physical shape and a data protocol. Anything that fits the shape and speaks the protocol works.

Iterators and Iterables

If you only learn one thing about Python containers, learn the difference between an iterable and an iterator.

An iterable is an object that we can iterate over, it's an object that can produce an iterator. To make this happen, it implements __iter__(), which returns a fresh iterator each time it's called.
An iterator is an object that produces values one at a time. It implements __next__() (and, by convention, __iter__() returning self). It raises StopIteration when there are no more values.

There are a few built-in iterables:

List
Tuples
Strings
Dictionaries

Let’s try an example to understand why a list is iterable but not an iterator:

nums = [1,2,3]
next(nums)
TypeError: 'list' object is not an iterator
it = iter(nums)
next(it)
1
next(it)
2

The for loop is doing this exact dance. for i in nums: it’s roughly doing:

_it = iter(nums)
while True:
    try:
        x = next(_it)
    except StopIteration:
        break

That’s the whole protocol. Once you understand this, things like for loops, generators, zip, map, comprehensions, unpacking, and *args all start to work the same way in your mind.

I'm more visual, so here's a sequence diagram showing the interaction over time between two objects.

Built-in Containers

This is the good part. Most of us learn Python's containers at the surface: lists are mutable and ordered, tuples are immutable and ordered, dicts are key-value pairs, sets are unique and unordered. That's correct but also useless most of the time. It tells you nothing about performance. Nothing about why your set of dicts blew up with a TypeError. Nothing about why your queue is suddenly O(n²).

Here's what to focus on instead. Every container in Python is three things at once:

A data structure: how it's stored in memory and how fast operations are.
A set of protocols: the special methods it supports.
A semantic choice: the meaning you communicate to other developers by using it.

List: the dynamic array

A python list is a dynamic array (called a vector or ArrayList in other languages).

Under the hood, it’s a block of pointers to python objects, with extra space preallocated at the end for appending new items.

┌────┬────┬────┬────┬────┬────┬────┬────┐
│ p0 │ p1 │ p2 │ p3 │ ▒▒ │ ▒▒ │ ▒▒ │ ▒▒ │   length=4, capacity=8
└────┴────┴────┴────┴────┴────┴────┴────┘
  ↑    ↑    ↑    ↑
  obj  obj  obj  obj

This one detail is what determines the entire complexity table.

Operation	Cost
`lst[i]`	O(1)
`lst.append(x)`	O(1) amortized
`lst.pop()`	O(1)
`lst.pop(0)`	O(n)
`lst.insert(0, x)`	O(n)
`x in lst`	O(n)
`len(lst)`	O(1)

Protocol: MutableSequence (which gives you Sequence + Iterable + Container + Sized + Reversible).

Semantic intent: An ordered, mutable collection of items that I expect to iterate over, append to, and access by index.

A common mistake is to use a list as a FIFO queue:

pending = []
pending.append(task)
task = pending.pop(0)

This is pretty expensive, the pop(0) shifts every remaining element one position to the left, and remember this operation is O(n). If this runs inside a loop it can accidentally become O(n²) .

Tuple: the record

We think of tuples as immutable lists, but that idea can mislead.

A tuple is better understood as a record: a fixed-size structure where each position can hold different types of data.

In practice:

Tuples represent structured data.
Lists represent collections of items.

# List
orders = [order1, order2, order3, ...]

# Tuple
db_row = ("alice", "active", 42)

Immutability is a result of this distinction, not the point.

And because tuples are immutable, they gain a few important properties that lists don’t have:

Hashable
Memory efficiency
Read-only contract

From a protocol perspective, tuples behave like read-only sequences and can also be hashable.

Semantically, a tuple means: “This is a single thing identified by its contents.”

This matters a lot in a production application, tuples are commonly used for:

Compound dictionary keys
Deduplication
Cache and memoization keys

Dict: the hash table

The king. If lists are the most common container in Python, dictionaries are probably the most important. A big part of Python's internals is built on top of dictionaries.

A dict is a hash table with open addressing. The keys you insert get hashed, and the hash points to a slot.

Operation	Average	Worst
`d[k]`, `d[k] = v`, `del d[k]`	O(1)	O(n) (hash collisions)
`k in d`	O(1)	O(n)
Iteration	O(n)	O(n)

From a protocol perspective, a dict implements MutableMapping, not Sequence.

There are two things you must remember about dictionaries:

Keys must be hashable
Insertion order is preserved

Set: the dict without values

A set is almost the same thing as a dictionary, except it only cares about keys.

It uses the same hash-table implementation underneath, which means it has:

O(1) average-time lookups, inserts, and removals
elements must be hashable
same "no duplicates" guarantee, duplicates are naturally eliminated

seen = set()
seen.add(user_id)
if user_id in seen:
        ...

# the antipattern
seen = []
if user_id in seen:
        ...
seen.append(user_id)

So in practice it's just a collection of unique keys. From a protocol perspective, a set implements MutableSet, which gives you built-in set operations like: |, &, -, ^

Semantically, a set means: “I only care about uniqueness and fast membership checks.”

Conclusion

These are some of the most commonly used containers in Python in our applications, and it's important to understand the trade-offs when deciding which one to use. When you reach for a container, try to ask these:

What's the access pattern?
Will it grow?
Does identity-by-contents matter?
What does the choice say to the reader?

These are the common bad choices we covered and what you should reach for instead:

You wrote	You probably wanted
`list.pop(0)` in a loop	`collections.deque`
`if x in big_list` repeatedly	`set` (convert once)
`f"{a}:{b}"` as a dict key	`(a, b)` tuple key
List of `(key, value)` pairs	`dict`
`dict[str, None]` for membership	`set`
Tuple with named indexes everywhere	`namedtuple` or `dataclass`

If you remember one thing: the container you pick is a contract, not a convenience.