DEV Community: bertrand HARTWIG

PostgreSQL: Which Queries Should You Optimize First?

bertrand HARTWIG — Thu, 14 May 2026 08:08:52 +0000

When investigating PostgreSQL performance, the usual starting point is pg_stat_statements. From there, many teams sort queries by mean_exec_time or total_exec_time and start optimizing the first rows in the list.

That approach is simple, but it often leads to the wrong priorities.

A query that takes five seconds but runs twice a day is not necessarily more important than a query that takes five milliseconds and runs millions of times. Conversely, a query with a high total execution time may simply be a normal core workload query, not necessarily the best optimization target.

The real question is not:

Which query is the slowest?

It is:

Which query has the highest operational impact and the clearest optimization potential?

This is the principle behind the query-ranking algorithm implemented in pgAssistant.

Read the full post here

Indexing every WHERE column is not PostgreSQL optimization

bertrand HARTWIG — Mon, 11 May 2026 15:20:15 +0000

One PostgreSQL indexing mistake I see often:

“The query filters on A, B and C, so let’s create an index on A, B, C.”

That may work, but it may also be the wrong index.

For composite B-tree indexes, PostgreSQL cares about predicate type, column order, selectivity, table size, and the actual execution plan.

In this post, I explain why equality predicates usually belong before range predicates, why n_distinct from statistics matters, and why a theoretically good index is useless if the planner never uses it.

I also show how pgAssistant turns this into an automated index recommendation workflow using EXPLAIN ANALYZE and planner statistics.

Full write-up:
https://beh74.github.io/pgassistant-blog/post/query_advisor/

Designing the Right PostgreSQL Index Using Query Plans and Statistics

bertrand HARTWIG — Mon, 11 May 2026 06:04:15 +0000

PostgreSQL index design is often misunderstood.

Many developers think that creating a good index simply means:

“Create an index containing the columns from the WHERE clause.”

In reality, efficient index design is far more nuanced.

The order of columns inside a composite index matters enormously, and the best choice depends on:

Predicate types (=, >=, BETWEEN, LIKE, etc.)
Column selectivity
Table size
PostgreSQL planner statistics
Actual execution plans

This article explains the core principles behind efficient PostgreSQL index design before showing how pgAssistant automates this process using execution plans and database statistics.

Why Index Design Is Difficult

Consider the following query:

SELECT
    order_id,
    customer_id,
    employee_id,
    order_date,
    ship_country
FROM public.orders
WHERE customer_id = $1
  AND employee_id = $2
  AND order_date >= DATE $3;

At first glance, several index definitions may appear reasonable:

(customer_id, employee_id, order_date)

(order_date, customer_id, employee_id)

(employee_id, customer_id, order_date)

But these indexes do not behave the same way.

Choosing the correct ordering requires understanding how PostgreSQL traverses B-Tree indexes.

The Fundamental Rule of B-Tree Indexes

For PostgreSQL B-Tree indexes:

Equality predicates should come first
Range predicates should come last

This is the single most important rule in multi-column index design.

Equality Predicates Are Highly Selective

Predicates such as:

=
IN (...)
IS NULL

allow PostgreSQL to navigate directly to a very precise section of the index tree.

In our query:

customer_id = $1
employee_id = $2

are equality predicates.

If the index begins with these columns:

(customer_id, employee_id, ...)

PostgreSQL can rapidly narrow the search space.

Conceptually:

customer_id = exact branch
employee_id = exact sub-branch

The planner can jump almost directly to the matching rows.

Why Range Predicates Should Come Last

Now consider:

order_date >= DATE $3

This is a range predicate.

Once PostgreSQL enters a range scan inside a B-Tree index, the remaining columns become far less useful for navigation.

For example, with this index:

(order_date, customer_id, employee_id)

the planner must first scan all matching dates:

order_date >= DATE $3

which may represent a very large portion of the table.

Only afterward can additional filtering occur.

This usually produces significantly more index scanning.

That is why range predicates are generally placed at the end of composite indexes.

Column Order Among Equality Predicates

Once equality predicates are identified, the next challenge is:

Which equality column should come first?

The answer depends on selectivity.

PostgreSQL exposes this information through planner statistics.

PostgreSQL Statistics Drive Good Index Design

For our query, PostgreSQL statistics are:

order_date [>=]:
    n_distinct=814
    null_frac=0.0000
    mcv_count=100
    histogram_bounds=101

customer_id [=]:
    n_distinct=89
    null_frac=0.0000
    mcv_count=89
    histogram_bounds=0

employee_id [=]:
    n_distinct=9
    null_frac=0.0000
    mcv_count=9
    histogram_bounds=0

The most important metric here is:

n_distinct

Understanding `n_distinct`

n_distinct estimates the number of distinct values in a column.

Higher n_distinct usually means:

higher selectivity
fewer matching rows
better filtering efficiency

In our example:

Column	n_distinct
customer_id	89
employee_id	9

customer_id is significantly more selective.

Therefore, PostgreSQL benefits more from filtering by customer_id first.

Why Selectivity Matters

Imagine the table contains 1 million rows.

Filtering by:

employee_id

may still leave:

1,000,000 / 9 ≈ 111,111 rows

Filtering by:

customer_id

may reduce the result to:

1,000,000 / 89 ≈ 11,236 rows

Starting with the most selective equality predicate drastically reduces the search space.

This improves:

index scan efficiency
cache locality
heap access reduction
execution time

The Correct Index Design

Applying these principles:

Equality predicates first
Most selective equality columns first
Range predicates last

produces:

CREATE INDEX CONCURRENTLY
    pga_idx_orders_customer_id_employee_id_order_date
ON public.orders
    (customer_id, employee_id, order_date);

This ordering allows PostgreSQL to:

Navigate efficiently using exact matches
Reduce scanned rows as early as possible
Apply the range scan only after narrowing the search space

Good Index Design Also Depends on Table Size

One of the biggest misconceptions about PostgreSQL optimization is:

“Indexes are always faster.”

This is false.

For small tables, PostgreSQL often prefers a Sequential Scan (Seq Scan) even when an index exists.

Why?

Because using an index has overhead:

traversing the B-Tree
reading index pages
performing heap lookups
random I/O access

For sufficiently small tables, scanning the entire table sequentially is cheaper.

Query Plans Matter More Than Theory

A theoretically perfect index is useless if PostgreSQL never uses it.

That is why index recommendation engines should never rely only on SQL syntax.

They must also inspect:

execution plans
estimated costs
table statistics
row estimates
planner decisions

The Importance of Execution Plans

The execution plan reveals how PostgreSQL actually executes a query.

For example:

EXPLAIN ANALYZE
SELECT ...

may show:

Seq Scan on orders

or:

Index Scan using ...

This distinction is critical.

A query may contain filter predicates that look index-friendly, but PostgreSQL may correctly determine that:

the table is too small
selectivity is too low
too many rows would still be scanned

and therefore prefer a sequential scan.

How pgAssistant Recommends Indexes

pgAssistant does not simply parse SQL queries.

It combines multiple sources of information:

1. Query Plan

pgAssistant analyzes nodes in the query plan to identify candidate index columns.

2. Predicate Types

It classifies predicates into categories:

Equality predicates

=
IN
IS NULL

Range predicates

>
>=
<
<=
BETWEEN
LIKE 'prefix%'

Equality predicates are prioritized before range predicates.

3. Column Statistics

pgAssistant uses PostgreSQL planner statistics such as:

n_distinct
null_frac
most_common_vals
most_common_freqs
histogram_bounds

to estimate column selectivity.

Columns with higher selectivity are prioritized earlier in the index definition.

4. Table Statistics

pgAssistant also evaluates table-level statistics, including:

estimated row counts
table size
planner cost estimates

This is extremely important because some tables are simply too small to justify an index.

In these cases, recommending an index would create unnecessary maintenance overhead without improving performance.

How pgAssistant Recommends PostgreSQL Indexes

Why Query Syntax Alone Is Not Enough

A good recommendation depends on:

predicate types
column selectivity
table statistics
planner estimates
execution plans
existing indexes already used by PostgreSQL

This is precisely the approach implemented by pgAssistant.

pgAssistant Uses Execution Plans First

pgAssistant starts from the PostgreSQL execution plan.

It analyzes:

EXPLAIN (ANALYZE, FORMAT JSON)

This is extremely important because the execution plan reveals:

whether PostgreSQL uses a Seq Scan
whether an Index Scan already exists
whether residual filtering still occurs after index access
whether planner row estimations are inaccurate
whether a composite index could reduce heap filtering

This avoids many false-positive recommendations.

A query may look index-friendly while PostgreSQL is already using the optimal access path.

pgAssistant Analyzes Existing Access Paths

The advisor first determines how PostgreSQL currently accesses the table.

Examples:

Seq Scan
Index Scan
Index Only Scan
Bitmap Heap Scan

This distinction is critical.

Sequential Scan Case

If PostgreSQL performs a Seq Scan, pgAssistant evaluates whether an index could realistically improve performance.

Indexed Access Case

If PostgreSQL already uses an index, pgAssistant does not stop there.

It also analyzes:

Index Cond
Filter
Recheck Cond
Rows Removed by Filter

This allows pgAssistant to detect situations such as:

Index Scan using idx_customer on orders
  Index Cond: (customer_id = 42)
  Filter: (employee_id = 5)

In this case, PostgreSQL uses an index, but still visits many rows that are later discarded by the executor.

pgAssistant can therefore recommend a more selective composite index such as:

(customer_id, employee_id)

instead of considering the existing index “good enough”.

Predicate Classification

Once predicates are extracted from the execution plan, pgAssistant classifies them by operator type.

Internally, predicates are ranked according to B-Tree efficiency.

Equality Predicates

Highest priority:

=
IN
IS NULL

These predicates allow PostgreSQL to navigate directly to a very small portion of the index tree.

Prefix Search Predicates

Second priority:

LIKE 'abc%'

Prefix searches can still benefit efficiently from B-Tree traversal.

Range Predicates

Lowest priority:

>
>=
<
<=
BETWEEN

Once PostgreSQL enters a range scan, subsequent columns become far less effective for index navigation.

That is why range predicates are typically placed last in composite indexes.

How pgAssistant Orders Index Columns

After classifying predicates, pgAssistant computes candidate index ordering.

The internal ordering logic is:

1. Equality predicates first
2. Prefix predicates second
3. Range predicates last
4. Inside each category:
   highest cardinality first

This logic is implemented directly inside:

reorder_index_candidate_columns()

The advisor therefore builds indexes that align with PostgreSQL B-Tree traversal behavior.

Why Column Cardinality Matters

pgAssistant uses PostgreSQL statistics to estimate selectivity.

The most important metric is:

n_distinct

which estimates the number of distinct values in a column.

Higher cardinality usually means:

fewer matching rows
better filtering
smaller index scan ranges

For our example:

customer_id [=]: n_distinct=89
employee_id [=]: n_distinct=9
order_date [>=]: n_distinct=814

Although order_date has the highest cardinality, it is a range predicate and therefore placed last.

Among equality predicates:

customer_id > employee_id

because:

89 > 9

The final ordering becomes:

(customer_id, employee_id, order_date)

pgAssistant Uses PostgreSQL Statistics

pgAssistant enriches every recommendation using planner statistics extracted from PostgreSQL.

Examples include:

n_distinct
null_frac
most_common_vals
most_common_freqs
histogram_bounds

The advisor even exposes these statistics in its recommendation reasoning.

Example:

customer_id [=]: n_distinct=89, null_frac=0.0000
employee_id [=]: n_distinct=9, null_frac=0.0000
order_date [>=]: n_distinct=814

This makes the recommendation transparent and explainable.

pgAssistant Also Uses Table Statistics

An index is not always beneficial.

This is one of the most important concepts in PostgreSQL optimization.

For small tables, PostgreSQL often correctly prefers:

Seq Scan

instead of:

Index Scan

because:

sequential reads are cheap
index traversal has overhead
heap fetches introduce random I/O
scanning the entire table may cost less

This is why pgAssistant also evaluates:

estimated row counts
table size
planner costs
execution frequency
workload intensity

The advisor does not blindly recommend indexes whenever a sequential scan appears.

Detecting Inefficient Indexed Access

One particularly powerful aspect of pgAssistant is its ability to analyze residual filtering.

For indexed scans, the advisor evaluates:

Rows Removed by Filter

If PostgreSQL retrieves many tuples from the index only to discard them afterward, pgAssistant detects that the current index may be incomplete.

Internally, the advisor computes:

Residual filter kept X% of tuples visited

This helps identify situations where adding an additional column to a composite index could drastically reduce heap filtering.

Detecting Planner Estimation Problems

pgAssistant also compares:

plan_rows
vs
actual_rows

Large estimation gaps may indicate:

stale statistics
data skew
correlation issues
missing extended statistics

This additional analysis improves the reliability of recommendations.

The pgAssistant Recommendation Algorithm

Conceptually, pgAssistant follows this workflow:

1. Analyze EXPLAIN ANALYZE JSON plan
2. Detect access paths
3. Extract predicates from:
   - Index Cond
   - Filter
   - Recheck Cond
4. Classify predicates by operator type
5. Rank predicates for B-Tree efficiency
6. Use PostgreSQL statistics to estimate selectivity
7. Order columns by:
   - predicate class
   - cardinality
8. Evaluate table statistics
9. Evaluate execution costs
10. Detect residual filtering
11. Compare against existing indexes
12. Recommend index only if beneficial

Example: Final Recommendation

Given:

SELECT
    order_id,
    customer_id,
    employee_id,
    order_date,
    ship_country
FROM public.orders
WHERE customer_id = $1
  AND employee_id = $2
  AND order_date >= DATE $3;

pgAssistant evaluates:

Column	Predicate	Priority	n_distinct
customer_id	`=`	Equality	89
employee_id	`=`	Equality	9
order_date	`>=`	Range	814

The advisor therefore generates:

CREATE INDEX CONCURRENTLY
    "pga_idx_orders_customer_id_employee_id_order_date"
ON "public"."orders"
    ("customer_id", "employee_id", "order_date");

This ordering follows PostgreSQL B-Tree optimization principles while also considering:

planner statistics
table characteristics
execution plan behavior
residual filtering
existing indexes already in use

Live demo

A public demo is available here:

https://ov-004f8b.infomaniak.ch/

Demo connection:

postgresql://postgres:demo@demo-db:5432/northwind

The public demo intentionally runs without AI.

Project links

GitHub: https://github.com/beh74/pgassistant-community
Documentation: https://beh74.github.io/pgassistant-blog/
Docker image: https://hub.docker.com/r/bertrand73/pgassistant

Feedback welcome

The project is still evolving and many parts can certainly be improved.

If you work with PostgreSQL and have ideas, feedback, or criticisms, feel free to open an issue or discussion on GitHub.

Thanks for reading.

pgAssistant 2.8 — Deterministic PostgreSQL Analysis with the new Global Advisor

bertrand HARTWIG — Fri, 08 May 2026 06:01:15 +0000

For the past months, I have been working on a simple idea around PostgreSQL tooling:

before using AI, start with deterministic analysis.

This is the direction behind pgAssistant 2.8.

This release introduces a new component called Global Advisor, alongside many improvements around ranking, schema analysis, maintenance diagnostics, and index recommendations.

The project remains open-source and focused on practical PostgreSQL analysis.

What is pgAssistant?

pgAssistant is an open-source PostgreSQL analysis tool.

It helps developers:

inspect database structures
analyze execution plans
detect schema and maintenance issues
review indexes and foreign keys
understand PostgreSQL behavior more easily

The project combines:

deterministic analysis
execution-plan analysis (EXPLAIN ANALYZE)
optional AI-assisted reasoning

The goal is not to replace PostgreSQL expertise.
The goal is simply to make PostgreSQL diagnostics more accessible and more contextual.

The main addition in 2.8: Global Advisor

Before pgAssistant 2.8, most checks existed independently.

Now they are consolidated into a single entry point:

Global Advisor

The Global Advisor performs a database-wide deterministic analysis and aggregates findings into a unified recommendation list.

Each recommendation now includes:

a rank
a confidence score
an estimated impact
an estimated implementation effort
a suggested SQL statement when relevant

The objective is not to claim certainty.

It is to help prioritize investigations.

Deterministic first

One important design choice in pgAssistant is that the Global Advisor is intentionally deterministic.

The analysis is based directly on PostgreSQL catalogs and statistics:

pg_stat_user_tables
pg_stat_user_indexes
pg_constraint
pg_index
pg_settings
pg_stats
execution plans

This means:

same input → same output
no hallucinations
explainable findings
reproducible analysis

AI is still supported as an optional layer.

Examples of checks now included

The Global Advisor currently includes checks such as:

missing indexes on foreign keys
redundant or duplicate indexes
unused indexes
invalid indexes
datatype inconsistencies on foreign keys
tables without primary keys
stale statistics
tables never vacuumed
estimated table bloat
excessive index-to-table ratio
low foreign key coverage
PostgreSQL configuration checks
sequences approaching exhaustion

Most recommendations also include suggested SQL.

Query analysis is still there

The query advisor based on real EXPLAIN ANALYZE plans remains a core part of pgAssistant.

The idea is now:

Global Advisor → broad database analysis
Query Advisor → detailed query-level investigation

These two approaches complement each other.

About AI

AI support remains optional.

pgAssistant can work entirely without an LLM.

When enabled, AI features receive contextual PostgreSQL information:

schema definitions
indexes
execution plans
statistics
database settings

This significantly improves the relevance of generated suggestions compared to generic SQL prompting.

Supported providers currently include:

Ollama
OpenAI-compatible APIs

Why I built it this way

A lot of PostgreSQL tooling focuses on metrics dashboards.

Those tools are useful, but I often felt there was still a gap between:

seeing a metric
understanding the cause
deciding what to change

pgAssistant tries to reduce that gap.

The project is still evolving, but the Global Advisor is an important step toward a more coherent analysis workflow.

Live demo

A public demo is available here:

https://ov-004f8b.infomaniak.ch/

Demo connection:

postgresql://postgres:demo@demo-db:5432/northwind

The public demo intentionally runs without AI.

Project links

GitHub: https://github.com/beh74/pgassistant-community
Documentation: https://beh74.github.io/pgassistant-blog/
Docker image: https://hub.docker.com/r/bertrand73/pgassistant

Feedback welcome

The project is still evolving and many parts can certainly be improved.

If you work with PostgreSQL and have ideas, feedback, or criticisms, feel free to open an issue or discussion on GitHub.

Thanks for reading.

pgAssistant v1.7 released

bertrand HARTWIG — Sat, 01 Feb 2025 13:09:19 +0000

I'm excited to share that we just released pgAssistant v1.7.

PGAssistant is an open-source tool designed to help developers gain deeper insights into their PostgreSQL databases and optimize performance efficiently.

It analyzes database behavior, detects schema-related issues, and provides actionable recommendations to resolve them.

One of the goals of PGAssistant is to help developers optimize their database and fix potential issues on their own before needing to seek assistance from a DBA.

🚀 AI-Powered Optimization: PGAssistant leverages AI-driven language models like ChatGPT, Claude, and on-premise solutions such as Ollama to assist developers in refining complex queries and enhancing database efficiency.

🔗 GitHub Repository: PGAssistant

🚀 Easy Deployment with Docker: PGAssistant is Docker-based, making it simple to run. Get started effortlessly using the provided Docker Compose file.

I’d love to hear your feedback! If you find PGAssistant useful, feel free to contribute or suggest new features. Let’s make PostgreSQL database easy for dev Teams !

pgAssistant - postgesql tool 4 dev

bertrand HARTWIG — Mon, 12 Aug 2024 05:00:03 +0000

I wrote this small tool because I noticed that our young developers were struggling to understand the behavior of their PostgreSQL database, and therefore had even more difficulty optimizing their databases.

Gradually, developers started using it, and the tool helped the development teams become much more autonomous, and me much less solicited.

Feel free to use it and give me your feedback.

I try to enhance the application whenever time allows.

You can find it there : https://github.com/nexsol-technologies/pgassistant

DEV Community: bertrand HARTWIG

PostgreSQL: Which Queries Should You Optimize First?

Indexing every WHERE column is not PostgreSQL optimization

Designing the Right PostgreSQL Index Using Query Plans and Statistics

Why Index Design Is Difficult

The Fundamental Rule of B-Tree Indexes

Equality Predicates Are Highly Selective

Why Range Predicates Should Come Last

Column Order Among Equality Predicates

PostgreSQL Statistics Drive Good Index Design

Understanding n_distinct

Why Selectivity Matters

The Correct Index Design

Good Index Design Also Depends on Table Size

Query Plans Matter More Than Theory

The Importance of Execution Plans

How pgAssistant Recommends Indexes

1. Query Plan

2. Predicate Types

Equality predicates

Range predicates

3. Column Statistics

4. Table Statistics

How pgAssistant Recommends PostgreSQL Indexes

Why Query Syntax Alone Is Not Enough

pgAssistant Uses Execution Plans First

pgAssistant Analyzes Existing Access Paths

Sequential Scan Case

Indexed Access Case

Predicate Classification

Equality Predicates

Prefix Search Predicates

Range Predicates

How pgAssistant Orders Index Columns

Why Column Cardinality Matters

pgAssistant Uses PostgreSQL Statistics

pgAssistant Also Uses Table Statistics

Detecting Inefficient Indexed Access

Detecting Planner Estimation Problems

The pgAssistant Recommendation Algorithm

Example: Final Recommendation

pgAssistant 2.8 — Deterministic PostgreSQL Analysis with the new Global Advisor

pgAssistant v1.7 released

pgAssistant - postgesql tool 4 dev

Understanding `n_distinct`