Why LLM Agents Still Can't Query NoSQL Databases

When it comes to SQL databases, LLMs are great at writing SQL. It's precise, expressive, and unambiguous. LLMs write it well. Connect an MCP server to Postgres and the agent can write queries directly and efficiently. It's a lot harder for agents to work with NoSQL databases, and given how much production data lives in them, I'm surprised there isn't more discussion about it.

Why SQL works so well for LLMs

The core reason SQL is a natural interface for language models is specificity. A SQL query says exactly what data you want, in exactly what shape, with exactly what conditions. There's no ambiguity for the model to navigate.

SQL is also one of the most well-represented languages in LLM training data. Decades of documentation, Stack Overflow answers, textbooks, and open source code have produced an enormous corpus for models to learn from. The syntax is consistent. The semantics are well-defined. When an LLM writes SQL, it's drawing on a deep, reliable foundation.

It also maps cleanly to tool calls. An agent calls run_sql with a query string and gets back rows. The interface is simple, inputs and outputs are typed, and errors are catchable. It's a well-defined contract between the agent and the data layer.

The NoSQL problem

The same isn't true for NoSQL, and the root cause isn't just syntax. It's that there's no shared interface. Each major NoSQL database has its own query model, its own access patterns, and its own constraints. For an LLM to query any of them effectively, it needs to understand not just the syntax but the underlying data model.

MongoDB uses a document query syntax built around JSON filters and aggregation pipelines. DynamoDB uses an API-style expression language built around partition keys and index access patterns. Cassandra has CQL, which looks like SQL but has strict limitations on what queries are allowed based on how tables are partitioned. Redis exposes commands rather than queries. Neo4j uses Cypher, a graph query language with its own pattern-matching syntax.

These aren't variations on the same problem. Each one requires the agent to internalize a different mental model for how data is stored and retrieved. There's no training data equivalent to decades of SQL documentation to draw from. And because developers' attention is split across all of them, the tooling and ecosystem around each is thinner than what SQL databases enjoy.

A concrete DynamoDB example

Take DynamoDB specifically. It requires a partition key for efficient queries. Without one, you're doing a full table scan, which reads every item in the table and applies filters afterward. It doesn't support JOINs, native aggregates, or GROUP BY.

Say an agent needs to answer: "Find how many orders were placed by region this week."

In SQL, that's straightforward:

SELECT region, COUNT(*) AS order_count
FROM orders
WHERE created_at >= CURRENT_DATE() - 7
GROUP BY region
ORDER BY order_count DESC

In DynamoDB, the agent can't write that query. It has to write code:

const results = {};
let lastKey;

do {
  const response = await dynamodb.scan({
    TableName: 'orders',
    FilterExpression: 'created_at >= :start',
    ExpressionAttributeValues: { ':start': startDate }
  });

  for (const item of response.Items) {
    results[item.region] = (results[item.region] || 0) + 1;
  }

  lastKey = response.LastEvaluatedKey;
} while (lastKey);

This is more code, harder to audit, and requires the agent to understand DynamoDB's pagination model. It also runs a full table scan rather than hitting an index, which is expensive on large tables. And this is a simple question. Anything involving a JOIN across two tables gets significantly more complex.

LLMs can do so much already. It's frustrating that they still can't reliably handle something this straightforward against a database as widely used as DynamoDB.

What teams do about it

Before reaching for a purpose-built solution, most teams working with DynamoDB work through a sequence of workarounds.

The first option is PartiQL, AWS's own SQL-like syntax for DynamoDB. It's built into the SDK and the console, so there's no setup required. The agent can write queries directly against DynamoDB without any additional infrastructure. The limitation is that PartiQL is a syntax layer, not a query engine. It translates SQL-style statements into native DynamoDB API calls, which means it inherits all of DynamoDB's query constraints. The agent still needs to supply a partition key for efficient queries. There are no JOINs, no GROUP BY, and no native aggregates. The syntax looks like SQL, but the access model is unchanged.

The second is exporting to S3 and querying with Athena. Engineers set up a DynamoDB export pipeline to S3 and configure Athena to query the exported files. Once that infrastructure is in place, the agent can write SQL against Athena directly. The tradeoff is freshness: the export isn't real-time, so the agent's results are only as current as the last export. Engineers also have to maintain the pipeline and monitor it for failures.

The third is streaming to a secondary store. Engineers pipe DynamoDB changes to Redshift, OpenSearch, or a relational database, and the agent queries the replica. Once the infrastructure exists, the agent gets a proper query interface. But it's querying a copy of the data, not the source. Engineers have to maintain the pipeline, and when it lags or fails, the agent's results are wrong without any obvious signal that something is off.

None of these are unreasonable choices. PartiQL is the lowest-friction starting point, but it hits a ceiling fast. The infrastructure-based approaches give the agent a proper query interface, but at the cost of engineering overhead and data that's never fully current.

What we built

We ran into this exact problem building an MCP server against DynamoDB. The agent could reason about the data well, but getting it out required knowledge of partition key layouts, GSI structures, and access patterns that wasn't realistic to expect the agent to have upfront.

So we built an SQL engine on top of DynamoDB and called it DynamoSQL. When the agent submits an ANSI SQL SELECT statement, DynamoSQL parses it, builds a query plan, and runs an optimizer that inspects the table's actual index metadata. The optimizer selects the best access path: a partition key Query if the WHERE clause supports it, a GSI Query if a matching index exists, or a Scan as a last resort. The agent doesn't configure any of this. It just writes SQL.

The execution layer handles what DynamoDB can't do natively. JOINs run across multiple DynamoDB API calls. GROUP BY, aggregates, subqueries, CTEs, UNION, ORDER BY, and HAVING all work in the engine. Results come back as rows. The agent doesn't need to know anything about the underlying data model.

The MCP server exposes a single run_sql tool. The agent writes a query, the engine executes it, and the result is a clean set of rows, the same interface you'd expect from any SQL database.

Where this leaves things

SQL databases and LLM agents are a natural fit. The interface is clean, the semantics are well-defined, and the tooling is mature. NoSQL databases are still mostly a compromise: a syntax layer that hits a ceiling, or export pipelines and secondary stores that require engineering overhead and never give the agent fully current data.

The fragmentation across NoSQL databases makes this hard to solve once and apply everywhere. But for teams using DynamoDB, the SQL layer approach works well in practice. The agent writes what it wants. The engine figures out how to fetch it efficiently.

The broader question of whether this is the right approach across all NoSQL databases is still open. But the cost of the status quo is real, and it deserves more attention than it's getting.