DEV Community: Chris Davies

You build it, you budget it? 💰

Chris Davies — Wed, 06 Aug 2025 15:05:00 +0000

📝 This is cross-posted from my personal site - https://chrisdavies.dev .

Figma's recent S-1 filing was called out in a flurry of headlines for declaring a spend of "$300,000 a day" on AWS. At face value, that number seems shocking, but perhaps the industry reaction itself is the main thing worth examining.

Many organizations subscribe to the "you build it, you run it" operating model, but this ownership rarely extends to costs. When was the last time your pager went off in the night because a service experienced an unusual spike in costs? ☎️🧑‍🚒

In this post, we look at:

Why costs are a shared responsibility.
What developers (and tooling providers) can do to help own them early on.
The vendor "lock-in" trade-off.

This post was largely inspired by SigNoz's post: "Observability isn't just for SREs".

FinOps 💰

Cost management can often feel far-removed from an engineer's remit. The reality is, behind staffing costs, hosting costs are one of the biggest line items for a business, and FinOps is one of the few engineering disciplines where you can know your impact on the bottom line within 24 hours.

Companies like 37signals have been vocal about their cloud exit, presumably feeling like this when it came to paying the invoice.

"Frivolous" Figma 🤑

Figma's reported daily spend immediately becomes a moot point when you hear that they are running with a 91% gross margin and their infrastructure spend is only 12% of their revenue. They have been public about the many ways they have reduced cloud spend.
Thankfully, a lot of the noise was debunked by prominent voices such as Corey Quinn. The reaction is interesting though - do engineers really know what a reasonable hosting cost is at their organization, or what gross-margin is?

Availability often trumps Costs (🧑‍🚒 > 💸)

It is understandable why many organizations implicitly prioritise availability over costs in their definition of "owning" a production system.

Visibility: The former is immediately visible to clients, so does not carry as much immediate reputational damage.
Complexity: It is easier to reason about. An SLA is a static target. Costs are dynamic - each part of the system's spend may correlate with different factors (e.g. scheduled bursts or spikes due to system activity).
Maturity: Startups desperately need market fit before they can set up a rigid FinOps strategy. If growth is sufficiently high in the early stages, costs are less of a concern.
Cost: The ironic one - engineers are expensive, so having them purely focus on this is sometimes not viable.

Despite feeling like orthogonal concerns, several concepts we use to describe availability could also carry over into the FinOps space.

Availability 🧑‍🚒	FinOps 💸
SLA	Target spend - a fixed $ value, or % of revenue.
Alert	A budget breach or anomaly alarm.
MTTD	Mean time (duration) to detect a cost spike.
MTTR	Mean time (duration) to recover from a cost spike.
Incident	An investigation into an unexplained cost.
Observability	Cost and usage reports/dashboards.

Conway's Law: A limiting factor? ⚖️

Many organizations choose to divide teams into Product and Platform responsibilities as a way of creating abstraction. As Conway's Law describes, a system architecture will become a representation of the organization structure. This results in situations where developers shipping features often do not understand the infrastructure that their code runs on. Greater abstraction means freedom at the infrastructure level, but also means less scrutiny on hosting costs for developers who aren't operating at that level of the stack.

The key trade-off in these team boundaries, is that many meaningful cost optimization opportunities arise in the intersection of what Product and Platform Engineering teams can achieve alone. If the responsibility to optimise costs lies with a central team, the goal becomes a moving target as micro-level decisions elsewhere slowly negate the impact of planned work. Efforts significantly reduce if a culture of awareness/ownership exists across teams.

A simplified model - responsibilities will vary by org, with impactful efforts sitting in the overlap.

Optimisations require partnership 🤝

In a series of projects spanning a few quarters, we managed to reduce our AWS spend by 40%. This was no mean feat, but a key lesson was that every optimisation required communication between Platform and Product teams. Whether gathering data, getting buy-in or keeping teams in the loop in case of unexpected outages.

Life is also made significantly easier if Product-focused teams can readily help explain ongoing cost changes (e.g., a new client onboard, a recent feature rollout etc.). If revenue increases proportionally, it becomes less of a concern and budgets can be adjusted to the new baseline.

Let's look at some common types of optimisation, and the level of their cross-team dependencies. These are placed in order of increasing dependency on Product context, to squeeze the most value out of each optimisation.

Settings ⚙️

Settings which are low-effort to change, and should arguably be defaults. Once enabled, they are immediately effective.
e.g.: Using intelligent tiering, bucket encryption keys, different architectures (e.g. Graviton), log classes.
Dependencies: Co-ordination is needed during testing and rollout to minimise impact on production environments.

Cleanup 🧹

Reducing the footprint of compute services and managing the lifetime of data.
e.g.: Removing unused services, adding lifecycle policies, TTLs, job timeouts.
Dependencies: Much like Hyrum's law for APIs, it is almost never clear which service or data is relied upon until it is deleted. Product teams can provide context on which service/data is valuable and for which initiatives/clients far quicker than metrics would alone. Metrics tell you the current state of the world - not what to expect in the near future.

Right-sizing 🤏

Matching the reserved capacity of compute services as closely to their usage as possible.
e.g.: Changing CPU and memory (statically or dynamically), removing redundant instances, adopting serverless.
Dependencies: If embracing serverless, there are fundamental limitations (e.g. memory, timeouts) that Product teams need to know about when shipping code. Right-sizing might mean use of horizontal or vertical scaling, which means choosing a sensible scaling strategy (e.g. scheduled, on-demand). This all requires crucial Product context.

Re-architecting 🎲

Changing the operating model or system architecture of a service, accepting a reasonable trade-off (e.g. performance, availability) for a significant cost reduction.
e.g.: Embracing event driven architecture (EDA), removing middleware (e.g. queues/load balancers/proxies), using Spot compute.
Dependencies: Adopting EDA might add integration latency at key points in a customer workflow. Using interruptible batch jobs could delay critical outputs if there is no checkpointing in use when the job restarts. Metrics can help guide decisions on how often middleware services add value, but Product context is needed to assess whether they are needed long-term. What might seem overkill for today's scale might be perfectly sensible for an upcoming initiative.

Code changes 🧩

Modifying the behaviour of application code to alter its usage of resources (e.g., databases, storage, APIs).
e.g.: Caching, "pushing down" data filtering, changing storage/API access patterns, reducing logging.
Dependencies: Often the hardest optimisations for Platform teams to spot if relying on infrastructure metrics alone. An isolated measure of CPU/memory usage or database accesses will not tell you whether there is room to meaningfully optimise behaviour. This is where an observability stack of Logs, Metrics and Traces matters. Everyday code changes can introduce large changes in the cost-profile of a service. Shifting left is even more important at this level (discussed below).

Shifting left ⏪

So far, we have only touched on optimisations once costs are already a problem. With greater ownership across teams, we can shift a lot of these concerns "left" earlier in the development cycle, to reduce the size of future problems. Not every developer needs to take on large, multi-week optimisation projects to be seen as "owning" costs.

A sad irony is that getting visibility into costs itself carries a cost (through additional services and SaaS subscriptions), but the payoff is often worthwhile.

Observability 📊

Giving teams visibility of their baseline costs is the first step. If using a cloud provider, a lot of these primitives are provided:

Tagging: Label infrastructure by team/project. Regularly monitor Cost Explorer dashboards (e.g. AWS/GCP/KubeCost) and schedule reports.
Alerting: Set up budgets and anomaly detection, with integrations to the communication platforms (e.g. Slack & incident.io).
Dependencies: Monitor usage metrics for each third-party provider / service / API that has its own pricing model.

Once observability is in place, let's look at some practical methods that enable cost ownership earlier on in development.

Designs 🎨

During the design of a new service or feature, ADRs (Architectural Decision Records) could explicitly consider the cost impact of decisions - forcing early discussion. Whilst it is tricky to provide exact dollar estimates, a "back of the envelope" calculation is made far easier by cloud providers' Pricing Calculators. In the age of AI and LLMs with spiralling costs there, there are even token calculators!

A mental model I like to use for Cloud costs is:

Compute = (Capacity * Time)

Storage = (Retention * Data Size) + Data Transferred

Logging = (Retention * Event Count * Event Size)

Tooling 🛠️

A range of tooling exists to target the IDE and Pull Request workflows, which developers can use to estimate and reduce cost impact before code is shipped.

For infrastructure changes, tools like Infracost offer comparisons in the IDE and PR reviews. At the application layer, I've had some success using Datadog's IDEA plugin to remove noisy logs and optimise frequently executed code. There is room for vendors to go further in this space, and bring cost awareness to the IDE for application code.

Image credit: Infracost

Artificial Intelligence 🤖

As AI tooling like Claude Code and CodeRabbit start to integrate into CLI/IDE and VCS software, there is a natural space to ensure costs are considered with every prompt.

Amazon's Kiro is an interesting proposition here. Providing "spec" documents and breaking tasks down like a human before code is generated would naturally lend itself to feeding in ADRs like those described above. If a file exists in the repository describing the system's tenancy model, budget and priorities as a system prompt, then the code these tools generate could reflect that.

"Lock-in": Friend or foe? 🔐

The final topic we will touch on here is vendor "lock-in". A lot of the scrutiny on Figma's IPO filings came from their $545m-per-year multi-year commitment to AWS. By admitting their reliance on a sole provider and solidifying plans to spend across a long-term horizon, it was seen by many as a big concentration risk.

As Corey Quinn points out, that size of commitment may be risky for a company hosting "lifted and shifted" VMs where compute costs come at a premium, but Figma's technical stack is deeply ingrained in AWS ecosystem already. Once you are far enough along the adoption curve of a particular cloud provider, it leaves a lot of money on the table if you do not make commitments. This could be explicit commitments via Savings Plans/reserved capacity, or implicit ones like moving "glue code" to the infrastructure layer. The latter can save many Engineering hours, in return for greater dependence on a specific provider's primitives.

Serverless Refactoring ☁️

One of my go-to talks from recent AWS re:Invent conferences is about Serverless Refactoring, which blurs the line between application and infrastructure concerns.

TL;DW (Too Long; Didn't Watch)

The next time you find yourself writing code to integrate two pieces of infrastructure, it's worth questioning if a different primitive exists instead.

Summary 🧵

This post tried to cover a lot of ground. Here are my main takeaways:

💸 Costs are a shared responsibility across development teams, like availability, with a more immediate ROI. Delegating responsibility to a single platform/FinOps team leaves many optimisations off the table, and may elongate projects.
💻 Developer tooling and AI can shift a lot of effort left. Producing code will become less of bottleneck as more of this responsibility is delegated to machines. However, understanding how that code impacts system reliability and costs will only matter more over time.
🔐 "Lock-in" and cloud commitments get a bad reputation, because it's harder to quantify the opportunity cost of not going "all-in" on providers' offerings.

As always, context matters, so the next time we see a headline about an organization's hosting costs, it's worthwhile treating it with the nuance it deserves, rather than being outraged by the headlines.

Get in touch 📧

As I say in my About page, I would love to hear from you. If you got to the end of this post and have anything to share, please get in touch on LinkedIn or Twitter.

In future posts, I may dive into some neat cost management measures I have seen in the wild. In the meantime, here are a few podcast recommendations:

🎧 Screaming in the Cloud (Last Week in AWS)
🎧 Cloud Masters (DoIt)

Just Use Postgres 🐘

Chris Davies — Sun, 06 Jul 2025 07:37:06 +0000

📝 This is cross-posted from my personal site - https://chrisdavies.dev .

A common adage for any developer choosing a database technology is "just use Postgres". While this advice remains well-intentioned, it deserves a fresh look in 2025. Thanks to a wave of recent innovations and acquisitions, PostgreSQL providers now resemble JavaScript frameworks of the 2010s - there’s seemingly a new one to consider every six months.

Just choosing PostgreSQL isn’t enough anymore - there is an entirely new decision tree to follow before you can launch a system. 🌳

TL;DR: The answer to "Which one do I choose?" is always "It depends", but this post signposts exactly what those deciding factors are. ⚖️

Postgres uses an elephant as its mascot. This means I have a good excuse to reminisce on my March 2025 trip to Thailand 🇹🇭

Postgres Renaissance 🎨

The saying "just use Postgres" is deliberately simplistic. It is intended to reduce analysis paralysis at a critical point in a system's lifetime. Postgres is the "boring" choice because it has been battle-tested in production for decades by thousands of companies. Being open-source and extensible, it has a thriving community. There is very little to gain by choosing a less-proven technology. Or one with restrictive licensing. Or adopting several solutions with their own cost models. It closes the fewest number of doors when a project's uncertainty is at its highest.

Recent Postgres versions and extensions (e.g pgvector) have introduced and significantly improved support for vector/JSON data, which makes it even harder to justify starting with a specialised store. Extensions like pg_duckdb support column-oriented, analytical workloads inside a row-oriented, transactional system. Many articles exist on the versatility of Postgres. Some even go as far as stating you can host a full REST API with PostgREST (though I haven't seen Production examples of this).

In recent times, database technologies derived from Postgres have exploded in number and seen rapid success. In the last few months alone:

🤑 Databricks acquired Neon, a serverless Postgres provider.
🤑 Snowflake acquired Crunchy Data, a hosted Postgres provider.
🎉 AWS Aurora DSQL was released.
🎉 PlanetScale expanded on its MySQL foundation, to offer a Postgres alternative.

Below is a non-exhaustive list of recent technologies derived from Postgres, and their offerings.

Technology	Description
Neon	Serverless, fully-managed instances with autoscaling and branching.
Nile	Serverless, fully-managed instances for multi-tenant applications (e.g. B2B SaaS).
Supabase	Open-source Firebase alternative - either managed or self-hosted.
TigerData (TimescaleDB)	Optimized for time-series and real-time analytics - either managed or self-hosted.
CockroachDB	Distributed SQL with Postgres compatibility - either managed or self-hosted.
PlanetScale Postgres	Managed instances, emphasizing performance and scalability based on NVMe SSDs.
Prisma	Serverless, fully-managed instances with a TypeScript Object Relational Mapper (ORM).
YugabyteDB	Distributed SQL with Postgres compatibility - either managed or self-hosted. Zero-downtime upgrades.
Crunchy Data	Fully-compatible Postgres instances for Enterprise - either managed or self-hosted.
Heroku	Managed instances integrated into the Heroku platform (PaaS).
AWS RDS / Aurora	Managed instances integrated into the AWS platform - provisioned or serverless.
Google Cloud SQL / AlloyDB	Managed instances integrated into the Google Cloud platform - provisioned or serverless.
Azure Database	Managed instances integrated into the Azure platform - provisioned or serverless.

Image credit: TigerData

Cautionary Tales ⚠️

Choosing Postgres over other relational databases doesn't always go to plan. Uber famously switched to MySQL after facing slowness during writes, favouring the way MySQL handles index updates. Other systems have faced issues such as table bloat and transaction ID wraparound due to Postgres’s MVCC model, which avoids locks in favor of versioning.

This is a good opportunity to preach the lessons from Designing Data Intensive Applications. Without understanding the underlying storage and retrieval technology these databases rely on, you are likely to face surprises in Production.

Recent Trends 🔨

In this section, we go over the main innovations that have emerged in recent years, which have caused the explosion in providers, each with their own unique offerings. As Postgres is open-source, it is possible to cherry-pick features and compromise on some fundamental aspects in order to unlock new behaviours.

Compute <> Storage Divide ➗

Postgres pre-dates the age of cloud and distributed systems. As such, it is monolithic (combining storage and compute on the same server) and process-oriented (rather than thread-oriented). To thrive in a cloud-native world, providers have decoupled storage and compute layers, which allows each layer to scale independently. This change enables serverless systems like Neon and AWS Aurora Serverless. Use of durable block storage like S3 means lightweight VMs can run the Query Engine and hold minimal state, allowing horizontal scalability. Different systems embrace this to different extents. For example, Aurora Serverless v2 incurs a 15 second cold-start when scaling up from zero.

Besides scalability, the other by-products of standalone storage are database branching and fast backups due to copy-on-write semantics. These are game-changing features that a traditional Postgres system cannot provide.

Relaxing Isolation for Speed 🏎️

Postgres is flexible in supporting 4 levels of transaction isolation at session or transaction level. By constraining the isolation level to Repeatable Read (using something like Snapshot Isolation) rather than Serializable, we require far less consensus before transactions can commit or return data.

In most applications, the key-set of data being read is far greater than the key-set of data being written, and reads outnumber writes by an order of magnitude. This realisation along with constraining isolation level to Repeatable Read is core to the way AWS DSQL scales. By constraining in one dimension, new technologies are unlocking scale that a traditional setup could not match.

New Data Primitives 📊️

Being a general purpose and extensible database, some providers have imposed an opinionated abstraction on top of core Postgres. For example, TigerData introduces hypertables to model time-series data, which would otherwise need manual partitioning and regular pruning via extensions like pg_partman and pg_cron.

AI companies make use of vector stores (through pgvector and HNSW indexes) to build RAG (Retrieval Augmented Generation) pipelines. The recent Databricks and Snowflake acquisitions show that engineers increasingly value an "all-in-one" platform rather than a standalone OLAP solution.

If you are new to Postgres, I recommend Hussein Nasser's YouTube channel, which has a detailed video explaining the Postgres internals.

Decision Points ⚖️

This section raises a set of questions I would consider when evaluating a Postgres provider to reduce a lot of future pain points. They are not ordered by priority.

Query Patterns ❓

What is the expected ratio of reads and writes - is a single instance sufficient? (Does the provider offer a range of instance sizes? Are there read replicas, connection poolers or sharding solutions?)
What periods of inactivity am I expecting? (Serverless is more economically-viable for spiky workloads. In practice, when baseline traffic varies by at least 50% from its peak.)
What levels of transaction isolation will I need? (This rules out a few distributed offerings.)
Does the provider have global endpoints / multi-region support? (Local benchmarks are no good if your database is on the other side of the world).

Compatibility 💘

Do we need runtime compatibility, wire-protocol, or something in between? (How many Postgres features will I have support for?).
What service limitations exist? (e.g., Limits on database object counts, concurrent connections, foreign keys, transaction size or timings).
What extensions are available? (e.g., Cloud platforms often expose only a subset).
Are system tables accessible, or can I easily monitor diagnostic information? (e.g., pg_stat-* tables).
What integrations exist with existing services? (e.g., Cloud providers offer archival or "zero-ETL" features).

Control 💪

Does a superuser exist? (Some platforms do not provide access).
How portable is my data? (Can the underlying backups and data files be exported externally?).
How much control do we need over major versions' adoption timelines? (Vendors often enforce a support window).
What methods are supported for upgrades? (e.g., In-place, logical replication, zero-downtime. Providers like YugabyteDB claim to have zero-downtime upgrades. Others need DIY solutions).
What is the pricing model? (Is billing broken down by storage, compute and I/O?).

Security & Risk 🎲

How much trust do you place in the company in the long term? (Cloud providers may sunset services but have better longevity. Newer providers may withdraw their free tier, or be acquired and change pricing).
What encryption and security models exist? (Is my infrastructure pooled or isolated? Can I rotate encryption keys used?).
Is there support for multi-tenancy or Row-Level Security (RLS)? (Nile aims to address this for B2B SaaS firms).
What SLAs exist and is downtime compensated? (e.g., During scaling events / maintenance windows).

Starting Point 📍

Having raised the important questions, it is clear that there is a lot to think about upfront. In the absence of any other information (and to avoid being overwhelmed!), I would adopt these principles.

Stay close to the roots 🪾: Maintain runtime compatibility with standard PosrgreSQL for as long as possible. This makes local testing and feedback loops much quicker, and means faster adoption of new versions.
Benchmark 📐: Measure against your own workloads - do not rely solely on online benchmarks.
Avoid sprawl 🐙: Default to Postgres for new use cases before reaching for new platforms or databases.
Know your options 🧠: Periodically evaluate which migration paths exist, their upgrade timelines, tenancy models and trends.

Once a solution is adopted at scale, migration to a new setup is always possible, but it takes significant effort (just ask Figma). There is no way to know the future, but with these principles in mind you will do better than most at adopting the right flavour of Postgres for your use case.

Summary 🧵

In this post, we revisited the advice to "just use Postgres" refreshed for 2025, in the broader context of acquisitions and recent product launches. The statement still holds true, but some nuance exists to answer the question of "what next?". These questions form a practical checklist for evaluating Postgres providers.

It is an exciting time to be a developer working with Postgres, as the technology continues to evolve almost 40 years after its inception at Berkeley.

In future posts, I may dive into some of the idiosyncrasies of Postgres or dive into systems like AWS' DSQL. There are some great creators in the space which make it far easier to keep up to date in the meantime. For example:

Get in touch 📧

As I say in my About page, I would love to hear from you. If you got to the end of this post and have anything to share, please get in touch on LinkedIn or Twitter.

Call My Agent 🤖

Chris Davies — Sat, 28 Jun 2025 09:03:20 +0000

📝 This is cross-posted from my personal site - https://chrisdavies.dev .

This week, we take a brief step into the world of GenAI Agents, having read "Principles of Building AI Agents" by Sam Bhagwat. 📚

The title is a terrible pun on a French TV show... 📺

The Book 📖️

I first learned about this book from a Tweet ("X"?) by Gergely Orosz. Out of curiosity, I downloaded the e-book and Sam kindly reached out to send across a physical copy. The need for a second edition just months after the first highlights how fast this space is moving.

It is a short read (2-3 hours) for technical readers that introduces a lot of breadcrumbs for people to go away and read more on. This means that by the end, you are not an expert, but carry a lot of the terms you need to do research and have conversations about the topic. It uses a top-down approach, building on top of LLMs, not explaining their foundations.

The book is available in virtual or physical form.

I am not being sponsored to write this - I just think it is great organic marketing and enjoy shorter reads.

Mastra is a company building a framework for building AI Agents and orchestrating them with workflows. Sam is a co-founder of the Gatsby web framework, and is taking an interesting bet on the TypeScript community as the place to start, with some early adoption examples.

In the next few sections, I'll go over some terminology and takeaways.

With thanks to Sam Bhagwat

My Two Cents 🪙🪙

Like most waves of change and hype in the tech industry, the GenAI topic (coined as "Software 3.0") causes divisiveness and scepticism. A lot of this reaction is justified.
AI is not a new field - it has been around for decades. However, the release of ChatGPT in November 2022 created a fresh wave of noise around Generative AI.

Not long ago, we had a hype cycle around Blockchain and Cryptocurrency, which did not cause the revolution it promised. Blockchain often felt like a solution in search of a problem. GenAI has a plethora of tangible use cases, but needs to answer difficult questions in many areas before it can allay concerns. Namely surrounding the ethical, legal, socioeconomic, environmental and security impact of adopting this technology at scale. This is why the marketing can sometimes feel premature.

We could place a lot of these hype cycles along different points of the Gartner Hype Cycle. Blockchain and Cryptocurrency has found its niche after the hype has calmed down. GenAI could follow a similar path once the dust settles.

To avoid bloat, this post purely focuses on the engineering discipline that the book introduces, rather than the impact of this technology or my personal use of it for productivity. I will save those for a future post.
Criticism, if justified, needs to come from a place of education.

Image credit: Gartner

Jargon Busting 🧑‍🏫

The book introduces a lot of terminology. A true test of my understanding is whether I can now explain these concepts, which is what this section tries to achieve. Feel free to skip this section if you already have a good understanding.

Disclaimer ⚠️: I did have some background knowledge before reading this book, but not much. Academic readers might cringe at my attempt.

Term	Definition	Example
LLM	Large Language Model. Probabilistic model trained on natural language which, when given a training corpus and input, will predict the most likely output using mathematical operations (e.g., matrix multiplication). These can be hosted or open source.	GPT-o1, Gemini, Llama
Prompt	Input to an LLM (commonly textual). These can be system or user level, which alters the scope (e.g. chat-level or individual query respectively). Prompt engineering is the practice of tailoring prompts to improve outputs.	`"ChatGPT is cool!"`
Token	Single raw chunk of text, taken from a larger document or textual input.	`["Chat", "G", "PT", " is", " cool", "!"]`
Embedding / Vector	Transformed `N`-dimensional data structure representing the semantic meaning of a token or document. `N` is typically ~1536. This structure is used for similarity searches in `N`-dimensional space, which gives a numerical reading on how relevant the stored content is to the input prompt.	`[0.12, ..., 0.95]`
Context Window	The number of text chunks (tokens in the input and output) which the LLM can handle at any one time when processing a query.	GPT-4 = ~128k tokens.
Agent	A long-lived process that uses LLMs to perform tasks. They are allocated memory, invoke tools, access resources, and maintain context over time.	AI assistant that books meetings.
MCP	Model Context Protocol. Defined by Anthropic which stipulates how an agent should retrieve information from a third-party source. An MCP Server (hosted locally or remotely) declares its prompts, tools and resources for MCP clients to use.	Agent that is given instructions for connecting to a database to find weather information.
Tool (MCP)	Method for the agent to perform a task e.g. a web resource lookup, or calling another Agent.	`fetch_weather(city)`
Resource (MCP)	Data source used by the agent when performing tasks e.g. a database.	`city_db`
Prompt (MCP)	Templated instructions for the agent to perform a task e.g. details needed to call a tool or read from a resource.	`"Search {query} in {resource}"`
A2A	Agent To Agent Protocol. Defined by Google which stipulates how an Agent should communicate with another Agent, given that they are likely to have been built independently.	`Agent1` asking `Agent2` to extract data.
RAG	Retrieval Augmented Generation. A method for enhancing the information available to an LLM by performing on-demand embedding and similarity search with a vector database, to enrich a prompt with added stored context.	Retrieval of FAQs before answering queries.
Vector Database	A data store specialised in storage and retrieval of `N`-dimensional vectors.	pgvector, Pinecone
Agentic Workflow	An emerging practice which defines the tasks agents perform as steps in a deterministic orchestration workflow - commonly as a graph.	`Fetch doc → summarise → email`
Guardrail	Measures which inject additional context in the prompts given to LLMs to minimise the risk of them producing insecure or malicious output. More of a guidance mechanism than a watertight security measure.	Requesting that PII is not returned.
Eval	Methods for evaluating the effectiveness and accuracy of an Agent.	OpenAI Evals

If you are a visual learner, then I am a fan of the ByteByteGo YouTube channel, which has a short video explaining some of these concepts.

Key Snippets ✂️

Despite being a short read, the book is full of advice from practical experience. This section summarises the main lessons.

Non-Determinism 🎲

Embrace not being sure of the answer. The book makes it very clear from the start that adopting this technology means being comfortable with non-determinism. A lot of Software Engineering is made easier by ensuring our programs exhibit repeatable, deterministic behaviour, so this introduces new challenges. In this field, the fundamental building blocks are probabilistic. The same input may generate infinite variations of output. Going back to the "Software 3.0" concept, even "2.0" (ML and Neural Nets) can be made deterministic (by fixing weights and parameters), so this marks a complete departure.

The idea of creating dynamic agents takes this one step further, as it might alter the agent environment such as the native spoken language and the LLM in use (based on business rules). This explodes the number of possible execution contexts.

Testing looks very different in a non-deterministic world. Instead of pass/fail semantics, Evals provide an interval of confidence ({0,1}) by which we can measure accuracy and reliability, and pick up on meaningful regressions or improvements. Because of this complexity, organizations often find it simpler to do A/B testing in production by measuring signals of user behaviour after deployment as a measure of effectiveness by proxy.

Because of all this non-determinism at the inner layer, imposing a graph-like structure on agentic workflows seems like a sensible method of constraining the probability space.

Size vs Cost vs Performance 💰

Start simple, then optimise. A trade-off is required between cost, performance and accuracy of the underlying models. Cheaper models will produce fast, less accurate responses, which might be enough for some use cases. The book advises starting with a costlier model (for greater accuracy) whilst volumes are low, then optimising. One should start by exhausting the LLM's context window, before building agents, workflows or RAG pipelines.

There are clever UX techniques one might employ to make it look like the agent is doing useful work before it outputs a final answer, showing its workings and reasoning. A few solutions mentioned in the book stream their responses to a client via web-sockets, making use of tools like ElectricSQL before a full answer is generated.

Hosted vs Open-Source 🥷

As for the choice between hosted versus open source models (e.g., Claude vs DeepSeek), it is a natural reaction to want to see the inner workings of these models. I am reminded of the choice organizations need to make around adopting cloud computing. A hosted model provides much less of the hassle in getting up and running with this technology, which doesn't differentiate a business from its competitors. On the surface there is not as much vendor lock-in (yet) compared to public cloud, but that may change. Each model has their own strengths, meaning that once a production system is tuned, it is hard to justify the effort to migrate. Some systems adopt a hybrid of models and route prompts based on rules (which is the equivalent of going "multi-cloud" to reduce concentration risk in my analogy).

Prompt Techniques 🔁

Structure matters. In most models there is a meaningful difference in output between "Zero-Shot" and "Chain Of Thought" prompting. Some models are specifically designed for everything-upfront interactions, rather than a chat. Much like a human would work better solving a problem if it had some examples, hints, and a schema (e.g., XML, JSON) to work with, so do LLMs.

The book also recommends trying a "seed crystal" approach, which involves asking an LLM how it should be prompted to do a task. Think of it like "meta-prompting".

Infrastructure 🛠️

Agents are stateful and resource intensive. This constrains the infrastructure that companies can use to host them. Most serverless infrastructure will buckle under their resource usage, or time-out within 15 minutes. With the move to represent agents as workflows (e.g. Mastra, LangGraph), I do wonder if there is space to move orchestration into the infrastructure, much like we have seen with solutions like AWS Step Functions, which replaces orchestration in code. If a breakthrough were to happen in this space, it would change the cost-profile of this technology, linking it more dynamically to usage. This is much more attractive for firms with limited budgets (e.g. start-ups).

There are a multitude of ways to orchestrate and run agents, which makes it harder to move this concern to the infrastructure level.

Example multi-agent architectures. Image credit: Principles of Building AI Agents

Takeaways ✍️

Having read the book and surrounding articles, I have some principles which I would adopt if I was to build a production agentic system.

Measure 📐: Ensure that the technology solves a real problem (e.g. reducing time or cognitive load in manual processes). Make observability a day 1 concern (e.g., tool usage, task completion times). Do not rely solely on Eval scores - use metrics that describe user behaviour or feature adoption rates. Use feature flags and circuit breakers to rollback if metrics suggest a negative impact. Understand costs before the system grows.
Build Incrementally 🛠️: Prioritise slower, more accurate models when volumes are low. Constrain possible outcomes by building workflows with very small scope tasks (e.g., each agent can access at most 1 tool).
Human-First 🧍‍♂️: Keep humans in the loop. Don't make Agents the only option (Klarna can attest here). Prioritise assistance over autonomy, allowing human overrides. Avoid introducing more risks than you started with.
Security-First 🔐: Don't rely on Guardrails as the only layer of defence. Do not expose data sources containing private data (even if you think the LLM has been told not to access it). Adopt AuthN/AuthZ techniques at each layer that allows for it. Store and rotate API keys as secrets outside the agent's control. Physically isolate any data which does not serve a useful purpose for the task at hand. Monitor traffic (e.g. queries made to tools) and deny agents' tool access based on anomalous activity (e.g. request limits breached, or a spike in "permission denied" errors).

Consensus 🌍

The current state of protocols in this space feels very much like an xkcd comic. Large companies are commercially incentivised to be the first to claim a standard, which leaves them half-defined on initial publication, relying on community contributions to improve.

In more established domains, we have neutral organizations which provide trust. IANA allocates IP addresses. The CVE registry standardises tracking of security vulnerabilities. SOC2 offers a set of controls for how organizations handle security and data. We would benefit from the same kind of unbiased, centralised certification for of what makes an Agent approved for production use.

Image credit: xkcd

Summary 🧵

In this post, we reviewed the terminology and topics discussed in "Principles of Building AI Agents" by Sam Bhagwat. If this blog sounded interesting, I'd recommend downloading or buying the book directly, and following their work.

It is my first meaningful deep-dive into the engineering practices behind this technology. This space is clearly changing rapidly. If we look back on this post in 1-2 years, I would be interested to see what changes. Perhaps the majority of future use cases won't even need an LLM.

As for next steps, there are some free courses on the topic by the likes of Mastra and Hugging Face, which I may dive into. I was thinking of writing a blog on how AI is changing my day-to-day role, but this article says it far better than I could.

If you have got to the end of this post and have recommendations, differing views or stories from real systems, please do get in touch.

Get in touch 📧

As I say in my About page, I would love to hear from you. If you got to the end of this post and have anything to share, please get in touch on LinkedIn or Twitter.

Java at 30 🎂

Chris Davies — Sat, 21 Jun 2025 10:51:58 +0000

📝 This is my first blog post, cross-posted from my personal site - https://chrisdavies.dev .

In this post: the features I love most about Java in 2025, and the wishlist I’d submit for the next 30 years.

Happy Birthday? 🎈

I have written Java code for most of my professional career. On 23rd May 2025, it celebrated its 30th birthday. The language is older than I am, and has undeniably stood the test of time.

Yet, in most development circles it gets a rather negative reception. Despite infamously powering "over 3 billion devices" and providing the secret sauce for some of the world's biggest companies, it is still seen as verbose and complex.
This isn't controversial - there are enhancement proposals dedicated to making Java a nicer language to learn.

As with everything in technology, you need to use the right tool for the job, and there are paradigms that even modern Java can't live up to. That said, its recent developments have taken a lot of great features from other languages and given it a fresh lease of life.

As we celebrate 30 years of the language, this blog post will explore some features I particularly admire, as well as a "wishlist" of features I have come across in other languages that would prove a very neat JEP in a future release.

Image credit: JetBrains

Java in a Nutshell 🌰

Earlier I spoke about the "right tool for the job" mantra. Being a general-purpose, Object-Oriented Programming (OOP) language, there are many use cases that Java is great for:

Web Applications (Microservices): Servers which can cope with many thousands of requests using normal multi-threading or reactive frameworks. Simple HTTP applications can be built without frameworks, but enterprise-scale systems may benefit from using Spring, Quarkus or Micronaut.
Enterprise: Traditional industries like banking and finance rely on a rich ecosystem of support for distributed APIs, queues, cryptography and persistence (JDBC).
Mobile Applications: Benefitting from its cross-platform "write once, run anywhere" tagline, Android applications primarily run on the JVM, via Kotlin or Java.
Gaming: Applications like Minecraft and Runescape.
Big Data: Streaming technologies like Apache Kafka, Hadoop and Flink are all Java. Column stores like Apache Cassandra are also Java based. Large datasets can be explored using Elasticsearch, which is also (you guessed it!) Java.

That said, it requires more ceremony than other languages, which makes it a less popular choice in the following domains:

Latency-sensitive applications: If manual memory management is important for performance, C++ or Rust is a far superior choice. Despite recent innovations with the Java Garbage Collector (e.g. zgc) and start-up times (e.g. CRaC and GraalVM), Java suffers more than most languages due to heavy-GC pauses or cold starts in serverless environments.
Scripting: For cron jobs or CI/CD integration, just use Python, or (even better) Bash.
Data manipulation: It takes a lot of code or third party libraries to manipulate data in common formats (e.g. CSV, JSON) and apply transforms. Using pandas, numpy or scipy is more intuitive.
AI and ML: Despite recent innovations (e.g. LangChain4j), the bulk of community contributions to these fields are in Python.
Web UIs: You will be hard-pressed to find anyone willing to pick a fight for Swing or JavaFX over React, Angular or Next.js. For desktop applications, there is a bit more of a debate, though it is largely used in the IDEs themselves.

Powerful features ⚡️

It goes without saying the impact that Java 8 had in 2014 was huge. Many people still describe it as the most revolutionary release we have had in the last 15 years.

default methods blurred the line between Interfaces and Abstract Classes, but have proven very useful in the production code I have shipped, such as when rolling out a new method on an existing API.
Lambdas, Streams and Optionals pushed Java into the functional domain more aggressively than ever, where it has been solidifying its place since.

In most production code, we don't want side effects when we manipulate objects, and we want to capture data in composable but deconstructable patterns. This philosophy has given us records, sealed classes and powerful pattern matching semantics. It is great that one can now fully exhaust all options in a switch statement without falling back to a default case and blowing up.

Use of var is controversial in some circles (we weren't always duplicating type definitions both sides of the assignment). That said, IDEs' type inferences have made the transition easier to manage.

Virtual threads provide a nice solution to tackle I/O intensive applications. In today's distributed world, I/O makes up a significant portion of most blocking code. Calling an HTTP endpoint, making an SDK call, executing a database transaction. All of these situations benefit from swapping out executors for ones which spin up virtual threads rather than platform threads. Just make sure you learn from others' mistakes!

In the near future, Stream Gatherers will provide to intermediate operations what Collectors did for terminal operations. Whilst map, filter and reduce have been core staples of the Streams API for years, there are other useful execution paths. Window functions or stateful scans (e.g. moving averages or deduplication) to name a few.

I am yet to be fully convinced on unnamed variables (_) as I worry they will impact readability, but I am sure time will win me over like it did with var.

Java Version	Features
8	Lambdas, Streams, Optionals, Functional Interfaces, `default` methods
9	Modules, JShell
10	`var` keyword
11	`HttpClient`
12–16	`switch` expressions, Text blocks
17	`record`, `sealed` classes, Pattern matching with `instanceof`
21	Virtual threads, Pattern matching with `switch`
22+	Stream Gatherers, Unnamed variables (`_`), Statements before `super`

Image credit: InfoQ

Feature wishlist 💡

Despite boasting an impressive list of recent and planned JEPs, there is still room for the language to grow. Based on my experience with other languages, there are a number of features that require a lot of boilerplate to support in Java. Native support for any of these would be welcome additions.

Disclaimer ⚠️: This is my opinion as a language user working in an ideal world. I realise that most of this would require large overhauls and break backwards compatibility.

Enhanced Data Classes

Immutable data classes finally came to Java with the advent of records. With a record, we have getters, hashCode(), equals() and toString() out of the box.
This is enough to deconstruct objects into their properties during switch statements, which is coming soon. A natural extension to this would be support for validation, default values and serialisation options.
Projects like Lombok and Joda-Beans have aimed to bring this richness to Java for years, with limited adoption.

A sketch of what this could look like is below - a data class with built-in validation and explicit support for JSON serialisation. This feature would accelerate most standard API development without requiring JsonObject or Gson.

public record Person(
    @NotBlank String firstName,
    @NotBlank String lastName,
    @NotNegative Integer age) implements Serializable<Json> { }

Map Streams

Since Java 8, we have been able to stream and process data collections. This is readable and very effective for simple collections like List or Set. However, Maps are not as easy to visualise in a Stream - we must use their entrySet(), which means reasoning about a Map.Entry at every step of the stream.
OpenGamma's Strata project provides a full implementation (sketched below) of what first-class Map support in Streams could look like.

public final class MapStream<K, V> implements Stream<Map.Entry<K, V>> {

  private final Stream<Map.Entry<K, V>> underlying;

  public static <K, V> MapStream<K, V> of(Map<K, V> map) {
    return new MapStream<>(map.entrySet().stream());
  }

  public <R> MapStream<K, R> mapValues(Function<? extends V, ? extends R> mapperFn) { /* Omitted */ }
  public <R> MapStream<R, V> mapKeys(Function<? extends R, ? extends V> mapperFn) { /* Omitted */ }
  public MapStream<K, V> filterKeys(Predicate<? extends K> keyPredicate) { /* Omitted */ }
  public MapStream<K, V> filterValues(Predicate<? extends V> valuePredicate) { /* Omitted */ }

  public abstract Map<K, V> toMap();
}

Explicit Mutable Collections

Kotlin does a very good job at providing immutability by default. Users are encouraged to use val by default, before falling back to var.
By extension, users have to opt in to mutable collections with the below factory methods.

mutableListOf("1", "2", "3")
listOf("1", "2", "3")

Java collections are mutable by default. List.of(...) and Set.of(...) use unmodifiable collections but do not make this obvious enough.

Delegate Methods

It is a common pattern that we want to wrap an implementation of an interface in another. For example, a caching or rate-limiting implementation which wraps an underlying service.
This separation of concerns is extremely powerful. To follow this pattern in Java, we must define a full implementation even if the behaviour we are overriding only applies to a subset of the methods. In Kotlin we can avoid the boilerplate, by using delegates.

interface Greeter {
    fun greet()
    fun farewell()
}

class SimpleGreeter : Greeter {
    override fun greet() = println("Hello!")
    override fun farewell() = println("Goodbye!")
}

// Partially override: only 'greet' is customized
class CustomGreeter(private val greeter: Greeter) : Greeter by greeter {
    override fun greet() = println("👋 Custom Hello!")  // overrides only this
}

fun main() {
    val greeter = CustomGreeter(SimpleGreeter())
    greeter.greet()     // prints: 👋 Custom Hello!
    greeter.farewell()  // prints: Goodbye!
}

String Templates

When constructing large bodies of text, Java's multi-line text blocks have been a great addition.
However, they are only readable if the full body of the text is static. Otherwise, we are forced to use String.format(...) to interpolate values, which is hard to read when using many injected values.

String Templates were briefly previewed in the JVM, but later removed. I do hope that something like this comes to the language in a future release. In Kotlin (amongst other languages), we can place values directly where they are used.

val payload = """
{
    "firstName": "$firstName",
    "lastName": "$lastName",
    "age": "$age"
}
""".trimIndent()

List Comprehension

Despite being harder to debug in long chains of calls, declarative Streams are more often than not easier to read than the imperative equivalent.
However, a lot of Streams are a variant of map and filter (or vice versa). I do wonder if we could borrow comprehensions from Python to make this a readable single line operation.

class_id = '123'
class_info = _get_class_info(class_id = class_id)
mature_students = [person.name() for person in class_info.members() if person.age() > 21]

Tuples

There are lots of occasions where we might wish to work with a Pair or Tuple of values for data manipulation and extraction, without declaring an entire class for them. OpenGamma's Strata has Pair and Tuple for this purpose, to make up for lack of support in the language.

In Python, one can declare a namedtuple with minimal ceremony and use them within the desired scope of the application.

Account = namedtuple("Account", ["account_id", "account_name"])

def get_eligible_accounts() -> List[Account]:
    all_accounts = _fetch_accounts()
    return [Account(account_id=a.id, account_name=a.name) for a in all_accounts if a.type == 'Primary']

Named Parameters

When calling methods accepting the same type in multiple places in its signature, I believe named parameters would make life much easier and avoid ambiguity. Especially in environments where type inference is not available.

full_name = _get_full_name(first_name = "Chris", last_name = "Davies")

def _get_full_name(first_name: str, last_name: str) -> str:
    return f"{first_name} {last_name}"

Nullable Types

null references were once described as the billion-dollar mistake. Indeed, Java makes it very easy to obscure when values could be null.
Despite the best efforts of the Optional type, it just isn't suitable for use everywhere. It is a great solution when returning values, but not as we pass them around the application.

By explicitly marking an object as a nullable type variant, we can enforce checks in the IDE which tell a programmer when a null reference has not been accounted for. Various annotations exist to hint that types could be null or not null, but they are not doing static code analysis on the same level as other languages.

data class UserProfile(
    val id: Int,
    val name: String,
    val email: String?,           // Optional — user may not provide one
    val dateOfBirth: LocalDate?,  // Optional — user may skip it
    val bio: String? = null       // Optional with default value
)

fun printProfile(profile: UserProfile) {
    println("User ID: ${profile.id}")
    println("Name: ${profile.name}")
    println("Bio: ${profile.bio ?: "No bio yet"}")
    println("Email: ${profile.email ?: "Not provided"}")
    profile.dateOfBirth?.let {
        println("Age: ${Period.between(it, LocalDate.now()).years}")
    } ?: println("DOB: Not provided")
}

Typed Strings

In many APIs we want to enforce a domain over our identifier types, instead of letting them be free-form Strings.
TypeScript brings type and set theory to the forefront of the language's design, which means it can do neat things like the below. AccountId and TransactionId are still treated as string, but require very minimal declaration to bring safety to our APIs. No more passing in the wrong string values everywhere.

type AccountId = string & { readonly brand: 'AccountId' };
type TransactionId = string & { readonly brand: 'TransactionId' };

class AccountService {

  deposit(transactionId: TransactionId, accountId: AccountId, amount: number, description: string): Transaction {
    // ...
  }
}

Result Types

A large part of the negative perception Java gets is due to Checked Exceptions. It is indeed clunky that a method might bubble up an Exception through many calls, forcing application programmers to handle it.

In Go, methods may return multiple objects, which allows us to check for an Error. The code block if err != nil is commonly seen as a result. Users are explicitly forced to consider that a return type could be an error at the call site, without bubbling it up further.
TypeScript has true-myth for a similar purpose. OpenGamma's Strata has Result, which looks like the below.

A native addition to the Java language would be welcome here. In most cases we want to face errors head-on.

public class Result<R> {
  private final R value;
  private final Failure failure;

  public static <R> Result<R> of(Supplier<R> callFn) {
    try {
      R value = callFn.get();
      return new Result<>(value, null);
    } catch (Exception ex) {
      return new Result<>(null, new Failure(FailureType.ERROR, "Error executing function", ex));
    }
  }

  public boolean isSuccess() {
    return value != null;
  }
}

Decorators

This section is largely about "syntactic-sugar". With Functions, annotations and Aspect-Oriented Programming through frameworks like Spring, Java could support this kind of behaviour. However, the ability for everyday programmers to make use of those tools is limited.
Alternatively, Python supports a neat concept which allows users to wrap the execution of a method in custom logic, which can be done for abstracting performance logging/metrics, or other kinds of validation.

def timer(func):
    """Print the runtime of the decorated function"""
    @functools.wraps(func)
    def wrapper_timer(*args, **kwargs):
        start_time = time.perf_counter()
        value = func(*args, **kwargs)
        end_time = time.perf_counter()
        run_time = end_time - start_time
        print(f"Finished {func.__name__}() in {run_time:.4f} secs")
        return value
    return wrapper_timer

@timer
def waste_some_time(num_times):
    for _ in range(num_times):
        sum([number**2 for number in range(10_000)])

Aside: Why does Java still beat other JVM languages' popularity? 🤔

A natural question might be: "If Java has largely copied its recent good features from other JVM languages, why did it not get replaced by them?".

Indeed, recent polling shows that it is standing strong compared to Kotlin, Scala, Groovy etc.
An esteemed colleague of mine -- Stephen Colebourne -- summarizes this far better than I ever could wish to in his 2024 Devoxx talk.

TL;DW (Too Long; Didn't Watch)

Had Kotlin been quicker to adopt a v1.0, we might all be writing Kotlin code today instead of Java.

Summary 🧵

This post explored the state of Java at 30. From the powerful features I admire to a "wishlist" of future JEPs. With the success of the 6-month release cadence, the language keeps evolving while maintaining strong backward compatibility.

Regardless of whether you think we will be discussing its 50th or 60th birthday in years to come (Spoiler: we definitely will), it is a very impressive piece of technology. I look forward to seeing how the language grows in the next 30 years.

Get in touch 📧

As I say in my About page, I would love to hear from you. If you got to the end of this post and have anything to share, please get in touch on LinkedIn or Twitter.