DEV Community: Dinuda Yaggahavita

What Is AI Coding Observability? Visibility Before Control

Dinuda Yaggahavita — Tue, 21 Jul 2026 11:30:03 +0000

AI coding observability is the practice of measuring how engineering teams use AI coding tools — including adoption, cost, model usage, reliability and plan utilization — without inspecting developers’ source code or work activity.

That last part matters.

When I say observability, I do not mean recording every prompt, reading code or producing a leaderboard of which developer used the most tokens. I mean answering much more basic questions:

What tools are we paying for?
Are people using them?
Which models do they depend on?
What keeps failing?
Are we buying capacity we do not need?
Are local models becoming more favourable for teams

Most companies cannot answer these questions today.

That is strange when you consider how quickly tools like Cursor, Claude Code, GitHub Copilot and Codex have moved from experiments into the everyday development environment.

AI coding became infrastructure before most organizations realized they owned infrastructure.

Nobody planned the AI coding stack

Very few companies deliberately designed their current AI coding setup.

One developer started paying for Cursor. Another preferred Claude Code. A team bought Copilot seats through GitHub. Someone connected an API key to Cline or Continue. A few engineers began running open models locally.

Each choice was reasonable on its own.

Together, they created an unplanned stack of tools, models, subscriptions and accounts.

Finance can see the invoices, but an invoice does not tell you whether a tool is useful. Engineering managers know what their teams say they use, but that does not show how usage changes over time. Vendor dashboards provide some answers, but only from inside their own products.

Cursor knows about Cursor. GitHub knows about Copilot. Anthropic knows about traffic reaching Anthropic.

The organization is left assembling the wider picture manually.

This is the gap we are building UseJunction to close: one observability layer across the AI coding tools an engineering team already uses.

The problem is larger than seat waste

Unused seats are the most obvious problem because they are easy to put into a spreadsheet.

If a company purchased 100 seats and only 43 are active, someone can cancel the rest before renewal. Useful — but not particularly deep.

The more interesting question is what is happening across those 43 active users.

Perhaps a developer has an allocated Cursor seat but does most of their work through Claude Code. Another might regularly exhaust their included plan usage and quietly switch to an API key. A third might prefer a local model for a particular repository.

A vendor dashboard can make one product appear underused while the developer is actually a heavy AI user elsewhere.

That is why AI coding observability needs to cover more than licenses. At minimum, it should help an organization understand five things:

Adoption: which tools are genuinely part of the development workflow
Cost: subscription, API and infrastructure spending in one place
Model usage: which models are being used across different tools
Reliability: latency, errors, rate limits and failed requests
Plan utilization: whether paid allowances are exhausted, balanced or wasted

These measurements are related.

A drop in usage may be an adoption problem. It may also mean a tool became slow, a plan reached its limit or developers found a better model somewhere else. Looking at one metric — or one vendor — can give you the wrong explanation.

A gateway is not always the right first move

The standard enterprise response to a fragmented stack is centralization.

Route every request through one gateway. Decide which models are allowed. Block everything else. Put policy in front of the problem.

There are situations where that is necessary. If source code is being sent somewhere it should not be, the company cannot wait six months for a beautiful analytics dashboard.

But I do not think a gateway should be the automatic starting point.

A gateway changes how developers work before the organization properly understands how they work. It can introduce latency, break features that depend on a vendor’s native API and turn the platform team into the owner of another critical service.

Worse, a badly designed control layer can push usage out of sight. Developers do not stop wanting a useful tool because it disappeared from the approved list. They find another account, another API key or another route.

Before inserting control into every request, I would want to know:

Which workflows are important enough to protect? Which tools are redundant? Where is sensitive data actually going? What would break if we standardized too early?

You need evidence to answer those questions.

That is what I mean by visibility before control. It is not an argument against governance. It is an argument against governing an environment you have not yet mapped.

Observe the infrastructure, not the developer

There is a dangerous version of this product category.

Take AI usage data, attach it to individuals and pretend it measures engineering productivity.

It does not.

One developer making 500 requests is not necessarily more productive than someone making 20. They may be asking the model to repair poor generations. They may be exploring an unfamiliar codebase. They may simply work differently.

The same problem already exists with commit counts and lines of code. AI gives companies even more activity data to misuse.

UseJunction is being designed around a simpler boundary: collect the operational information needed to understand the AI system, without turning it into surveillance.

That means looking at information such as the tool, model, request status, latency and plan consumption. It does not require reading the developer’s source code or capturing the contents of every conversation.

There will always be pressure to collect more because more data looks useful on a product roadmap. I think restraint is part of building this category correctly.

If developers believe observability is secretly performance monitoring, they will avoid it. And they will be right to.

Why I ended up working on this

Before UseJunction, I built Tallei around a different version of the same fragmentation problem.

People were working across ChatGPT, Claude and other assistants, repeatedly moving context between them. The interesting part was not any single assistant. It was the behaviour across them: where information came from, what people repeatedly explained and how work moved between tools.

AI coding has the same structural problem.

The developer’s workflow does not belong to Cursor, Anthropic, OpenAI or GitHub. It runs across them.

Yet almost every dashboard is designed as if its vendor represents the whole environment.

That mismatch is what pulled me toward UseJunction.

We are starting with observability because I do not think the first version should try to become the company’s AI police, universal router and procurement system at the same time.

The first job is simpler: show engineering and platform teams what is actually happening.

UseJunction is being built as an open-source, self-hostable system because some organizations will reasonably refuse to send this data to another external SaaS product. Even operational metadata can reveal meaningful information about engineering activity. Teams should be able to decide where that data lives.

What UseJunction should tell you

Imagine renewal season is approaching.

You have Cursor, Copilot and Claude subscriptions spread across several teams. Some developers also use Codex, Cline, Continue or local models. Finance wants to know what can be cancelled. Engineering does not want a cost-cutting exercise to remove tools people genuinely depend on.

Today, this usually turns into a combination of vendor exports, surveys and guesswork.

UseJunction should let you see the environment as one system:

Which tools are active?
Which plans are approaching their limits?
Where are paid seats sitting unused?
Which teams are seeing failures?
Are developers moving to another model when one becomes unreliable?
Are local models becoming a meaningful part of the stack?

The answer may be that the company should standardize.

It may also be that different teams genuinely need different tools.

Observability should not begin with a preferred conclusion. Its job is to make the trade-offs visible.

Start with the boring questions

A company does not need an elaborate AI governance programme to begin.

Start with the questions that should already have answers:

How many AI coding products are we paying for?
Who has access?
Which ones have been active recently?
What portion of each plan is being consumed?
Are API costs growing outside the subscription budget?
Which services are regularly slow or unavailable?

Then define what you will not collect.

That second step is easy to ignore, but it matters. If the system does not need source code or prompt contents to answer an operational question, do not collect them by default.

Once the environment is visible, decisions become less theatrical.

You can cancel genuinely unused seats. Increase capacity where developers repeatedly hit limits. Investigate unreliable providers. Decide whether a central gateway would solve a real problem. Introduce policies around specific risks instead of applying one broad restriction to every team.

The point is not another dashboard

The world does not need a prettier collection of token charts.

The useful outcome is being able to make decisions about AI coding infrastructure without relying on vendor claims, employee surveys or whoever has the strongest opinion in the meeting.

That is the product we are trying to build with UseJunction: an open-source observability layer for AI coding tools, models and plans.

The product begins with visibility. Over time, that visibility may support routing, configuration and policy controls. But those controls should be built on evidence from the real environment — not assumptions about how developers are supposed to work.

If your company already uses several AI coding tools, you already have an AI coding stack.

The only question is whether you can see it.

P.S. You can self host it too: Github Repo

I Built Tallei to Stop Repeating Myself Across AI Tools

Dinuda Yaggahavita — Thu, 23 Apr 2026 14:36:42 +0000

I built Tallei because I was tired of repeating myself to AI.

My workflow was already spread across multiple tools. I would:

Brainstorm in Claude
Generate images and graphs on ChatGPT
Gather & analyze in Perplexity
Argue with the content on Gemini 😅

Every switch came with a cost -> lost context & time wasted.

I had to re-explain the project.
Re-state my preferences.
Re-summarize decisions that had already been made.
Re-upload documents or re-copy the important parts.

The assistants were getting better. The workflow between them was getting worse.

So I built Tallei — a cross-AI memory layer that sits underneath the tools I already use and helps them work from the same context. Not another chat app. Not a replacement for ChatGPT or Claude. A shared memory system that makes those tools feel less disconnected.

The problem was not intelligence. It was continuity.

A lot of people talk about which model is smarter.
That was not the pain I felt.
The real pain was continuity.

I could have a great working session in one assistant and then lose the thread completely when I moved to another. What mattered was not just text. It was the state of the work:

what I was trying to do
what had already been decided
what documents mattered
what constraints were important
what kind of responses I preferred

That state kept dying at the border between tools.

Tallei started from a simple idea: your context should not disappear just because you switched AI assistants.

How Tallei can help you

Tallei is designed to store and retrieve the parts of context that are worth carrying forward across tools.

That includes things like:

preferences
facts
decisions
recent notes
document-related context

The goal is simple:

reduce re-explaining
keep projects coherent across assistants
make handoffs between tools smoother
let documents stay useful after the first chat window is closed

If you only use one assistant, native memory may be enough.
But if your work spans multiple tools, you start to feel this gap very quickly.

The first version worked, but barely

The first version of Tallei proved the idea, but the experience was rough.

I had the basics in place:

a backend
authentication
memory save and recall
an MCP server
a dashboard
integrations for multiple assistants

Technically, it worked.
But when I tested recall properly, it took over **30 seconds** in the early version.

That is unusable for a product like this.
Memory cannot feel like a background report. It has to feel immediate. If recalling context takes that long, the user would rather just manually explain things again.

That was the first big product lesson: this could not just be “smart.” It had to be fast enough to disappear into the workflow.

1. My first big mistake: I optimized for speed and made it worse

When I started fixing latency, I made a mistake that taught me a lot about memory products.
I got recall much faster.
But I also made it less trustworthy.

The faster version sometimes returned the wrong memory first, especially on new or slightly different queries. The system looked impressive on latency, but it was no longer reliable. And for a memory system, that is deadly.

A fast wrong answer is worse than a slower correct one.
Because once the user stops trusting recall, the whole product starts falling apart. They hesitate. They second-guess what is injected. They stop depending on the system.

That was the moment I stopped thinking only about speed and started thinking more clearly about trust.
The recall system that actually worked
The version that finally felt right was simpler.

Instead of trying to be overly clever, I rebuilt recall around a few practical buckets of context:

preferences
long-term memory like facts and decisions
short-term memory like recent notes and events

That structure worked better because it reflected how context is actually in our brains.

Some things should almost always be available, like preferences.
Some things matter over time, like key decisions.
Some things are recent and situational, like what happened in the last few days or weeks.

The goal was not to create the most complicated retrieval system possible. The goal was to reliably bring the right categories of context into the conversation without flooding it.

That made Tallei feel much more stable.

It also led to a much better outcome on performance, while keeping accuracy intact. The “dump-all when it fits, retrieve selectively when it does not” model ended up being much more practical than forcing heavy retrieval logic on every request.

Documents were one of the hardest parts

One of the biggest things I realized while building Tallei was that memory is not just about facts like “I prefer concise answers.”

Documents matter a lot more than that.

People want assistants to remember:

the PDF they uploaded
the contract they reviewed
the research they already summarized
the notes from a brainstorm
the key takeaways from a long session

That is where a lot of the handoff pain really lives.

So Tallei handles documents in two ways:

01. Document notes

These are lighter-weight summaries, key points, or takeaways.
They are useful when you want the important context from a document without dragging the whole document into every conversation.

02. Document blobs

These are for the full source material when you need more complete archival context.

On top of that, each document gets a @doc: reference, and documents can be grouped into lots with @lot: references. That makes it easier to organize related material and carry it forward in a structured way instead of treating every file like an isolated upload.

03. Document lots

These are multiple documents referenced and usually used together. This was really useful since people usually reference an average 2 documents per AI query, and running and finding 2 are costly and takes up memory. We need to make sure to not overbloat the context.

This was important for the product because real work is rarely just one memory at a time. It is often a set of connected documents, notes, and decisions.

2. Another mistake: I built things because they sounded smart

At one point, I built a graph layer to map entities and relationships across memories.
It sounded like exactly the kind of thing a memory product should have.

In practice, it added complexity without enough value.

It made the system heavier, introduced messy entity labeling problems, and did not improve the actual user experience enough to justify keeping it. So I removed it.

That was another important lesson from building Tallei:

not every advanced feature belongs in the product just because it sounds intelligent.

Sometimes the better product decision is to remove complexity and focus on what actually helps users keep their thread across tools.

Why I think this matters

We are not another all-AI chat router interface✌️.

That means the moment your workflow becomes multi-tool, you start feeling the missing layer.

That missing layer is what I wanted Tallei to address.

Not by replacing the assistants, but by helping them feel connected, and not distrupting your flow.

I do not think the future of AI work is one assistant doing everything. I think a lot of real workflows will stay spread across multiple systems, each better at different moments. If that is true, then shared context becomes important infrastructure.

Open source matters here too

Tallei Github Repo
Tallei is also open source, which was important to me from the start.

If you are building a memory product — something that stores preferences, facts, notes, and document context — people should be able to inspect how it works. Open source helps make the architecture, trade-offs, and implementation more transparent.

It also makes it easier for others to understand that this is not just a vague AI promise. It is an actual system with real product decisions behind it.

Who this is for

Tallei is for people whose AI workflow already spans more than one tool.

That could be:

founders moving between strategy, research, and execution
developers using different assistants for architecture, debugging, and building
writers switching between ideation, editing, and fact-checking
researchers comparing outputs across models
anyone tired of pasting the same context over and over again

If that sounds familiar, then you probably already understand the problem Tallei is trying to solve.

Final thought

I did not build Tallei because I thought the world needed one more AI product.

I built it because I personally felt the pain of context breaking between assistants, over and over again.

The core idea stayed simple the whole way through:

your context should survive the handoff within your AI tools

That means preferences should carry over.
Documents should stay useful.
Important decisions should not disappear.
And switching tools should not mean starting from zero.

That is what Tallei is built for.