DEV Community: Jay Ahuja

Your Teams Can Work Differently. Your Engineering Data Shouldn't.

Jay Ahuja — Mon, 21 Sep 2026 20:43:03 +0000

The Enterprise Reporting Problem

A board-level question can sound deceptively simple:

Is our investment in AI delivering results?

Answering it across a large engineering organization can be anything but simple. Imagine fourteen teams working across three Jira instances. One of those instances came through an acquisition that is still being integrated. Most teams use GitHub, while the platform organization uses GitLab. There are four CI systems because different engineering leaders made different technology choices over the years.

None of these decisions are inherently wrong. The problem begins when leadership asks for one answer across all of them. Suddenly, the organization needs analysts, custom scripts, exports, spreadsheets, and weeks or months of reconciliation.

Eventually, a number emerges. But nobody is entirely comfortable defending it.

That is the question tax.

A question that should take minutes becomes an engineering analytics project.

CleverDev solves that.

The Root Cause Isn't Tool Sprawl

The obvious diagnosis is usually:

We have too many tools.

Therefore:

We need fewer tools.

But that isn't necessarily what causes reporting to break. The deeper problem is that different teams assign different meanings to the same metric.

Cycle time is a perfect example. One team starts measuring when the ticket is created. Another begins at first commit. A third uses a workflow with an “In Refinement” state that can legitimately last several days.

All three teams can defend their methodology. All three may be producing accurate data.

But when those numbers are rolled into one enterprise average, the result isn't necessarily meaningful. You're combining measurements that don't represent the same thing. The reporting exercise becomes difficult because someone has to resolve these semantic differences manually.

And when the exercise is repeated, those decisions may be made differently.

A dashboard can standardize presentation without standardizing meaning.

The Temptation to Force Everyone Onto One Stack

The usual answer is an enterprise consolidation initiative.

Standardize Jira. Standardize CI. Standardize workflows. Create a universal definition of done.

On paper, this sounds efficient. In practice, it introduces a different set of costs.

Consolidation Takes Time

Changing tools across many teams is not simply a technology migration.
It requires teams to reconsider how they work.
That can take years.

Standardization Can Remove Useful Autonomy

A team may have a unique workflow because its technical environment genuinely requires it.
Forcing that team into a common process solely for reporting convenience can make engineering less effective.

The Environment Will Change Again

Even a successful consolidation won't remain static.
Acquisitions happen.
New platforms emerge.
Teams have new requirements.
Exceptions become necessary.

The result is an ongoing enforcement burden. And even with one tool, teams can still use it differently.

Standardized tooling does not automatically produce standardized meaning.

The Better Layer to Standardize

There is a layer underneath process where engineering teams become much more comparable.

Events.

A commit happened.
A review happened.
A build executed.
A deployment occurred.

These events are not opinions about how engineering should work. They're records of what actually happened. That's where standardization becomes valuable.

Teams should be free to decide:

How they manage backlogs
How many review states they need
How they structure branches
How they organize releases

Those choices can remain local. What changes is how the resulting events are represented for enterprise analysis.

Standardize the record, not the workflow.

Creating a Common Event Model

When a source is connected to CleverDev, its events are mapped into a shared event model. A team's Jira states can be translated into the common model. A CI system's stages can be mapped into the same structure. A different tool can use different terminology while still contributing consistent underlying events.

This happens once at the source connection.

The team's workflow doesn't need to change. No migration is required. No developer needs to adopt a new process.

From there, metric definitions can be centralized. Cycle time becomes a platform-level definition rather than a team-specific interpretation. The organization can ask the same question across teams and get an answer derived using the same underlying model.

And when another source is added, it gets its own mapping.

The existing environment doesn't need to be redesigned.

Don't Fill the Gaps With Assumptions

A good measurement system also needs to be honest about what it doesn't know.

Suppose a team doesn't record the workflow state required to calculate review latency.

The answer shouldn't be fabricated. It shouldn't be silently excluded.

Instead, the limitation should be explicit:

The organization cannot report review latency because that state isn't captured.

That kind of transparency is valuable. It tells leaders exactly what is measurable and what isn't.

A visible data gap is better than a misleading enterprise metric.

No Migration Program Required

This model changes what enterprise reporting requires.

You don't need to migrate every repository. You don't need to force every team onto the same Jira instance. You don't need a committee to create a universal process. You don't need to spend eighteen months consolidating tools only to discover that teams still interpret the data differently.

CleverDev works with the environment that exists today. An acquired organization can be connected as-is. An unusual CI pipeline can be mapped as-is.

When the technology estate changes in the future, the new source can be mapped without rebuilding everything around it.

There are legitimate reasons to consolidate tools:

Security may be one.
Licensing may be another.
Operational simplicity can be another.

If those are the goals, consolidation can still make sense.

But if the goal is simply to make enterprise engineering visibility possible, you don't need to standardize the teams to standardize the data.

AI Has Made the Old Reporting Model Too Slow

Historically, organizations could tolerate fragmented reporting because leadership questions were often periodic. A quarterly review could justify a quarter of analysis.

That model is becoming obsolete.

AI is accelerating the entire development lifecycle. Requirements are created faster. Code is produced faster. Testing can happen faster. Delivery moves faster.

But human judgment remains a scarce resource. That means engineering leaders increasingly need answers continuously.

Is this release safe?
Is AI improving our productivity?
Is a dependency putting our delivery date at risk?

A reporting process that requires months to answer those questions is fundamentally out of sync with the pace of modern engineering.

Continuous questions require continuous intelligence.

Otherwise, the question tax becomes so high that leaders eventually stop asking for the data. And when that happens, organizations return to instinct.

Two Questions, One Foundation

The questions engineering leaders need answered are increasingly clear:

Can we trust this code enough to ship it?

Can we prove that AI is making engineering faster?

CleverDev is built around providing the underlying intelligence needed to answer both. It connects to the tools engineering organizations already operate. It captures events as they happen. And it creates the lineage that connects those events across the lifecycle.

That means understanding:

Why code was created.
Why it changed.
Who changed it.
What it affected.

The teams don't need to work the same way. They simply need their engineering events represented in a consistent underlying model.

That is how organizations gain enterprise visibility without sacrificing team autonomy.

The Outcome

The goal is not another consolidation initiative. It is not another dashboard.

It is a consistent engineering record that works across the organization you already have.

Different teams. Different tools. One underlying view of engineering reality with CleverDev.

Is AI Really Making Your Engineering Team More Productive?

Jay Ahuja — Fri, 18 Sep 2026 11:54:06 +0000

Introduction: AI Adoption Is Easy. Proving Its Impact Is Hard.

AI coding tools have been part of your engineering organization for a while now.

There probably wasn't a lengthy pilot followed by months of evaluation. Engineers started using them, adoption spread, and before long AI coding assistance became part of everyday development.

Then the investment appeared on the budget.

And eventually, someone in the boardroom asked the obvious question:

What did we actually get for it?

The dashboards probably have plenty of answers.

You can see how many suggestions were accepted, how much code was generated, how many seats are active, and how adoption has grown.

Those numbers may all be accurate.

But they don't answer the question that matters:

Did your organization actually deliver more?

That is the question behind CleverDev Productivity.

The goal isn't to measure how enthusiastic engineers are about AI tools. It is to measure whether those tools translated into better delivery outcomes.

See more about CleverDev here.

The Problem With AI Activity Metrics

Most AI coding tools are very good at telling you how often they are being used.

They can report:

Suggestions accepted
Lines of code generated
Active seats
Adoption rates

These are useful engagement signals.

But engagement isn't ROI.

A tool can be heavily adopted without producing a measurable increase in delivered software.

That distinction becomes particularly important when the person reviewing the investment is a CFO or board member.

Engineering analytics platforms introduce another limitation.

Many of them work through periodic snapshots. They collect information from engineering systems on a schedule and report what changed between those points.

That model works reasonably well when the question is:

“What happened last quarter?”

But AI investment creates a different question:

“Did delivery improve after we introduced AI?”

Answering that requires looking at change over time, not simply viewing isolated snapshots.

The result is an uncomfortable situation:

Engineering leaders have more dashboards than ever, yet may still lack a defensible answer about AI's actual business impact.

Measure Delivery, Not AI Usage

If the objective is to determine whether AI is making engineering more productive, the measurement should focus on delivery outcomes.

Three measures are especially important:

Throughput: Did the organization actually ship more?
Rework: Is less engineering work having to be repeated or redone?
Quality: Are teams delivering without increasing the number of defects?
These measures focus on outcomes rather than activity.

They tell you what changed in the delivery system—not simply how much an AI tool was used.

There are also business-level outcomes that help validate the picture:

Were epics actually completed?
Were commitments met?
Did incident rates remain stable?

A productivity improvement that increases throughput while simultaneously increasing production incidents isn't necessarily productivity.

It may simply be moving the cost of engineering work into a later quarter.

That is why throughput, rework, and quality need to be considered together.

And this is also why AI suggestions accepted, lines generated, and seats adopted should not be treated as primary ROI measures.

They tell you whether the tool is being used.

They don't tell you whether the organization delivered more.

Why CleverDev Doesn't Claim to Know Which Lines AI Wrote

There is a tempting approach to AI productivity measurement: determine which code was generated by AI and which was written by humans.

It sounds precise.

In practice, it isn't.

Modern software development is too collaborative and iterative to draw a reliable line at the individual code level.

An engineer might ask AI for an initial implementation, rewrite much of it, combine it with existing code, refactor it weeks later, and modify it again as requirements evolve.

Where exactly does AI contribution begin and end?

Any percentage assigned to that process would involve assumptions. CleverDev doesn't pretend otherwise.

We don't attribute individual lines or pull requests to AI versus humans.

Instead, we measure what can be measured reliably: How did the organization perform before AI adoption, and how does it perform now?

That is a temporal comparison.

Your organization had a delivery baseline before AI.

It has a delivery state after AI adoption.

The difference between those states can be measured using actual engineering outcomes.

It's a before-and-after comparison of your organization—not a guess about which lines an AI model generated.

See how CleverDev can help you to improve your teams productivity.

What If AI Arrived Before You Started Measuring?

This is a common concern. Maybe your organization adopted AI two years ago and never established a formal baseline.

Does that mean the opportunity to measure its impact has passed?
No.
Your baseline already exists. It lives inside the engineering history your organization has been creating all along. Git contains historical development events. Your ticketing system contains work history. CI pipelines contain execution records.
Incident systems contain production outcomes. These systems already contain the story of how software moved through your organization before AI adoption.

CleverDev can use that history to reconstruct the baseline.

The important distinction is:

Reconstruction, not recollection.

You don't need to ask teams to remember how productive they were two years ago.

You don't need to rely on an old spreadsheet or a subjective estimate.

CleverDev goes back to the events that were actually recorded and calculates the same delivery measures from that historical data that it calculates today.

Because the baseline is derived from engineering events, it can also be examined by:

Team
Repository
Quarter
Program

The result is a measurable comparison based on your own history.

And instead of waiting two quarters for enough new data to accumulate, the comparison can be available in days.

Why Events Matter More Than Snapshots

The architecture behind this approach comes down to one important principle:

Metrics can be calculated from events. Events cannot be reconstructed from metrics.

If you continuously capture what happened, you can calculate different measures from those events later. You can create a snapshot for any point in time. You can define a new metric after the data was collected. You can go back and analyze historical performance using a different lens. But if you only captured periodic snapshots, the activity between those snapshots is gone. Imagine a pull request opened on Monday and approved on Friday. A periodic snapshot might tell you that it existed and eventually merged. It won't necessarily tell you whether it moved smoothly through review, was rejected, rewritten, reverted, or reviewed multiple times.

The important engineering story happened between the snapshots.

CleverDev captures engineering events continuously and connects them through lifecycle lineage.

That lineage helps explain:

Why code was created
Why it changed
Who changed it
What it affected downstream

This event-based architecture is what makes historical reconstruction possible. Click here to see more about CleverDev platform

AI Can Make Coding Faster Without Making Delivery Faster

There is another reason AI productivity gains can appear smaller than expected.

AI can make individual tasks faster. But an organization isn't simply a collection of coding tasks. A developer may finish their work faster and still be blocked by:

Another team's dependency
An integration issue
A pending decision
A handoff
A process bottleneck

When coordination is the constraint, faster coding doesn't necessarily move the delivery date. The organization simply accumulates completed work in front of the bottleneck. That can make an AI investment appear ineffective even when the AI tools themselves are working exactly as intended. The missing gain was absorbed by the delivery system. CleverDev surfaces these cross-team execution risks while there is still an opportunity to address them.

What CleverDev Productivity Gives You

CleverDev connects with the engineering systems your teams already use, including:

Planning and issue-management tools
Source control
CI/CD
Testing
Deployment
Incident and operations systems Your teams don't need to change how they work. Developers don't need to install additional software or modify their workflow. Individuals aren't scored. The focus is on understanding organizational delivery outcomes.

CleverDev is also designed with enterprise data boundaries in mind. Your engineering events and resulting dataset remain inside your network, while AI capabilities can run on your own LLM infrastructure. Nothing needs to cross your perimeter. This matters because AI adoption has already created significant data-exposure concerns for enterprise organizations. A productivity measurement platform should not introduce another layer of risk.

Prove the Value on Your Own Data

You don't need to make a company-wide commitment to find out whether CleverDev can answer your question. Start with one team or one program. First, agree on what “worth it” means. Define the question and the standard the answer needs to meet. Then connect CleverDev to the tools you already use. CleverDev reconstructs the historical baseline from your own engineering events and compares it with current delivery performance. Within a day or two, you can begin looking at the answer using your own data.
No vendor-generated benchmark. No assumptions about which lines AI wrote. No waiting months to establish a baseline.

Just evidence from your own engineering organization.
The decision is then yours to make.
Click here to talk to a CleverDev Engineer