You Can't Add Up AI Agent Wait Time: Measure Overlapping Runs Instead

I verified the measurement design, the rounding boundary, the sample output, and the conclusion in the Japanese source. AI drafted this English adaptation, reorganized it for dev.to, generated the diagram, and reran the sample code. #ABotWroteThis

Quick answer (TL;DR): How to measure AI agent wait time

When parallel AI runs overlap, adding their wait durations can exceed the time that actually passed. Export one start and completion timestamp per request, treat each request as an interval, and compute gross, union, overlap, and parallelism separately.

That mismatch is not a calculation bug. It occurs because overlapping intervals are counted more than once.

The mistake is treating “time spent waiting for AI” as if it were one measurement. It is not. At minimum, you need to separate:

• the sum of every agent's wait interval
• the elapsed time covered by at least one wait
• the amount of overlap
• average and maximum concurrency
• requests that were still running when you collected the data
• the time the human was actually blocked

Those values answer different questions. Compressing them into one total makes the result hard to explain and easy to misuse. Keep unfinished requests right-censored instead of mixing them into completed-duration averages, and do not equate AI runtime with human labor time.

For your first sample, keep unfinished requests with end=None, then run the interval code below.

Why activity trackers miss AI wait time

I started this investigation by comparing long AI response waits with an editor-based activity tracker.

The overlap was effectively absent. That result makes sense: while an agent is working, I am often not touching the editor. The tracker I used stops counting after an inactivity threshold, so an agent can be running while that tracker reports no active work. As a documented example of this design, WakaTime says its plugin stops tracking after the configured keystroke timeout.

There is a subtle distinction here. The tracker may still contain an event inside the same period, but that does not mean it counted the entire interval as active time. You have to compare the computed activity spans, not merely ask whether any event exists.

Extending the tracker's inactivity threshold would make it count more of the wait, but that changes the definition of “active work.” It does not solve the separate problem of measuring agent execution.

This creates an important divergence:

More AI-agent runtime can exist without being added to activity-tracker time.

Neither number is necessarily wrong. They measure different systems.

Why adding every wait interval breaks

Suppose agent A runs from minute 0 to 20 and agent B runs from minute 10 to 40.

Adding their durations gives 50 minutes. The clock advanced only 40 minutes. The extra 10 minutes is overlap.

Parallel agent workflows make this normal. While one request is running, you can start another. The more work overlaps, the less useful the raw sum becomes as a measure of elapsed time.

I use four separate metrics:

Metric	Definition	What it tells you
gross	Sum of all wait intervals	A proxy for total agent workload, not elapsed time
union	Length of the timeline covered by at least one wait	How much clock time contained agent activity
overlap	gross - union	How much execution overlapped
parallelism	Number of active waits over time	Average and peak concurrency

The union is computed by sorting intervals by start time and merging those that overlap. Parallelism can be computed with a sweep line: convert each start into +1, each end into -1, sort those events, and accumulate the active count over time.

The sort dominates the cost, so the algorithm is O(n log n). You do not need to compare every interval with every other interval.

A runnable Python example

The following example uses rounded, fictional offsets. They are not my production measurements.

Save the code as measure_waits.py, then run it with:

python measure_waits.py

from dataclasses import dataclass
from typing import List, Optional, Tuple


@dataclass
class WaitInterval:
    start: float          # Minutes from an arbitrary origin
    end: Optional[float]  # None means still running at collection time


def resolve(intervals: List[WaitInterval], until: float) -> List[Tuple[float, float]]:
    """Clip unfinished intervals at the collection time."""
    return [(iv.start, iv.end if iv.end is not None else until) for iv in intervals]


def gross_minutes(closed: List[Tuple[float, float]]) -> float:
    return sum(end - start for start, end in closed)


def union_spans(closed: List[Tuple[float, float]]) -> List[Tuple[float, float]]:
    """Merge overlapping intervals into a minimal non-overlapping set."""
    ordered = sorted(closed)
    merged: List[Tuple[float, float]] = []
    for start, end in ordered:
        if merged and start <= merged[-1][1]:
            merged[-1] = (merged[-1][0], max(merged[-1][1], end))
        else:
            merged.append((start, end))
    return merged


def union_minutes(spans: List[Tuple[float, float]]) -> float:
    return sum(end - start for start, end in spans)


def parallelism(closed: List[Tuple[float, float]]) -> Tuple[float, int]:
    """Return time-weighted average and peak concurrency during waits."""
    events: List[Tuple[float, int]] = []
    for start, end in closed:
        events.append((start, 1))
        events.append((end, -1))
    events.sort()

    count = 0
    max_count = 0
    weighted = 0.0
    active = 0.0
    previous_time = events[0][0]

    for time, delta in events:
        elapsed = time - previous_time
        weighted += count * elapsed
        if count > 0:
            active += elapsed
        count += delta
        max_count = max(max_count, count)
        previous_time = time

    return (weighted / active if active else 0.0), max_count


intervals = [
    WaitInterval(0, 20), WaitInterval(10, 40), WaitInterval(30, 40),
    WaitInterval(120, 150), WaitInterval(130, 210), WaitInterval(200, 230),
    WaitInterval(460, None), WaitInterval(470, None),
]
collected_at = 480.0
closed_all = resolve(intervals, collected_at)

gross = gross_minutes(closed_all)
spans = union_spans(closed_all)
union = union_minutes(spans)
average_parallelism, max_parallelism = parallelism(closed_all)

print(f"gross: {gross:.0f} min")
print(f"union: {union:.0f} min")
print(f"overlap (gross - union): {gross - union:.0f} min")
print(
    "parallelism during waits avg/max: "
    f"{average_parallelism:.1f} / {max_parallelism}"
)

gross: 230 min
union: 170 min
overlap (gross - union): 60 min
parallelism during waits avg/max: 1.4 / 2

In this fictional example, gross wait time is about 230 minutes, but only about 170 minutes of the timeline contained any wait. Roughly 60 minutes were counted more than once because requests overlapped.

That does not prove that the workflow was efficient. More concurrent work naturally creates more overlap. Whether that overlap produced better output is a separate question.

Do not mix unfinished waits into completed durations

Some requests will still be running when you collect the data. Their final duration is unknown.

If you clip those intervals at collection time and then mix them into the completed-duration average, you pretend they ended at that moment. A request that started shortly before collection appears artificially short. Depending on when open requests began, the same mistake can also push an average upward. The result changes with the arbitrary collection time.

This is a standard right-censoring problem. Keep at least two views:

1. For timeline occupancy, you may clip open intervals at collection time so the union has a defined endpoint.
2. For request-duration statistics, report completed requests separately and list open requests as right-censored.

Request state	Timeline union	Completed-duration average
Completed	Include the full interval	Include
Still running	Clip at collection time	Exclude and report as right-censored

Do the same with synthetic test runs, abandoned retries, and sessions replaced by a later run. Decide the exclusion rules before computing the metric, or the number will drift each time you rerun the analysis.

Wait time is not human blocked time

Even the union does not tell me how long I was personally blocked.

During an agent run, I may be watching the screen, working in another application, starting another agent, or away from the desk. Request logs alone cannot distinguish those states.

The union is therefore an upper frame for possible human waiting, not a measurement of human labor. Measuring actual blocking would need another data source, such as foreground-window transitions, completion-notification interactions, or explicit task-switch events.

I did not have that evidence, so I do not claim to have measured it.

This boundary matters because a large gross value can sound like an impossibly long workday. It is only impossible if you silently redefine concurrent agent runtime as human time.

A minimal measurement workflow

If you want to implement this for your own agents, start here:

1. Identify one event that marks the start of a request and one that marks its completion.
2. Store each request as (start, end). Keep end=None when it is still running.
3. Compute gross, union, overlap, and parallelism as separate functions.
4. Keep completed durations and right-censored requests in separate datasets.
5. Define how to exclude retries, synthetic checks, and replaced sessions.
6. Run the analysis twice against the same input and confirm identical output.
7. Preserve the raw values privately, then round only the figures you publish.

Also decide the aggregation window. Daily windows need a rule for intervals that cross midnight. Weekly windows can hide peaks. Comparisons only make sense when the window and exclusions stay consistent.

Do not evaluate AI adoption by time used

This measurement changed how I think about AI-use evaluation programs.

“Hours of AI use” and “number of parallel agents” are weak performance indicators. Gross runtime can be increased simply by launching more work. That does not show that output improved, human effort fell, or the result needed less rework.

I would rather evaluate what changed because AI was used:

• quality and quantity of useful output
• decisions that still required human judgment
• rework and escaped defects
• actual human blocked time, when it can be measured
• cycle time from request to verified outcome

Agent runtime can still be an operational metric. It helps with capacity planning, rate limits, and cost analysis. It should not be mistaken for labor or impact.

The main lesson is simple: measure overlapping runs as intervals, and keep agent activity, elapsed time, human attention, and outcomes in separate columns.

FAQ

Can I use gross wait time as my total work time?

No. Gross counts overlapping requests more than once and measures agent workload, not human labor. Use union for elapsed timeline occupancy, then measure human blocking separately if you have the evidence.

Should an unfinished AI request be included in the average duration?

No. Keep it right-censored until it completes. You may clip it at collection time for a timeline union, but do not mix that clipped value into completed-request averages.

Does more parallelism mean the workflow is more productive?

Not by itself. Parallelism describes concurrency. Compare it with verified output, rework, defects, and human blocked time before treating it as an improvement.