Four times a year, a public company gets on a call with analysts and says what it intends to do next. A margin it expects to reach. A plant it expects to commission. A growth rate it expects to hold. These are the most concrete forward statements a company makes outside a filing, and they are made on the record, with a date attached.
Almost nobody stores whether they were closed out.
We built a store. As of 4 September 2026 it holds 24,328 forward commitments made by 1,648 Indian listed companies, extracted from 8,220 parsed earnings-call transcripts. Each commitment carries a verdict describing what later calls did with it.
The interesting engineering is not in the extraction. Pulling a forward statement out of a transcript is a solved-enough problem. The interesting engineering is that the outcome of a row lives in a document that does not exist yet, and that the most common failure mode of that outcome is not a wrong value — it is nothing at all.
This post is about what that does to a data model, and to every rate you compute from it.
The shape of the problem
A naive schema treats a transcript as the unit: one call in, N statements out. That schema cannot answer the only question anyone actually has, which is what happened to the thing they said.
So the unit is not the statement. The unit is the commitment: a durable entity that is born on
one call, carries a current state, and gets re-evaluated every time that company speaks again. It
needs a stable identity, the document it came from, the document it was last judged against, and a
horizon — because "we'll get there by FY27" and "we'll get there next quarter" are not the same
row even when they read alike.
That shape turns the job from classification into longitudinal record linkage: for each open
commitment, does this new call say anything about it, and if so, what?
That framing is the whole post. Two consequences fall straight out of it, and both of them bit us.
Consequence 1: your label set is not binary, and pretending otherwise is a choice
The verdict vocabulary we ended up with has twelve values. Here is the actual distribution across all 24,328 rows:
| Verdict | Count |
|---|---|
| new | 7,486 |
| on_track | 5,203 |
| achieved | 3,558 |
| missed | 2,380 |
| ghosted | 2,271 |
| at_risk | 880 |
| delayed | 800 |
| revised_down | 638 |
| revised_up | 597 |
| achieved_diluted | 376 |
| diluted | 137 |
| achieved_delayed | 2 |
Nobody wants twelve numbers. Everybody wants one: how often do they do what they said? Getting from the table to the number is a mapping decision, and the mapping is where an honest dataset goes wrong quietly.
revised_up is a company raising its own bar and then clearing the higher one. Is that kept? achieved_diluted is hitting the number after the target was watered down. Is that kept? at_risk is management itself flagging trouble before the deadline — that is not a miss, it is a disclosure.
There is no correct answer. There is only a stated one. If you publish a rate derived from a multi-class vocabulary, publish the mapping in the same sentence, or don't publish the rate. We learned this the ugly way. A published FAQ of ours defined our kept rate with a shorter mapping than the one the code actually used — the code folded in classes the FAQ's sentence never mentioned. Nobody typed a wrong number. Somebody wrote down a definition from memory, and the definition and the query drifted apart while both kept working.
Ours is the one the code computes, stated in full: achieved + achieved_diluted + on_track + revised_up, over commitments old enough to have resolved — 57.8%. Note what is not in that denominator, which is the subject of the next section but one.
Consequence 2: the label that means "nothing happened"
ghosted is the row that made this project interesting and the row that makes it dangerous.
It means: the commitment was raised on one call and was never mentioned again on any subsequent earnings call in the transcripts we parsed. Not achieved, not missed, not revised, not withdrawn. It simply stopped being part of the company's account of itself.
There are 2,271 of them, and 1,000 of the 1,648 companies in the corpus have at least one.
An absence label breaks three assumptions you did not know you were making.
It is a statement about your corpus, not about the world. Our corpus is earnings-call transcripts. A company can address a target in a press release, an investor day, an annual filing or a conference and our pipeline will never see it — and the commitment will still read ghosted. That is not a bug, it is the definition, and it means the field name is a lie unless the documentation carries the scope with it. We hit exactly this case while drafting an article: our ledger marked a target ghosted because no later call closed it out, and the company had addressed the milestone publicly, off-call, where our corpus cannot see. Our verdict was correct about our corpus and would have been false as a sentence about the company. We pulled the example.
If you take one thing from this post, take that: an absence label needs its observation boundary attached to it at every point of use, including in the column name if you can get away with it. never_mentioned_again is a worse column name than never_mentioned_again_on_a_later_call_in_corpus, and the second one is the one that stops somebody writing a false sentence eighteen months later.
It is manufactured by recall failures. Every commitment your linker fails to match in a later call becomes a ghost. A miss in detection does not produce a missing value — it produces a confident wrong verdict, which is much worse. Ghosting rate is therefore the metric most sensitive to linker recall, and the only class where a false positive is silent. We treat the transition into ghosted as the highest-cost transition in the system, and it is the one place where a non-detection is worth a second pass rather than a write.
It cannot be computed until enough later calls exist. Which is the third consequence, and it deserves its own section.
How we actually link a commitment to a later call
Everything above assumes the hard part is solved. It is not, and this is the section I would
have wanted from someone else's post, so here is ours, mechanism first.
The naive implementation is one prompt: here is a transcript, here are 30 open commitments for
this company, tell me which ones it addresses. It works in a demo and it quietly destroys the
dataset, because every commitment the model fails to mention in its answer becomes a ghost. The
failure is silent, it is correlated with how many open items you pass in, and it looks exactly
like the finding you are trying to measure. The most expensive label in our schema is produced
by our own recall failures. That single sentence dictated the whole design.
What we run instead, in four steps:
1. Cheap candidate generation, wide. For each open commitment, pull candidate passages from
the later call by lexical and embedding retrieval over the subject terms — the metric, the plant,
the segment name. Tuned for recall, not precision. Retrieving ten irrelevant paragraphs costs
tokens. Missing the one relevant paragraph costs a wrong published claim.
2. One judgement per commitment, never a batch. Each open commitment gets its own call with
its own text in the prompt, judged against its own candidates. This is more expensive per row and
it is the single highest-value decision in the pipeline. A batch judgement gives the model an
implicit budget, and items fall off the end of the list — the drop rate is invisible because a
dropped item and a genuinely unaddressed item produce the same output.
3. Asymmetric burden of proof. "Addressed" and "not addressed" are not symmetric errors.
Marking something addressed when it was not creates one wrong row. Marking something silent when
it was discussed creates a false claim about a named public company. So silence requires the
stricter bar: retrieval must return nothing above threshold and the judgement must be
unambiguous. Anything in between is held, not resolved — it stays new for another quarter
rather than becoming a ghost.
4. Store the evidence span, not just the verdict. Every transition writes the quote that
caused it and the call it came from. Two reasons, and the second is the one that pays. First,
re-running a call must be idempotent, and you cannot check that against a bare enum. Second, when
a reader disputes a verdict — and they do — the answer is a quote and a date, not "the model said
so." A ledger of verdicts with no evidence column is not auditable, and an unauditable ledger is
not publishable.
The measurable effect of step 3 is that our ghosting rate is a lower bound. Held rows are
excluded from the denominator rather than counted as silence, so the estimator is biased toward
under-reporting the finding we are most interested in. That is the direction you want the bias to
run when the finding is the story.
The denominator is a design decision
Look at the table again. new — 7,486 rows, the single largest bucket — means raised recently, no later call has happened yet. These rows have not failed to resolve. They have not had the chance.
So there are two defensible ghosting rates:
Sketched generically — ledger is your entity table, state its current label, 'unresolved'
the censored value:
-- Both rates, side by side, so the difference is impossible to miss.
SELECT
count(*) AS all_rows,
count(*) FILTER (WHERE state <> 'unresolved') AS resolvable,
count(*) FILTER (WHERE state = 'silent') AS silent,
round(100.0 * count(*) FILTER (WHERE state = 'silent')
/ count(*), 1) AS pct_of_all,
round(100.0 * count(*) FILTER (WHERE state = 'silent')
/ nullif(count(*) FILTER (WHERE state <> 'unresolved'), 0), 1)
AS pct_of_resolvable
FROM ledger;
-- all_rows | resolvable | silent | pct_of_all | pct_of_resolvable
-- 24328 | 16842 | 2271 | 9.3 | 13.5
Both are true. The second is about 45% larger than the first.
The first one is the one you compute by accident, because a bare count(*) is the easiest query anyone ever writes. It is also the one that answers a question nobody asked. "9.3% of everything we have ever recorded" mixes in rows whose outcome is still in the future; as those rows age, the number drifts upward on its own, without any company behaving any differently.
The second is the one that answers how often does a commitment go quiet, and it is the one we publish — 13.5%, more than one in eight — with the exclusion stated next to it every time.
This is right-censoring wearing business clothes. If you have ever done survival analysis you have seen it: you cannot count subjects still under observation as subjects who did not experience the event. Business datasets get this wrong constantly, because the censored rows do not announce themselves. They just sit in the table looking like data.
And the exclusion we publish is still the crude one, which is the honest thing to say about it. It removes rows with no later call. It does not wait for each commitment's own stated horizon to elapse — a three-year capacity target raised once and not repeated on the next call reads as silent in our ledger long before its own deadline arrives. The horizon is recorded; it is not yet in the denominator. The rigorous version of this section fits a survival curve over quarters-since-commitment and reads off the probability of never being revisited given full exposure. We publish the restricted-denominator figure instead, because it is legible to a reader who will not check a hazard function — and we say which estimator we skipped so a reader can price the difference. State the estimator you did not use. It is cheaper than being asked.
The other denominator: pooled or per-company?
Same corpus, same verdicts, one more decision. To get a kept rate you can either pool every commitment and take one ratio, or compute a rate per company and average those.
- Pooled across commitments old enough to have resolved: 57.8%
- Mean of per-company rates (n = 1,459 companies with at least one resolvable commitment): 56.7%
A point apart, which looks like a rounding argument, and is not. They answer different questions. The pooled figure is dominated by companies that make a lot of commitments — a talkative company with 60 tracked targets counts sixty times. The mean-of-means gives a company with three commitments the same weight as a company with sixty.
Here the two happen to sit close together. That is a property of this corpus on this date, not a law. The gap widens whenever behaviour correlates with volume, which in this domain it plausibly does. The rule that survives: name the unit of aggregation in the sentence that contains the number. "57.8% of resolvable commitments were kept" and "the average company keeps 56.7% of its resolvable commitments" are different sentences, and you do not get to use them interchangeably because they came out close. Neither of them is "57.8% of commitments we track" — that sentence is a third thing, it is smaller, and it is wrong, because a third of the ledger has no outcome yet.
The chart we deleted
We also hold a US corpus: 37,529 commitments across 885 US-listed companies. The obvious next move writes itself. Two ledgers, same schema family, one question — do managements in one market follow through more than the other?
We are not going to answer that, and the reason is the most useful thing in this post.
Coverage depth differs. The US corpus averages 6.99 distinct quarters of coverage per company; the Indian corpus averages 4.18. The two are not observing the same length of time, so they are not equally exposed to the event. Any per-company count compared across them is measuring transcript availability.
The label vocabularies differ. The US ledger has an explicit not-reported / unverifiable class. The Indian schema does not. That single asymmetry siphons unknowns out of the US ghosting bucket before any company behaviour enters the picture — it moves the number mechanically.
The extraction specs differ. The two corpora were built under different prompt specifications, at different times. Two annotators with different instruction sheets do not produce comparable rates, and it does not matter that both annotators are the same model family.
Any one of those three is sufficient to invalidate the comparison. All three are present. The headline you could write from dividing our two totals — and a reader can, they are both in this post — would be a fact about our pipeline wearing the costume of a fact about corporate behaviour.
So: the totals are here, in one paragraph, with the caveat welded to them, and there is no comparison chart. Deleting a finished chart is the least popular thing you can do in a data project and it is usually the correct one.
Three assertions worth more than the test suite
Everything above is about publishing a number correctly. This part is about noticing when the
number is wrong, and it is the section I would keep if I could keep only one.
We had every figure in this post sourced, dated and re-derived against production. A reviewer
then re-derived them again and found the ledger was wrong. Our data is partitioned by market,
and sitting inside the Indian partition were 51 commitments belonging to seven US-listed
companies. Not many rows. But they were inside every count I had just carefully written down.
The tell had been on the page the whole time: the file claimed more companies with commitments
than we had companies with transcripts. A company cannot make a commitment on a call we do not
hold. One number was impossible next to the other, and both were published, in the same table.
No test caught it because this is not what tests are for. A unit test asks did this function do
what I said. What was needed was a different question: do these numbers contradict each other?
Three queries, run against production on a schedule, would have caught it on day one.
-- 1. A subset cannot be larger than its superset.
-- No entity can appear in the ledger unless you parsed a document for it.
SELECT
(SELECT count(DISTINCT entity_id) FROM ledger) AS entities_in_ledger,
(SELECT count(DISTINCT entity_id) FROM source_doc) AS entities_with_documents;
-- ASSERT entities_in_ledger <= entities_with_documents
-- Ours came out 1,655 vs 1,654. That one row of slack was 51 contaminating rows.
-- 2. The state vocabulary is closed. Anything outside it is a pipeline bug,
-- not a new category.
SELECT state, count(*)
FROM ledger
WHERE state NOT IN ( /* your enumerated states */ )
GROUP BY 1;
-- ASSERT zero rows
-- 3. A row can only be resolved against a document that exists.
-- Anything not censored must have a LATER document to have been judged by.
SELECT count(*) AS impossible_resolutions
FROM ledger l
WHERE l.state <> 'unresolved'
AND NOT EXISTS (
SELECT 1 FROM source_doc d
WHERE d.entity_id = l.entity_id
AND d.doc_date > l.created_at);
-- ASSERT 0
Note what these have in common. None of them needs to know the right answer. They only need two
numbers that must move together, or a set that must be empty. That is the whole trick, and it is
why they are cheap enough to run on every load.
The uncomfortable corollary: a wrong number that is internally consistent will survive review.
A wrong number that is not will be caught by the first careful outside reader — which is a far
worse way to find out. Every rate in this post survived the correction, as it happens; none of the
contaminating rows carried the state the study is about, so 13.5% is still 13.5%. Only the counts
moved. But I did not know that until someone checked, and "it turned out fine" is not a process.
The bug that ate a week
One concrete war story, because it generalises.
The Indian pipeline derives a fiscal-quarter label for each transcript. Indian fiscal years run April–March, filings are inconsistent about what they call a period, and our derivation had an off-by-one for a class of documents. It is on file as a defect.
The failure was not that some labels were wrong. It is that the label was being used as the join key for an outcome claim — "management said X in Q2, and by Q4 it had gone quiet." Get the label wrong and the sentence is wrong about a named public company, in a way that looks perfectly well-formed and is very hard to notice.
The fix has two halves. The label is now derived from the call date and the period phrase parsed
out of the document body, never from the filename or the exchange filing metadata — and the
filename-derived label is kept in its own column, so a disagreement between the two is a
detectable event rather than a silent overwrite. The second half is a rule: a derived label may
order rows; it may not be the evidence in a claim. If a public sentence names a quarter, that quarter is verified against the source document, not against our own derived column. Anything we cannot verify gets written without the label — "on a later call" is weaker prose and a much stronger claim.
What we are not publishing, and why
The extraction specification and the schema stay private — the prompts, the field layout, the
schema enforcement, the model configuration. The SQL above is deliberately written against
generic table and column names; the pattern is the point, not our layout. So does the row-level ledger; a dump of which company went quiet on what is the
product. Corpus-level aggregates are free, and so is everything above about denominators,
censoring and absence labels.
Which means: you cannot recompute the numbers in this post, and you should weigh them
accordingly. What you can check is every step of the reasoning, and the three assertions run
on your own tables. That is the part meant to be reused.
The six rules, condensed
- If the outcome of a row lives in a future document, model the row as an entity with a lifecycle, not as an output of one document.
- An absence label carries its observation boundary everywhere it goes, including into the column name.
- Exclude censored rows from the denominator, and say that you did, in the same sentence as the number.
- Name the unit of aggregation. Pooled and per-company are different questions that produce similar-looking numbers.
- Write the invariants that must hold between your numbers and run them on every load. A wrong figure that is internally consistent will survive every review you can afford.
- Work out which of your two error directions is the expensive one, and make the pipeline pay for the cheap one. Ours: a false ghost is a false claim about a named company, so silence carries the stricter burden of proof and the headline number comes out understated.
If you build one of these, I would genuinely like to hear which of the six you had to learn the
expensive way. My guess is 2 or 6.
The corpus figures above are from Inve's earnings-call ledger, as of 4 September 2026: 24,328 tracked commitments, 1,648 Indian listed companies, 8,220 parsed transcripts, 2,271 carrying a ghosted verdict. "Ghosted" means not mentioned again on a subsequent earnings call in our parsed corpus — a company may have addressed the target elsewhere. The corpus is proprietary, so these aggregates are not something you can recompute; the reasoning about denominators, absence labels and aggregation is, and that is the part meant to be reused. I work on this at Inve, which is a commercial product, so weigh the numbers accordingly. Drafted with AI assistance and edited by hand.
Top comments (0)