An SEC company facts response can contain several values for the same reporting period. A later filing may repeat a prior-year comparison, and an amended filing may add another observation. If your loader keeps whichever row happens to appear last, the answer can change when the input order changes.
Consider two observations for the same company, concept, unit and period. The original filing reports 100; a later amendment reports 105. A backtest with a cutoff before the amendment must see 100. A current research view may prefer 105 after checking the amendment. A single drop_duplicates(keep="last") cannot serve both views without a policy for availability and filing priority.
Keep the observations before selecting one
The SEC company facts API returns facts grouped by taxonomy, concept and unit. Individual observations carry filing metadata such as accession number, form and filing date. Preserve the raw response and include those fields in your normalized records. For a duration fact, retain both start and end; for an instant fact, retain the end date and a missing start. Keep the entity, taxonomy, concept and unit alongside them.
The following example uses invented values to show selection behavior. It assumes the records already have matching entity, concept, unit and period. The cutoff is a filing-date cutoff, not an intraday availability timestamp.
from datetime import date
observations = [
{"value": 105, "filed": "2025-05-15", "form": "10-Q/A",
"accession": "amendment"},
{"value": 100, "filed": "2025-05-01", "form": "10-Q",
"accession": "original"},
]
def latest_available(rows, cutoff):
eligible = [r for r in rows if date.fromisoformat(r["filed"]) <= cutoff]
if not eligible:
return None
return max(eligible, key=lambda r: (r["filed"], r["accession"]))
assert latest_available(observations, date(2025, 5, 10))["value"] == 100
assert latest_available(observations, date(2025, 5, 20))["value"] == 105
Sorting by accession here only makes ties reproducible; it does not establish which same-day filing was accepted first. If the cutoff needs intraday precision, join to filing acceptance timestamps and define the cutoff in a consistent timezone. For a date-only rule, decide explicitly how to handle multiple filings on the cutoff date. A conservative daily backtest can defer all same-day filings until the next trading session.
A later filing is not always a correction
The latest available value can be a useful default for a research view, but the selection result still needs review. A later 10-Q can repeat an earlier comparative period. An amendment can cover a limited part of a filing. A changed value can reflect a correction, reclassification, a different context, or an extraction problem. The filing itself supplies the explanation.
Store the selected value with the selected accession and the selection policy. Keep the competing observations too. If a user asks why a chart changed, you can show which source was selected before and after the new filing arrived. For an original-as-reported view, use a separate policy and identify the original filing rather than assuming the earliest row in an API response is correct.
Periods and contexts still matter
A fiscal quarter's cash-flow number and a year-to-date cash-flow number can end on the same date. Grouping on end alone can collapse them together. Include the start date for durations, and compare only periods with compatible meaning. Likewise, do not add USD and shares or treat per-share amounts as total dollars. Unit and concept are part of the identity of a fact.
Company facts is useful for standardized entity-wide facts. It is not a substitute for inspecting the original filing when a table uses dimensions or a company-specific concept. When two values disagree, open the filing and check its context and surrounding note before labeling either one a restatement.
A practical storage design has two layers: an append-only observation table and a selected-fact view parameterized by cutoff and policy. Recompute the affected selection when a filing arrives, while leaving earlier point-in-time views reproducible. The selected view can then feed ratios without erasing the source records used to calculate them.
For a related period mismatch, see SEC XBRL Frames Can Mix Fiscal Quarters.
Top comments (0)