DEV Community: Christian Pichichero

Why a Lease Is Not Enough to Stop Duplicate Scheduled Jobs

Christian Pichichero — Thu, 05 Nov 2026 21:14:47 +0000

A cron job that matters enough to run on two machines has changed category. It is no longer a timer with a function attached. It is a distributed system.

On one machine, the scheduler starts a job and assumes it owns that execution. Add a second machine for availability, and both schedulers may observe that the same job is due. If duplicate work means sending the same cleanup query twice, that may be acceptable. If the job submits a settlement instruction or broadcasts a blockchain transaction, the second execution may be irreversible.

This is where “at most once” stops being free.

The basic lease

A common coordination mechanism is a lease stored in a shared database. The row usually contains:

A job identifier
The current owner
An expiry timestamp
A monotonically increasing fencing token

Acquisition must be atomic. Two workers cannot read an expired row and then both decide they own it. A transaction, row lock, or compare-and-swap update must select one winner.

Conceptually:

with transaction:
    lease = lock(job_id)

    if lease.expires_at <= database_time():
        lease.owner = this_worker
        lease.expires_at = database_time() + lease_duration
        lease.fence = lease.fence + 1
        acquired_fence = lease.fence
    else:
        acquisition_failed

Using the lease store’s clock avoids making acquisition depend on the wall clocks of separate workers. The owner can renew the lease while running, provided the update checks both its identity and fencing token.

This prevents normal overlap. It does not stop a stale process.

The paused-worker failure

Suppose scheduler A acquires a lease with fencing token 41. It reads the input and then pauses for longer than the lease duration. Causes include a long garbage-collection pause, CPU starvation, a frozen virtual machine, or a network partition.

The lease expires. Scheduler B acquires it with token 42 and performs the job.

Scheduler A then resumes. Its local process still has the state saying it acquired the lease. Checking its own wall clock does not repair the situation. Even if it checks the database before acting, another pause can occur between that check and the external side effect.

A lease answers, “Who may work during this time window?” It cannot, by itself, make an old worker forget what it was doing.

Longer lease durations reduce how often this race appears, but also delay recovery when a worker actually dies. Frequent renewal narrows the window but does not remove process pauses or network uncertainty. A timeout is a failure detector, not proof that the old worker has stopped.

Fencing the side effect

A fencing token turns each successful acquisition into a newer generation. Every operation sent to the protected resource includes that token.

The resource records the highest token it has accepted. Once it has seen token 42, it rejects an operation carrying token 41. Scheduler A can wake up, but it can no longer act as the current owner.

For a database-owned resource, the check can be part of the same transaction as the write:

if request.fence < resource.highest_fence:
    reject_as_stale
else:
    resource.highest_fence = request.fence
    apply_operation()

Equal tokens may need separate operation identifiers so retries from the current owner are deduplicated. Fencing distinguishes lease generations; it does not replace request-level idempotency.

The important limitation is that the resource performing the side effect must participate. Putting a token in application logs does nothing. If an external API cannot inspect a fencing token, the scheduler cannot force it to reject a stale caller.

Irreversible execution exposes the gap

Consider a scheduled job that constructs and submits a blockchain transaction. Scheduler A prepares the action and pauses. Scheduler B takes the newer lease and submits its transaction. Scheduler A resumes and submits a separately constructed valid transaction.

The old lease cannot unsend either transaction. Both may be accepted if they are distinct and valid under the relevant account rules.

Rebroadcasting identical signed bytes is a different case because the transaction identity can provide deduplication. That protection disappears when separate executions create distinct transactions, use different nonces, or represent the same business action through different requests.

A duplicated email is usually an annoyance that can be observed and handled. A duplicated asset transfer or swap changes durable state. Detection after the fact is not coordination.

What to use when the destination cannot fence

If the destination has no fencing mechanism, move deduplication to the narrowest durable boundary you control:

Assign a stable execution ID to each logical schedule slot, such as job-name/2026-09-02T10:00Z.
Store that ID under a uniqueness constraint before creating the external action.
Use an outbox so job-state changes and dispatch intent are committed together.
Pass the execution ID as an idempotency key when the destination supports one.
Separate retries of one logical execution from creation of a new execution.

An outbox alone still has an ambiguity window: a dispatcher can send a request and crash before recording the acknowledgement. Endpoint idempotency or a fenced single writer is needed to close that gap.

If no layer can reject duplicate requests, honest semantics are limited. Marking a job complete before the side effect can avoid duplicate attempts but may lose work after a crash. Recording completion afterward supports retries but permits duplicate execution. That is a product decision, not a database trick.

Disclosure: I build Tradevo, software where creators describe an automated crypto strategy in plain English, historically test and paper-trade its deterministic rules, submit it for human review, set a free or monthly creator price, and publish approved strategies, while other users can discover, subscribe to, allocate USDC to, and run published strategies from a wallet they control.

The practical rule

Use leases to select an active worker. Use fencing tokens to reject workers from expired lease generations. Use stable operation IDs to deduplicate retries. For irreversible side effects, verify that the final boundary enforces one of those mechanisms.

Without that final enforcement point, “we have a distributed lock” describes an intention, not at-most-once execution.

What Each Test Layer Proves When Code Moves Funds

Christian Pichichero — Thu, 29 Oct 2026 19:52:49 +0000

Your onchain executor can pass its unit tests, produce valid calldata, and still send funds through an unintended path. The difficult part is not writing more tests. It is knowing what each test environment proves and where its evidence stops.

A useful testing ladder has distinct stages: pure simulation, forked-chain execution, a production canary using a trivial amount, and staged production use. None is a substitute for the others.

The most dangerous design is often a dryRun boolean scattered across the codebase. If some transaction paths respect it while another path broadcasts, the dry run creates false confidence. A system with no dry-run mode is visibly dangerous. A partially enforced one looks safe until the missed branch moves funds.

Start with a deterministic execution plan

Separate deciding what to do from doing it. The planner should consume explicit inputs and produce an execution plan without holding signing or broadcast capabilities.

A plan might contain:

chain identifier
contract and token addresses
calldata
value to send
approval requirements
slippage bounds
deadline
expected state transitions
reasons the action should be rejected

The same planner should run in simulation, fork tests, canaries, and production. Environment-specific code should sit behind the transport boundary.

Conceptually:

market snapshot + strategy state
              |
              v
       deterministic planner
              |
              v
       validated transaction plan
              |
      +-------+--------+
      |                |
 simulation transport  live transport
 no signing material   signing + broadcast

This is stronger than checking a flag immediately before an RPC call. In simulation mode, the process should not load a signer capable of broadcasting. The restriction is structural rather than conventional.

Pure planning tests are good at checking arithmetic, rounding, token decimals, state transitions, allocation rules, and invariants such as “a rejected plan cannot contain an executable transaction.” Property-based tests are useful here because many failures live at boundaries: zero balances, minimum trade sizes, stale timestamps, or values just inside a slippage limit.

What this layer does badly is represent external state. A mock contract behaves as its author expects, not necessarily as the deployed contract behaves. A simulated quote does not expose an RPC outage, a changed pool state, or a transaction competing in the public mempool.

Use a fork to test contract interactions

A forked chain lets the executor interact with deployed bytecode and a captured state snapshot. This catches errors that pure simulation tends to miss:

incorrect ABI encoding
approval and allowance assumptions
token decimal mistakes
revert conditions in deployed contracts
multi-call ordering problems
unexpected behavior from non-standard tokens

Fork tests should execute the complete sequence, not isolated calls. If production performs an approval followed by a swap, the test should run that sequence through the same transaction-building code.

Pin the block used by the test. Without a pinned state, a passing test can become a failing test because liquidity, balances, or contract state changed rather than because the code changed. It is also useful to maintain fixtures for awkward historical states instead of testing only a convenient recent snapshot.

A fork still does not reproduce production. It usually omits real RPC latency, provider rate limits, signer failures, relayer behavior, gas sponsorship, mempool ordering, and reorgs. Local mining also tends to be more predictable than public block production. A fork proves that a transaction can execute under the captured state. It does not prove that the transaction will execute under the state present when production broadcasts it.

Make dry run a capability boundary

A safe dry run should be incapable of broadcasting, not merely instructed not to.

Keep the live signer and broadcast client out of the dry-run process. Give that process a simulation transport that supports calls and gas estimation but has no method or credentials for sending transactions. If a code path unexpectedly requests a broadcast, it should fail closed and emit an observable error.

Then test the boundary itself. Exercise every action type through dry-run mode and assert that no live transport was constructed. Include cancellation, retries, approval refreshes, cleanup jobs, and administrative paths. Background workers are easy to overlook because the visible strategy path may respect dry run while a retry queue still holds an executable payload.

Audit logs should record the plan, validation decisions, simulation result, selected transport, and whether signing material was loaded. Logs are not an enforcement mechanism, but they make boundary violations easier to diagnose.

The trade-off is duplication pressure. Simulation and live transports can drift. Reduce that risk by sharing the planner, serializer, and validators while keeping the final send capability separate.

Use a canary to test the production path

A canary sends a real transaction using the production RPC, signer, policy checks, monitoring, and confirmation logic, but with the smallest practical exposure permitted by the venue and token precision.

This stage can reveal configuration errors that earlier layers cannot: wrong chain selection, unavailable signing credentials, broken gas handling, provider restrictions, stale configuration, and monitoring that watches the wrong address.

A canary is still weak evidence. It covers a particular route under a particular market state. It may not exercise retries, cancellation, partial completion, or adverse ordering. Success should not cause the system to bypass limits or broaden access automatically.

Canaries also need containment. Use a dedicated wallet, strict token permissions, bounded transaction value, and alerts tied to expected transaction hashes and state changes. Stop after unexpected behavior rather than repeatedly retrying an uncertain action.

Production needs continuing controls

After a canary, staged activation is safer than treating deployment as the end of testing. Keep explicit limits on value, action frequency, allowed contracts, tokens, and slippage. Monitor both submitted transactions and intended plans so that “nothing happened” is detectable alongside incorrect execution.

Plan for ambiguous outcomes. An RPC timeout does not establish that a transaction was rejected. Before retrying, inspect the transaction hash, sender nonce, and chain state. Blind retries can duplicate an action when the initial request reached the network but its response did not reach the executor.

Pausing should block new plans at the scheduler and block broadcasts at the transport boundary. Relying on one check leaves another path available.

Disclosure: I build Tradevo, software where creators describe crypto strategies in plain English, historically test and paper-trade the compiled deterministic rules, submit them for human review, set a free or monthly price, and publish approved strategies, while other users can discover, subscribe to, allocate USDC to, and automatically run those rules from their own wallets.

The testing ladder works because each stage answers a narrower question. Simulation checks planning logic. Forks check interaction with captured contract state. Canaries check the real operational path with contained exposure. Production controls limit what happens when assumptions fail. Treating any stage as proof of the next is where expensive mistakes begin.

Free Financial Data APIs for Backtesting: An Honest Comparison

Christian Pichichero — Sat, 24 Oct 2026 17:32:08 +0000

Free financial data can support serious backtesting—but only if the source matches the question.

A price-momentum test needs historical prices and corporate-action handling. A macro strategy needs revision-aware economic series. A fundamental strategy needs values as investors actually knew them on each historical date. Those are different data problems.

This guide compares SEC EDGAR, yfinance, Stooq, FRED, Alpha Vantage’s free tier, and Tradevo Data’s public proof pack. The goal is not to declare one winner. It is to show where each source may be useful, where it can fail, and whether it is point-in-time safe for the intended test.

The short answer

SEC EDGAR: a free primary source for US filings, but substantial engineering is required to produce clean point-in-time panels.
yfinance: convenient for exploration and price-data prototypes, but do not assume every historical or fundamental field is point-in-time safe.
Stooq: potentially useful for accessible market-history research; validate adjustments, symbols, and coverage before relying on it.
FRED: useful for macroeconomic research, with ALFRED often the better choice when historical revisions matter.
Alpha Vantage free tier: convenient packaged APIs for prototypes; check current limits, history, adjustment rules, and PIT behavior in its documentation.
Tradevo Data: an honest budget-tier option for research-grade point-in-time US equity fundamentals, with a free public proof pack and a paid API. It is not a broad market-data terminal and does not cover prices, non-US companies, or delisted companies.

Comparison table

Source	Best for	Free access	Point-in-time-safe?	Main caveat
SEC EDGAR	Primary US filing data	Yes	Potentially, if modeled by public filing date	Raw filings, amendments, tags, units, fiscal periods, and issuer history require careful processing
yfinance	Fast price-data exploration and notebooks	Yes	Not automatically	A convenient interface does not itself prove that historical fundamentals were available on the simulated date
Stooq	Downloadable market history and prototyping	Yes	Depends on field and methodology	Verify corporate-action adjustments, survivorship, identifiers, and coverage
FRED / ALFRED	Macroeconomic and rate series	Yes	FRED latest values: not necessarily; ALFRED vintages: potentially suitable	Release lags and revisions must be modeled explicitly
Alpha Vantage	Packaged market and fundamental APIs	Free tier available	Do not assume	Check current limits, output history, adjustment methods, and filing-time semantics
Tradevo Data proof pack	Inspecting PIT US fundamentals and methodology	Yes	Designed for PIT research	Small proof pack: 5 companies, latest 3 fiscal years, and 225 rows
Tradevo Data API	Budget-tier PIT US equity fundamentals	Card-backed 7-day trial; renews at $29/mo unless canceled	Uses `first_filed <= as_of`	No prices, TTM, non-US, delisted, or Parquet coverage

“Point-in-time-safe” is not a permanent badge attached to a vendor. It depends on the field, timestamp, revision policy, universe construction, and how your backtest queries the data.

1. SEC EDGAR: a free primary source

SEC EDGAR is a primary source when you need US company disclosures. The underlying filing data is public domain, and publication information can be used to determine when facts became public.

That does not make an EDGAR download immediately backtest-ready.

You still need to handle:

original filings and amendments;
company and ticker identity changes;
XBRL concepts that vary across issuers;
duplicate facts, units, dimensions, and contexts;
annual, quarterly, year-to-date, and instant-period distinctions;
fiscal calendars and restatements;
publication-time cutoffs;
QA and reproducible snapshots.

EDGAR can support point-in-time research because publication events are available. Your resulting database is PIT-safe only if your transformation preserves and models them correctly.

When EDGAR wins

Build directly from EDGAR when you need custom tags, filing text, footnotes, dimensions, forms outside a standardized dataset, or complete control over methodology. It may also win when cash cost matters more than engineering time.

For background, see point-in-time fundamentals data.

2. yfinance: convenient, with limited guarantees

yfinance is popular because it makes exploratory work easy. It can be useful for pulling historical prices, testing notebook logic, checking symbols, or building a first-pass prototype.

The risk is treating interface convenience as evidence of point-in-time correctness.

For price backtests, inspect whether you use adjusted or unadjusted prices and how dividends and splits are represented. For fundamental backtests, ask whether a field is today’s reconstructed history or the value actually available on the simulated date. If the source does not expose publication timestamps and revision vintages, do not infer PIT behavior.

When yfinance wins

It may win when speed and accessibility matter more than contractual or PIT guarantees: teaching, exploratory analysis, rough price-based prototypes, and sanity checks. If your strategy does not depend on historical accounting vintages, a specialized PIT fundamentals product may be unnecessary.

3. Stooq: practical market history with validation required

Stooq offers accessible historical market data and may be useful for research prototypes. Its downloads can be attractive when you want to get a price-based test running without building an ingestion system.

As with any market-history source, validate the exact dataset you consume. Questions include:

Are prices adjusted, and for which corporate actions?
Are delisted securities represented?
How stable are symbols and identifiers?
What are the timezone and session conventions?
Does your universe introduce survivorship bias?

Historical prices avoid some accounting-publication issues, but they do not automatically solve corporate-action, delisting, or universe bias.

When Stooq wins

It may win for low-cost price research, cross-checking, and prototypes where its available markets and conventions fit your design. Review its documentation and test known events before committing to a production pipeline.

4. FRED and ALFRED: consider vintages for macro backtests

FRED is a widely used free source for economic and financial series. It can support research involving rates, inflation, labor, production, and other macro variables.

The central backtesting trap is revision history. Today’s value for a historical month may not equal the value investors saw at the time. Economic releases can be revised repeatedly.

ALFRED preserves vintages for many series, making it potentially more appropriate when a strategy responds to what was known on a historical date. You must still model release timing. A value labeled for one month may have been published later.

When FRED or ALFRED wins

They may win for macro signals rather than equity-fundamentals research. Consider ALFRED vintages where revisions could affect the result; standard FRED data may be sufficient when latest-revised history is explicitly what you want.

5. Alpha Vantage: packaged APIs for quick prototypes

Alpha Vantage packages multiple categories of financial data behind documented endpoints and offers a free tier. That may reduce setup work compared with normalizing multiple sources.

Before using it in a backtest, see its pricing page and documentation. Confirm current request limits, historical depth, corporate-action treatment, field definitions, and whether fundamental values are tied to filing-availability dates.

A reported fiscal-period end is not an availability date. Joining a December year-end value to December prices can leak information from a filing published later.

When Alpha Vantage wins

It may win when you want one packaged interface, modest prototype volume, or data categories outside a narrow PIT fundamentals product. Evaluate each endpoint separately rather than assuming one methodology applies across the service.

6. Tradevo Data: inspect the free PIT proof pack first

Tradevo Data provides point-in-time US equity fundamentals from SEC EDGAR public-domain data. The public GitHub proof pack contains 5 companies, their latest 3 fiscal years, 225 rows, and the full methodology, with no signup.

Each row includes:

first_filed: the date the value became public;
original_value: the first-reported PIT-safe value;
latest_value: the current revision;
a restatement flag for changes greater than 0.5% on the same tag, including amendments;
qa_status.

The annual dataset contains 632,466 point-in-time rows across 5,168 US companies, 16 annual concepts, up to 12 fiscal years, and 37,804 labeled restatements. The concepts are Revenue, NetIncome, Assets, StockholdersEquity, OperatingCashFlow, EPSDiluted, DilutedShares, GrossProfit, OperatingIncome, PretaxIncome, IncomeTaxExpense, CapitalExpenditures, CashAndCashEquivalents, CurrentAssets, CurrentLiabilities, and NetPPE.

Quarterly coverage includes 1,089,635 rows across 5,382 companies and 7 quarterly concepts. Q4 is reported where tagged or derived and labeled where supported. Derived Q4 EPS and shares are not provided.

The JSON endpoint is:

/v1/fundamentals?ticker&as_of[&concept][&period=annual|quarterly]

Annual is the default period. The server applies first_filed <= as_of, inclusive on the date. This addresses one common form of lookahead bias in fundamental backtests, but users must still handle matters such as trading-session timing and universe construction.

The card-backed 7-day free trial covers 10 companies, a rolling 3-year history, and 100 requests per day, with no bulk. It renews at $29/mo unless canceled. Pro provides complete available history, 5,000 requests per day, and bulk access through /v1/download and /v1/snapshot?as_of, each with period=annual|quarterly. Parquet is not included.

This is intentionally narrower than many alternatives: no price data, TTM, non-US companies, or delisted-company coverage.

Review the free proof pack before considering the API:

https://github.com/christianpichichero-max/pit-fundamentals

https://tradevodata.com/?utm_content=blog-free-financial-data-apis-for-backtesting&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate

Documentation: https://tradevodata.com/docs?utm_content=blog-free-financial-data-apis-for-backtesting&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate

When Tradevo Data wins

It may win when you want standardized point-in-time US equity fundamentals, server-side historical-date filtering, restatement fields, and JSON bulk access without building an EDGAR normalization pipeline. It does not win when you need prices, filing text, custom XBRL tags, TTM metrics, non-US or delisted coverage, Parquet, or a broader multi-asset platform.

Point-in-time safety checklist

Whatever source you choose, ask:

What was known? Is the value original, latest-restated, or unspecified?
When was it known? Use a publication or filing date, not merely a fiscal-period end.
Was the date available intraday? Daily backtests may need a next-session rule.
Was the security in the universe then? A current constituent list creates survivorship bias.
How were prices adjusted? Splits, dividends, and symbol changes can alter results.
Are revisions preserved? This matters for both company fundamentals and macro data.
Can you reproduce a snapshot? Save raw responses, query dates, documentation versions, and code.

When to build it yourself

Build your own pipeline when your research requires data no standardized product exposes, when you need auditable transformation control, or when ongoing engineering is part of your process.

For EDGAR fundamentals, expect work involving taxonomy mapping, amendment handling, fiscal-period inference, duplicate resolution, QA, identity history, and refresh monitoring. “Free data” means no license fee; it does not mean zero cost.

Do not assume that easy ingestion makes a dataset PIT-safe. The difficult work is establishing that every historical row was available when your simulation says it was.

A practical selection rule

Choose the narrowest source that answers your research question honestly:

price prototype: yfinance or Stooq may be enough;
macro strategy: consider FRED, or ALFRED when revisions matter;
custom US filing research: build from SEC EDGAR;
packaged multi-category prototype: evaluate Alpha Vantage endpoint by endpoint;
standardized PIT US fundamentals: inspect Tradevo Data’s free proof pack, then compare its API with credible alternatives such as Sharadar, Tiingo, and QuantConnect. See each provider’s pricing page and documentation.

No vendor removes the need for validation. A defensible backtest starts with explicit assumptions about availability, revisions, adjustments, and universe membership—not with the prettiest API response.

Not investment advice. Verify current competitor features, documentation, and pricing on each provider’s website.

“Non-Custodial” Is a Spectrum, Not a Boolean

Christian Pichichero — Thu, 22 Oct 2026 18:51:10 +0000

When a wallet integration is described as “non-custodial,” the useful engineering question is not whether the label applies. It is: who can move the assets, under what conditions, and who must cooperate to end that access?

That framing exposes distinctions hidden by a binary label. An app may never possess a user’s root key while still operating a delegated signer that the user cannot directly disable. Another system may split key material across several parties, none of which can act alone. Both may call themselves non-custodial, but their failure modes are different.

Here is a practical spectrum, ordered from stronger to weaker unilateral user control. Lower positions are weaker specifically in the user’s ability to exclude other parties without their cooperation. They may still offer better recovery, automation, or support.

1. A plain externally owned account

With a plain EOA, one private key controls the account.

Who can move assets? Anyone with that private key.

Who must cooperate for third-party access to end? No third party has access unless the key was shared or compromised. If it was compromised, access cannot be selectively revoked. The user must move assets to a new account.

This model is simple to reason about, but unforgiving. There is no native permission scoping, spending limit, signer expiry, or recovery process. Automation usually means exposing the key to software that must remain available, which undermines the reason for choosing the model.

An EOA provides strong independence from operators, not necessarily strong operational security.

2. A smart account with a user-revocable permission

A smart account can let an owner authorize another key to perform a restricted set of actions. The restriction might cover certain contracts, tokens, methods, time periods, or spending amounts.

Who can move assets? The owner and any signer accepted by the account’s current contract logic, each within whatever rules that logic enforces.

Who must cooperate for delegated access to end? If the owner can submit an on-chain transaction that disables the permission, no operator needs to cooperate. The owner does need the chain to include that transaction and must retain whatever authority the account recognizes as its owner.

This is a meaningful improvement for automation because revocation is independently verifiable and user-initiated. It also adds contract risk. A bug in permission checks, upgrade logic, or account recovery can be more consequential than a broken user interface.

The important evidence is not a settings toggle. It is the contract path that removes the permission and proof that the user can call it without an operator signature.

3. A delegated session signer removed by the operator

Some embedded wallets support a secondary signer controlled by an application or its execution service. A policy may restrict what that signer can request.

Who can move assets? The user’s root authority can move assets, while the delegated signer can perform actions accepted by the wallet or signing provider’s policy.

Who must cooperate for delegated access to end? If no user-callable revocation path exists, the operator must remove its signer. A user who controls or can export the underlying wallet can instead move to a fresh wallet to eliminate residual access.

This model is useful for unattended execution because the user does not need to approve every action. It is weaker than unilateral smart-account revocation because cancellation and signer removal are separate events. A successful cancellation request may stop the application’s executor immediately while signer removal remains an operator-side, best-effort operation.

Policy placement matters too. An API policy enforced by a signing provider is not an on-chain permission. An executor check is not a wallet-level restriction. These controls may be useful, but documentation should identify the enforcing component and explain what happens if that component is unavailable or faulty.

4. Threshold MPC custody

“MPC wallet” describes a family of designs, not one custody model. In a threshold setup, several key shares participate in signing. The threshold and share holders determine the actual control structure.

Who can move assets? Any coalition that satisfies the signing threshold. Depending on the design, that might be the user plus a provider, several provider systems, or the user acting through multiple devices.

Who must cooperate for provider access to end? That depends on whether the user can rotate or replace shares without the provider. If resharing needs provider participation, ending provider access requires cooperation. If the user independently controls enough shares to authorize rotation, the answer is different.

MPC can support recovery and hide key-management complexity. It can also make authority difficult to inspect because the relevant policy lives in protocols, infrastructure, and contractual procedures rather than a public account contract.

Ask for the threshold, the share holders, the recovery path, and the share-rotation procedure. “We never reconstruct the key” does not answer who can authorize a signature.

5. An exchange account

In a conventional exchange account, the exchange controls the blockchain keys and maintains an internal ledger for customers.

Who can move assets? The exchange, through its signing infrastructure.

Who must cooperate for the user to withdraw or terminate access? The exchange must process the withdrawal. The user cannot independently move blockchain assets represented by an account balance.

This model can offer account recovery, customer support, transaction screening, and familiar authentication. Its trade-off is direct: the operator controls asset movement.

What to ask instead of accepting the label

For any automated wallet product, request concrete answers to these questions:

Which keys or shares can authorize movement?
Where is each restriction enforced: account contract, signing provider, executor, or user interface?
Can the user disable delegated access without operator cooperation?
Does authorization expire automatically?
What happens if cancellation succeeds but signer removal fails?
Can the user export the wallet or migrate assets without permission?
Are policy changes auditable?
Which controls remain effective if the application server is compromised?

Disclosure: I build Tradevo, software for describing a crypto strategy in plain English, compiling it into deterministic rules, testing and paper-trading it, submitting it for human review, setting a creator price, publishing approved strategies, and subscribing with a chosen allocation; execution uses a scoped and revocable delegated session signer attached to the user’s embedded wallet, while the token allowlist, per-subscription allocation cap, and slippage limits are enforced by the signing provider’s policy and the executor, the authorization does not expire by itself, cancellation stops execution and triggers best-effort server-side signer removal, and users can export their wallet for zero residual access.

“Non-custodial” is useful only as the start of an architecture review. The real boundary is who can sign, who can change the signing rules, and whether the user can exclude every other actor without asking for help.

How to Backtest Value Investing Without Lookahead Bias

Christian Pichichero — Wed, 21 Oct 2026 18:47:11 +0000

A value backtest can be mathematically correct and still use information that investors did not have at the time.

The common failure is simple: join historical prices to today’s cleaned fundamentals by fiscal period. That can place an annual report into the simulation before it was filed, overwrite original figures with later amendments, or include companies selected from today’s surviving universe.

To backtest value investing without lookahead bias, you need three controls:

As-of filing dates: a fundamental becomes eligible only when it was public.
A point-in-time universe: membership must reflect what was investable on each rebalance date.
As-reported values: use the first-reported figure when modeling what an investor originally saw.

These controls apply whether your signal is P/E, P/B, gross profitability, operating margin, or a composite quality score.

For more background, see /blog/lookahead-bias-fundamental-backtests and /blog/point-in-time-fundamentals-data.

Why ordinary fiscal-period joins fail

Suppose a company’s fiscal year ends in December. Its annual revenue, earnings, equity, and cash-flow figures are not necessarily public on December 31. They become usable when the filing reaches SEC EDGAR.

A join like this is therefore unsafe:

fundamentals["year"] = fundamentals["fiscal_period_end"].dt.year
prices["year"] = prices["date"].dt.year
panel = prices.merge(fundamentals, on=["ticker", "year"])

It matches by reporting period, not availability. A January rebalance could receive figures filed weeks later.

The correct question is not:

Which fiscal period belongs to this calendar year?

It is:

What was the latest eligible filing publicly available by this rebalance date?

That requires an availability field such as first_filed and an as-of join where first_filed <= rebalance_date. Same-day inclusion should follow a documented convention. Tradevo Data’s server-side as_of filter is same-day inclusive.

Step 1: define the strategy before touching the data

Write down the rules first. At minimum, specify:

rebalance frequency and exact dates;
eligible securities on each date;
price used for signal formation and execution;
accounting concepts and denominator definitions;
treatment of negative earnings or equity;
filing availability rule;
restatement policy;
transaction costs, liquidity constraints, and missing-data policy.

For example, a simple value-and-quality screen might rank eligible companies on:

Earnings yield: net income divided by market capitalization;
Book-to-market: stockholders’ equity divided by market capitalization;
Gross profitability: gross profit divided by assets;
Operating quality: operating income or operating cash flow relative to assets.

Using yields rather than raw P/E or P/B often makes ranking rules easier to define around negative denominators. That is a design choice, not a guarantee of better results.

Step 2: build a point-in-time universe

Correct fundamentals do not fix survivorship bias.

If you start with companies that exist today and project them backward, failed, acquired, or delisted firms may disappear from the historical sample. The backtest then answers a different question: how today’s survivors would have performed historically.

A proper universe table should contain effective membership intervals or date-stamped snapshots, including delisted securities where the research question requires them. It should also preserve historical identifiers because tickers can change or be reused.

Tradevo Data covers US companies but does not provide delisted coverage. It should not be treated as a complete point-in-time security master. For research requiring a survivorship-free universe, pair fundamentals with an appropriate historical universe and price source—or choose a provider that integrates those components.

Step 3: select fundamentals by `first_filed`

For every rebalance date and company:

Exclude rows where first_filed is after the rebalance date.
Choose the latest eligible fiscal observation under your strategy’s period rules.
Use original_value for the information first reported to the market.
Retain the restatement indicator for diagnostics.
Apply a lag only if it represents an explicit operational assumption, not as a substitute for filing dates.

A blanket rule such as “wait three months after fiscal year-end” is only an approximation. Some filings arrive earlier or later. Exact filing dates are cleaner because they model actual public availability.

In Tradevo Data, each row includes first_filed, original_value, latest_value, a restatement flag, and qa_status. Restatements are labeled when a same-tag change, including amendments, exceeds 0.5%. The dataset contains 37,804 labeled restatements.

A small worked example

Assume a strategy rebalances on two dates and the sample contains these conceptual rows:

ticker	concept	fiscal period	first_filed	original_value
ABC	NetIncome	prior annual period	before first rebalance	available then
ABC	NetIncome	latest annual period	after first rebalance	not yet available
ABC	StockholdersEquity	latest annual period	before second rebalance	available then

At the first rebalance, the latest-period net income must be excluded because its first_filed date is still in the future. At the second rebalance, it becomes eligible if the filing date is on or before that date.

The same principle applies to every ratio. For a point-in-time P/E calculation, both price and EPS must be known at signal time. If you calculate market capitalization from shares, the share figure also needs a valid availability rule. Never silently combine a historical price with a later-revised denominator.

Pandas example using the free sample

The public proof pack contains 225 rows for five companies, covering their latest three fiscal years, with full methodology. Download it without signup at:

https://github.com/christianpichichero-max/pit-fundamentals

The core pandas operation is a backward as-of join:

import pandas as pd

fund = pd.read_csv("fundamentals.csv", parse_dates=["first_filed"])
rebalances = pd.DataFrame({
    "ticker": ["AAPL", "AAPL"],
    "rebalance_date": pd.to_datetime(["2023-02-01", "2023-11-15"])
})

# Example concept; repeat or pivot for equity, assets, and quality inputs.
eps = fund.loc[fund["concept"].eq("EPSDiluted")].copy()
eps = eps.dropna(subset=["first_filed", "original_value"])

# merge_asof requires sorted keys.
eps = eps.sort_values(["ticker", "first_filed"])
rebalances = rebalances.sort_values(["ticker", "rebalance_date"])

pit = pd.merge_asof(
    rebalances,
    eps,
    left_on="rebalance_date",
    right_on="first_filed",
    by="ticker",
    direction="backward",
    allow_exact_matches=True
)

# Attach a price observable at the chosen signal timestamp.
pit["pe"] = pit["price_at_signal"] / pit["original_value"]

In production, add safeguards:

Filter annual versus quarterly periods explicitly.
Resolve duplicate tags using documented taxonomy and QA rules.
Ensure the selected row represents the intended fiscal horizon.
Exclude or separately classify negative and zero denominators.
Preserve rows with missing data rather than filling them from the future.
Test that every selected first_filed is on or before the rebalance date.
Keep signal time separate from execution time to avoid same-close assumptions.

A useful invariant is:

assert (pit["first_filed"] <= pit["rebalance_date"]).all()

Step 4: choose an explicit restatement policy

There are two legitimate research questions:

What could an investor have calculated then? Use original_value.
What does the company currently report for that historical period? Use latest_value.

For a historical trading simulation, the first question is usually the relevant one. Using revised values can leak later amendments into earlier portfolio decisions.

Still, keep both fields. Comparing original and latest values can reveal whether a signal is sensitive to revisions. A restated flag is also useful for audit trails and robustness checks.

Step 5: test the pipeline for leakage

Add automated tests rather than relying on visual inspection:

No fundamental has first_filed > rebalance_date.
Universe membership is valid on the rebalance date.
Prices are timestamped no later than signal formation.
Execution occurs after the signal is observable.
Revisions do not overwrite original values.
Missing historical values remain missing.
Fiscal periods do not accidentally mix annual and quarterly data.
Split adjustments and share counts use compatible conventions.

Also rerun the backtest with conservative execution delays. A result that disappears under modestly stricter timing assumptions may be driven by timestamp choices rather than the intended factor.

In the current five-company, 225-row public proof pack, the 224 rows with reliable filing dates showed a measured mean lookahead of 35.2 days and a maximum of 48 days. These sample statistics apply only to those 224 reliable-filing-date rows and must not be generalized to the full Tradevo Data dataset.

Data options: a fair comparison

Option	Strengths	Limitations	Better fit when
Build directly from SEC EDGAR	Maximum methodological control; public-domain source	Taxonomy normalization, amendments, filing timestamps, and QA require substantial work	You need custom concepts or can maintain an ingestion pipeline
Tradevo Data	Point-in-time US equity fundamentals, original and latest values, filing dates, restatement labels, API and bulk JSON	No delisted, non-US, TTM, or Parquet coverage; not a full market-data stack	You want a budget-tier PIT fundamentals layer and can supply universe and prices
Sharadar	Credible established alternative; available products may support broader research workflows	Coverage and licensing details vary by product	You want its specific ecosystem; see their pricing page: https://www.nasdaq.com/solutions/data-link
Tiingo	Credible API-oriented market-data alternative	Check its current products against your exact PIT and restatement requirements	You want its API workflow; see their pricing page: https://www.tiingo.com/
QuantConnect	Integrated research and execution environment with supported datasets	May be less suitable if you want to own every storage and pipeline component	You prefer a managed algorithmic-research platform; see their pricing page: https://www.quantconnect.com/

Verify current documentation, coverage, licensing, and pricing directly with each provider.

When they win: another option may be better

Choose another provider or platform if you need delisted-company coverage, a ready-made survivorship-free universe, international equities, TTM fields, Parquet delivery, integrated prices and corporate actions, or an execution environment. Tradevo Data does not provide those capabilities.

Sharadar, Tiingo, or QuantConnect may win when their current coverage, ecosystem, licensing, or integrated workflows better match your requirements. Building directly from SEC EDGAR may win when you need maximum methodological control or custom concepts. Verify each option’s current documentation and pricing before choosing.

An integrated platform may also be the better choice when research speed matters more than controlling each transformation. Paying for a broader stack can be rational if it reduces engineering and audit costs.

When to build it yourself

Build from SEC EDGAR when your research depends on custom taxonomy mappings, filing-document lineage, unusual accounting concepts, or transformation rules that a standardized API cannot expose.

Be realistic about maintenance. You will need to parse filings and amendments, normalize company identifiers and units, resolve duplicate facts, distinguish duration from instant concepts, track revisions, derive only defensible periods, and retain reproducible snapshots. You also need tests for silent upstream changes.

A hybrid approach is often practical: use a normalized source for standard concepts, then independently validate critical rows against EDGAR.

A practical starting point

Tradevo Data provides 632,466 annual point-in-time rows across 5,168 US companies, 16 annual concepts, and up to 12 fiscal years where available. It also provides 1,089,635 quarterly rows across 5,382 companies and seven quarterly concepts. Q4 is reported where tagged or derived and labeled where supported; derived Q4 EPS and shares are not provided.

The JSON fundamentals endpoint, /v1/fundamentals, accepts ticker, as_of, optional concept, and period=annual|quarterly; annual is the default. Bulk JSON is included with Pro through /v1/download and /v1/snapshot?as_of, each supporting period=annual|quarterly. Parquet is not included. Documentation is available at https://tradevodata.com/docs?utm_content=blog-how-to-backtest-value-investing-without-lookahead-bias&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

Start by auditing the public proof pack and reproducing the as-of join yourself:

https://github.com/christianpichichero-max/pit-fundamentals

If the schema fits your process, the card-backed seven-day free trial covers 10 companies, a rolling three-year history, and 100 requests per day without bulk. It renews at $29 per month unless canceled. Pro provides complete available history, 5,000 requests per day, and bulk access:

https://tradevodata.com/?utm_content=blog-how-to-backtest-value-investing-without-lookahead-bias&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate

The goal is not to make a backtest look better. It is to ensure every simulated decision uses only the universe, prices, and accounting information that were actually available at that moment.

Not investment advice. Verify competitor coverage, licensing, and pricing directly.

SEC EDGAR Companyfacts API Tutorial

Christian Pichichero — Sat, 17 Oct 2026 17:22:41 +0000

The SEC EDGAR APIs provide public filing data. For developers building US equity research pipelines, two resources are especially useful:

Companyfacts: XBRL facts reported by one company.
Submissions: filing history and filing metadata for one company.

The data is valuable, but it is not a ready-made point-in-time fundamentals panel. You still need to resolve tickers to CIKs, select tags and units, distinguish annual from quarterly facts, track filing dates, handle amendments and restatements, and prevent future information from leaking into historical tests.

This tutorial covers that plumbing directly. If you would rather inspect a finished example first, Tradevo Data publishes a no-signup proof pack at https://github.com/christianpichichero-max/pit-fundamentals.

The two SEC endpoints you need

SEC requests use a zero-padded, 10-digit Central Index Key rather than a ticker.

https://data.sec.gov/api/xbrl/companyfacts/CIK0000320193.json
https://data.sec.gov/submissions/CIK0000320193.json

The first URL returns company facts grouped broadly as:

facts -> taxonomy -> concept -> units -> observations

A typical observation may contain:

fy: fiscal year
fp: fiscal period, such as FY or Q1
form: filing form, such as 10-K or 10-Q
filed: filing date
start and end: fact period boundaries
val: reported value
accn: filing accession number
frame: an SEC calendar frame when available

The submissions response contains filing metadata, including accession numbers, forms, filing dates, report dates, and primary documents. It is useful for joining facts back to filings and inspecting amendments.

Respect SEC access rules

Consult the SEC's current developer and fair-access guidance before operating an automated client. Identify requests through a descriptive User-Agent, use a real application name and contact address, and apply conservative rate limiting.

A conservative serial Python session looks like this:

import time
import requests

HEADERS = {
    "User-Agent": "FundamentalsResearch/1.0 research@example.com",
    "Accept-Encoding": "gzip, deflate",
}

session = requests.Session()
session.headers.update(HEADERS)


def sec_get(url):
    response = session.get(url, timeout=30)
    response.raise_for_status()
    time.sleep(0.11)
    return response.json()

For production ingestion, add caching, exponential backoff, retry limits, logging, and a shared rate limiter. Do not let multiple workers independently assume they each have the full request allowance.

Resolve a ticker to a CIK

The SEC publishes a company-ticker mapping at:

https://www.sec.gov/files/company_tickers.json

Use it to build a case-insensitive lookup and pad the numeric CIK:

TICKERS_URL = "https://www.sec.gov/files/company_tickers.json"


def ticker_to_cik(ticker):
    records = sec_get(TICKERS_URL)
    target = ticker.upper()

    for record in records.values():
        if record["ticker"].upper() == target:
            return str(record["cik_str"]).zfill(10)

    raise KeyError(f"Ticker not found: {ticker}")

Cache this mapping. Tickers are convenient identifiers, but they can change or be reused; keep the CIK in your internal security model.

Download companyfacts and submissions


def get_companyfacts(cik):
    url = f"https://data.sec.gov/api/xbrl/companyfacts/CIK{cik}.json"
    return sec_get(url)


def get_submissions(cik):
    url = f"https://data.sec.gov/submissions/CIK{cik}.json"
    return sec_get(url)


cik = ticker_to_cik("AAPL")
companyfacts = get_companyfacts(cik)
submissions = get_submissions(cik)

Companyfacts may include standard us-gaap concepts and company-specific extension concepts. Do not assume every issuer reports the same economic item under the same tag throughout its history.

Extract a concept without losing provenance

Here is a minimal extractor for observations associated with a concept and preferred unit:

import pandas as pd


def extract_concept(data, concept, taxonomy="us-gaap", unit="USD"):
    concept_data = data["facts"][taxonomy][concept]
    observations = concept_data["units"].get(unit, [])

    rows = []
    for item in observations:
        rows.append({
            "entity": data.get("entityName"),
            "cik": data.get("cik"),
            "taxonomy": taxonomy,
            "concept": concept,
            "label": concept_data.get("label"),
            "unit": unit,
            "start": item.get("start"),
            "end": item.get("end"),
            "value": item.get("val"),
            "filed": item.get("filed"),
            "form": item.get("form"),
            "fiscal_year": item.get("fy"),
            "fiscal_period": item.get("fp"),
            "accession": item.get("accn"),
            "frame": item.get("frame"),
        })

    return pd.DataFrame(rows)


revenue = extract_concept(companyfacts, "RevenueFromContractWithCustomerExcludingAssessedTax")
revenue = revenue[revenue["form"].isin(["10-K", "10-Q", "10-K/A", "10-Q/A"])]

Keep the accession number, filing date, form, period boundaries, taxonomy, concept, and unit. Collapsing immediately to fiscal_year -> value destroys information needed to audit duplicates and revisions.

Gotcha 1: XBRL tag drift

Revenue is not always represented by one universal tag. An issuer may use a legacy standard tag, migrate to a newer tag, or report an extension concept. Similar problems occur with cash flow and balance-sheet items.

A durable concept mapping therefore needs:

Candidate tags for each economic concept.
Valid units and expected period type.
Issuer- and date-aware precedence rules.
Overlap checks when tags coexist.
QA flags rather than silent guesses.

Do not add every revenue-like tag together. Two tags may be alternative representations of the same total, or one may be a component of another.

Gotcha 2: annual, quarterly, and Q4 values

A fact from a 10-Q may cover a quarter or a year-to-date interval. A 10-K annual fact does not automatically provide a separately tagged fourth quarter.

You can sometimes derive Q4 as annual minus the first three quarters, but only when periods, units, concepts, and accounting scope align. Label derived values explicitly. Per-share values and diluted share counts are especially unsafe to derive through simple subtraction because averaging and weighting matter.

Treat fp, form type, start, end, and duration together. The frame field can help, but it is not present or sufficient for every observation.

Gotcha 3: restatements and duplicate periods

The same fiscal period may appear in an original filing, a later comparative filing, and an amendment. Those values can differ.

For each logical fact, preserve at least:

the first date it became public;
the first-reported value;
later reported values;
accession and form provenance;
a revision or restatement indicator.

A latest-only table answers, “What does the record say now?” A point-in-time table answers, “What could have been known on this date?” Those are different datasets.

Gotcha 4: filing lag and lookahead bias

Fiscal period end is not the information-availability date. If a company's year ends in December and its filing arrives later, a backtest must not expose that filing in December.

For an as-of date, filter facts using filing availability:


def as_of_view(df, as_of):
    cutoff = pd.Timestamp(as_of)
    work = df.copy()
    work["filed"] = pd.to_datetime(work["filed"])
    work = work[work["filed"] <= cutoff]

    keys = ["cik", "concept", "unit", "start", "end"]
    return (
        work.sort_values([*keys, "filed", "accession"])
            .groupby(keys, as_index=False)
            .tail(1)
    )

This is only a starting point. Same-day treatment, amendments, tag equivalence, duplicate contexts, and filing acceptance timestamps may require an explicit policy. For the broader research issue, see https://tradevodata.com/blog/lookahead-bias-fundamental-backtests?utm_content=blog-sec-edgar-companyfacts-api-tutorial&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate and https://tradevodata.com/blog/point-in-time-fundamentals-data?utm_content=blog-sec-edgar-companyfacts-api-tutorial&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

Raw EDGAR versus a normalized API

Question	Build directly on SEC EDGAR	Use Tradevo Data
Source	Public SEC filing data	Point-in-time US equity fundamentals from SEC EDGAR
Access model	Raw companyfacts and submissions JSON	One normalized JSON endpoint plus Pro bulk downloads
Concept mapping	You design and maintain it	16 annual concepts and 7 quarterly concepts
Point-in-time logic	You implement filing-date and revision rules	Server-side `as_of`, inclusive when `first_filed <= as_of`
Restatements	You identify and retain revisions	`original_value`, `latest_value`, restatement flag, and `qa_status`
Coverage	Depends on the pipeline you build	Annual: 5,168 US companies and 632,466 rows; quarterly: 5,382 companies and 1,089,635 rows; no non-US or delisted coverage
Formats	SEC JSON and your own storage	JSON API and bulk downloads; Parquet is not included
Cost	Public source; engineering and operations remain	Card-backed seven-day trial that renews at $29/month unless canceled

Tradevo Data is an honest budget tier of research-grade point-in-time data, not the only credible option. Sharadar, Tiingo, and QuantConnect may fit different workflows. See their pricing pages and documentation directly: https://data.nasdaq.com/databases/SF1, https://www.tiingo.com/pricing, and https://www.quantconnect.com/pricing.

When alternatives win

Raw EDGAR wins when you need custom XBRL concepts, filing-level text, bespoke amendment policies, extension-tag research, broader filing coverage, or complete control over lineage. It can also win when maintaining ingestion and normalization is part of your research advantage.

Sharadar, Tiingo, or QuantConnect may win when their coverage, delivery model, integrations, research environment, or supported workflows better match your needs. Verify current features and terms in each provider's documentation and see their pricing pages directly: https://data.nasdaq.com/databases/SF1, https://www.tiingo.com/pricing, and https://www.quantconnect.com/pricing.

Tradevo Data is not a fit if you need non-US or delisted coverage, Parquet delivery, derived TTM values, or concepts outside its supported annual and quarterly sets.

When to build it yourself

Build directly on EDGAR if you can commit to monitoring schema changes, updating mappings, rerunning QA, storing historical revisions, and documenting every point-in-time decision. The source is public; the practical cost is sustained engineering attention.

When a maintained layer is more practical

A normalized service can be useful when your objective is testing research ideas rather than operating XBRL infrastructure. Tradevo Data exposes:

/v1/fundamentals?ticker&as_of[&concept][&period=annual|quarterly]

Annual is the default. Pro provides complete available history, 5,000 requests per day, and bulk access through /v1/download and /v1/snapshot?as_of, each supporting period=annual|quarterly.

Its annual dataset contains 632,466 point-in-time rows across 5,168 US companies, with up to 12 fiscal years and 16 concepts: Revenue, NetIncome, Assets, StockholdersEquity, OperatingCashFlow, EPSDiluted, DilutedShares, GrossProfit, OperatingIncome, PretaxIncome, IncomeTaxExpense, CapitalExpenditures, CashAndCashEquivalents, CurrentAssets, CurrentLiabilities, and NetPPE. Its quarterly dataset contains 1,089,635 rows across 5,382 companies and 7 concepts. Q4 is reported where tagged or derived and labelled where supported; EPS and share counts are not derived for Q4. TTM values are not provided.

Each row includes first_filed, original_value, latest_value, a restatement flag for same-tag changes above 0.5%, including amendments, and qa_status. The dataset labels 37,804 restatements.

Start with the public proof pack: https://github.com/christianpichichero-max/pit-fundamentals. It contains 225 rows covering five companies and their latest three fiscal years, plus the methodology, without signup.

If the schema fits your work, the card-backed seven-day free trial at https://tradevodata.com/?utm_content=blog-sec-edgar-companyfacts-api-tutorial&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate covers 10 companies, a rolling three-year history, and 100 requests per day without bulk. It renews at $29/month unless canceled. Documentation is at https://tradevodata.com/docs?utm_content=blog-sec-edgar-companyfacts-api-tutorial&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

Not investment advice.

Why a Scheduler Cannot Trust Its Own Database

Christian Pichichero — Thu, 15 Oct 2026 18:40:43 +0000

If your scheduler submits transactions, its database is not a record of current chain state. It is a local projection assembled from RPC reads, transaction submissions, receipts, logs, and assumptions about what happened between those events.

That projection is useful. Scheduling would be impractical if every decision required rebuilding state from chain history. But the projection is always slightly wrong. The important engineering question is not whether it becomes stale; it is how the system behaves when it does.

There are two failure directions: stale-permissive state and stale-restrictive state. They have different costs and need different controls.

The database is authoritative about intent, not chain facts

A scheduler can authoritatively say, “I created this job,” “I attempted this transaction,” or “I assigned this idempotency key.” Those are facts owned by the application.

It cannot authoritatively say, “This wallet still has enough balance,” “This nonce is unused,” “This contract remains unpaused,” or “That transaction is final.” Those are chain facts. The database can cache observations of them, but an observation has a block context and an age.

Even an RPC read is not an timeless answer. A read against a recent block can be displaced by a reorganization. Different RPC nodes can briefly disagree. A transaction visible in the pending pool may disappear, be replaced, or execute with different surrounding state than the scheduler expected.

Treating cached chain data as ordinary application state hides all of this uncertainty.

Stale-permissive: the database says yes

Stale-permissive state means the local model allows an action that current chain state rejects or makes unsafe.

Common examples include:

A cached balance still includes funds already spent elsewhere.
A nonce has been consumed by another transaction.
An authorization was removed after the last sync.
A contract parameter changed, invalidating the scheduler’s assumptions.
A previous transaction succeeded, but the local worker timed out and scheduled the same intent again.

The obvious cost is a reverted transaction and wasted gas. The less obvious cost is incorrect execution. A transaction can succeed while doing something different from what the scheduler intended because price, liquidity, contract configuration, or account state changed before inclusion.

Preflight simulation helps, but it is not a lock on future state. Other transactions can land between simulation and execution. The right interpretation is “this action was valid against this observed state,” not “this action will remain valid.”

Safety-critical predicates therefore need checks as close to submission as practical. The scheduler should also encode constraints in the transaction itself when the contract interface supports them. A database check can reduce bad attempts; transaction-level constraints can stop execution when state moves beyond accepted bounds.

Stale-restrictive: the database says no

Stale-restrictive state is the mirror case. The database blocks an action that chain state would allow.

This often happens when:

A worker marks an intent as pending and crashes before clearing its lease.
A transaction is replaced, but the database keeps watching the old hash.
A receipt or log is missed because an RPC request failed.
A transaction succeeds while the local record remains in a submitted state.
A temporary chain condition clears, but the scheduler never revisits the job.

This direction usually preserves funds but harms liveness. Jobs remain stuck, valid execution windows pass, and operators have to repair records manually. Over time, conservative patches can turn transient uncertainty into permanent blockage.

A common mistake is to treat “unknown” as “failed” or “still pending.” Unknown should be its own state. It tells the reconciliation process to gather evidence rather than blindly submit again or wait forever.

A simple failure sequence

Consider a scheduler that records an intended transaction and then broadcasts it:

intent: created
transaction: submitted
local state: pending
worker: crashes
chain: transaction succeeds
receipt update: never written

On restart, resubmitting may duplicate the action. Refusing to resubmit may leave the job stuck indefinitely.

The scheduler needs a third path: inspect the account nonce, transaction receipt, relevant logs, and resulting contract state. It can then classify the intent using chain evidence. That process is reconciliation.

Reconciliation is a control loop

Reconciliation should run continuously, not as an incident script. A practical design separates command handling from observation:

Persist an intent before submission, with an idempotency key tied to the logical action.
Record every submission attempt and transaction hash.
Advance state through explicit transitions rather than overwriting a generic status field.
Scan receipts and logs from a durable checkpoint.
Re-read critical contract state for records that remain uncertain.
Expire worker leases independently from transaction status.
Revisit recently confirmed records when the chain can still reorganize.
Rewind derived state when a previously observed block leaves the canonical chain.

Logs are efficient for broad discovery, while direct reads are useful for checking current state. Neither is sufficient alone. A log scanner can miss data because of checkpoint bugs or provider failures. A direct read can show the current value without explaining which local intents produced it.

Reconciliation also needs bounded concurrency and rate controls. An outage can create a large backlog, and an eager recovery worker can overwhelm the same RPC provider it depends on. The recovery path should be designed as normal production traffic, not exceptional traffic with unlimited resources.

Decide where freshness is worth its cost

Reading chain state before every action reduces some stale-permissive failures, but adds latency and consumes RPC budget. It can also create a false sense of certainty because the state can change after the read.

Skipping fresh reads improves throughput, but shifts more responsibility to transaction constraints, idempotency, and later reconciliation. For low-risk observations, cached state may be acceptable. For actions involving balances, permissions, nonces, or irreversible contract calls, fresher evidence is usually worth more.

There is no universal freshness policy. The honest trade-off is latency and RPC usage versus correctness risk. Make that decision per predicate rather than applying one cache duration to the whole scheduler.

Disclosure: I build Tradevo, software that compiles plain-English crypto strategies into deterministic rules, tests and paper-trades them, supports review and marketplace publishing, and executes subscribed rules on Base from wallets users control.

A reliable scheduler does not eliminate stale state. It makes staleness visible, limits what stale assumptions can authorize, and continuously compares its local model with chain evidence. Reconciliation is not cleanup after the system fails. It is part of the system’s normal operation.

How to Find Financial Restatements in SEC Filings

Christian Pichichero — Wed, 14 Oct 2026 18:44:48 +0000

Financial restatements can be useful research signals. They show that a previously reported figure changed after becoming public—sometimes because of an error, sometimes because of an amendment, and sometimes because the company changed how a fact was tagged or presented.

For researchers, the central question is not merely: What does the latest filing say?

It is: What was first reported, when did it become public, and what changed afterward?

Answering that question requires point-in-time data. A current-only financial database may replace an original value with a revision, obscuring how the historical record changed.

This guide explains how to find financial restatements in SEC filings, compare original and revised values, and avoid false positives.

What counts as a financial restatement?

In a broad research workflow, a restatement flag can mean a meaningful change to a previously reported financial fact for the same company, fiscal period, and accounting concept.

That is broader than searching only for filings explicitly labeled as restatements. A changed value may appear in:

An amended filing such as a 10-K/A or 10-Q/A
A later filing that presents comparative prior-period figures
An XBRL fact using the same concept and period but a revised value
A filing that corrects or updates an earlier disclosure

Not every change means fraud, distress, or even an accounting error. Units, decimals, contexts, fiscal periods, taxonomy choices, and presentation changes can all create apparent differences. Treat a restatement flag as a research lead, not a verdict.

The reliable method: preserve original and revised values

To detect revisions, retain filing-time observations rather than storing only the latest value.

For each fact, capture relevant fields such as:

Company identifier and ticker
SEC filing accession or source filing
Filing or acceptance date
Form type
Fiscal period and fiscal year
XBRL concept or tag
Units
Context dimensions
Reported value
Whether the filing is an amendment

Then group comparable observations by company, period, concept, units, and compatible context.

Within each group, sort observations by the date they became public. The earliest valid observation becomes the original value. The most recent comparable observation becomes the latest value.

A simple relative-change test is:

relative_change = abs(latest_value - original_value) / abs(original_value)

Tradevo Data labels a row as restated when the same-tag value changes by more than 0.5%, including amendments. This is its documented classification rule, not a determination that a change is legally or economically material.

The edge case is an original value of zero. Relative change is undefined there, so a detection system needs a separately documented rule rather than an arbitrary percentage.

Why the same-tag requirement matters

Comparing values across different XBRL concepts can produce misleading alerts.

For example, a company may move from a standard taxonomy concept to a custom extension or disclose a related but not identical measure. Even when labels look similar, the accounting meaning may differ.

A conservative detector can begin by comparing values reported under the same tag. This may not capture every economically equivalent revision, but it reduces semantic mismatches and makes each flag easier to audit.

A more advanced pipeline can map extensions to standard concepts, but that requires validation and should preserve the original tags for inspection.

A practical SEC EDGAR workflow

1. Identify the original filing

Start with the relevant 10-K or 10-Q in SEC EDGAR. Record the filing date, accession, fiscal period, units, context, and reported value.

2. Search for amendments

Check for 10-K/A or 10-Q/A filings. Amendments are an obvious place to look, but do not stop there. A later annual or quarterly filing may present a changed comparative prior-period value.

3. Match the accounting fact

Compare the same XBRL concept for the same fiscal period, units, and compatible context. Watch for consolidated versus segment facts and continuing operations versus total-company measures.

4. Compare chronologically

Sort matched observations by when they became public. Preserve the first-reported value even if a later filing changes it.

5. Apply a documented rule

Use a documented classification rule, such as Tradevo Data's greater-than-0.5% same-tag rule. Keep the underlying observations so researchers can investigate or override the classification.

6. Review the filing narrative

Read amendment explanations, footnotes, auditor language, accounting-policy disclosures, and internal-control sections. A numeric change identifies where to look; the filing may explain why it happened.

Why current-only databases can hide the signal

Suppose a company first reports an accounting value and later revises it. If a database keeps only the revised figure, a historical backtest may behave as though that figure was known on the original date.

That creates lookahead bias.

A point-in-time record can instead separate:

first_filed: the date the value became public
original_value: the first-reported, point-in-time-safe value
latest_value: the current revision
restated: whether the same-tag change exceeds 0.5%
qa_status: a quality-control field

For an as_of query, Tradevo Data includes rows where first_filed <= as_of, with the selected date included. For a deeper treatment, see https://tradevodata.com/blog/lookahead-bias-fundamental-backtests?utm_content=blog-how-to-find-restatements-in-sec-filings&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate and https://tradevodata.com/blog/point-in-time-fundamentals-data?utm_content=blog-how-to-find-restatements-in-sec-filings&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

Using Tradevo Data to surface revisions

Tradevo Data provides point-in-time US equity fundamentals sourced from public-domain SEC EDGAR data. Its annual dataset contains 632,466 point-in-time rows across 5,168 US companies, covers 16 annual concepts, and labels 37,804 restatements. It also contains 1,089,635 quarterly rows across 5,382 companies and seven quarterly concepts.

The API exposes one fundamentals endpoint:

/v1/fundamentals?ticker&as_of[&concept][&period=annual|quarterly]

Annual is the default period. Server-side as_of filtering uses first_filed <= as_of, including the selected date.

Each row includes first_filed, original_value, latest_value, a restated flag, and qa_status. Researchers can use these fields to screen for changed facts before reviewing the underlying SEC filings.

Tradevo Data covers up to 12 fiscal years. It does not cover non-US or delisted companies and does not provide TTM figures. Quarterly Q4 is reported where tagged or derived and labeled where supported; Q4 EPS and share values are not derived. Parquet format is not included.

Options compared fairly

Option	Strengths	Limitations to evaluate	Best fit
SEC EDGAR	Primary public source with complete filing context and no vendor normalization	Building structured point-in-time data requires ingestion, XBRL normalization, chronology, deduplication, and QA	Researchers who need maximum control and can maintain infrastructure
Tradevo Data	Original and latest values, filing chronology, restatement flags, server-side `as_of`, and bulk access on Pro	US scope; no non-US, delisted, TTM, or Parquet coverage	Budget-conscious researchers who want structured point-in-time fundamentals
Sharadar	Credible established alternative	Check its current coverage, methodology, and point-in-time semantics against your requirements	Teams whose needs match its ecosystem; see their pricing page: https://data.nasdaq.com/databases/SF1
Tiingo	Credible API-oriented market-data provider	Verify current restatement fields, historical semantics, and coverage for your use case	Developers seeking a broader API stack; see their pricing page: https://www.tiingo.com/pricing/overview
QuantConnect	Integrated research and execution environment with supported datasets	Evaluate whether its platform model fits a need for standalone filing data	Researchers building inside its environment; see their pricing page: https://www.quantconnect.com/pricing/

Competitor capabilities and pricing can change. Review each provider's current documentation, methodology, and pricing page rather than comparing labels alone.

When another option wins

Use SEC EDGAR directly when you need complete filing narratives, exhibits, custom tags, or provenance beyond normalized numeric rows.

A broader vendor may win when you require international issuers, delisted-company coverage, additional asset classes, or a larger integrated data catalog. Sharadar or Tiingo may be a better fit when their broader products, coverage, or existing integration matches your workflow. QuantConnect may win when research, data, backtesting, and execution need to live in one environment.

Tradevo Data is positioned as an honest budget tier of research-grade point-in-time fundamentals—not as the only credible or affordable option.

When to build it yourself

Build your own pipeline when restatement detection is core intellectual property, when you need custom materiality rules, or when vendor normalization would conceal details essential to your research.

Be prepared to handle:

SEC submissions and accession histories
Filing dates and acceptance timestamps
XBRL contexts, units, dimensions, and custom extensions
Fiscal-calendar irregularities
Duplicate and conflicting facts
Amendments and later comparative disclosures
Taxonomy migrations
Reproducible quality checks

The advantage is control. The trade-off is continuing maintenance and validation, not just the initial parser.

Verify the method before paying

The public proof pack contains 225 rows for five companies, covering their latest three fiscal years, with full methodology and no signup:

https://github.com/christianpichichero-max/pit-fundamentals

Inspect the chronology, source logic, original values, and revisions before deciding whether the normalization fits your research.

If it does, the card-backed 7-day free trial at https://tradevodata.com/?utm_content=blog-how-to-find-restatements-in-sec-filings&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate covers 10 companies, a rolling 3-year history, and 100 requests per day without bulk. It renews at $29 per month unless canceled. Pro provides complete available history, 5,000 requests per day, and bulk annual or quarterly downloads through /v1/download and /v1/snapshot?as_of. Documentation is at https://tradevodata.com/docs?utm_content=blog-how-to-find-restatements-in-sec-filings&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

Restatement flags are best used as triage: identify changed facts, inspect the filing sequence, read the disclosures, and decide whether the revision is relevant to your research.

Not investment advice.

Fundamentals Data for Finance LLM AI Agents: A Practical Grounding Layer

Christian Pichichero — Sat, 10 Oct 2026 18:44:17 +0000

Finance AI agents have a basic reliability problem: language models can produce plausible answers, including plausible numbers. Plausible is not enough when a user asks what a company reported, when the information became public, or whether the figure was later restated.

One practical approach is a grounding layer built from dated financial facts.

For developers building retrieval-augmented generation systems, filing copilots, screening agents, research assistants, or automated company briefs, point-in-time fundamentals can provide a structured bridge between SEC filings and model-generated explanations. The model can reason over the data, but it should not invent the data.

Tradevo Data provides point-in-time US equity fundamentals sourced from SEC EDGAR. Its annual dataset covers 5,168 US companies, 632,466 point-in-time rows, 16 annual concepts, and up to 12 fiscal years. Its quarterly dataset contains 1,089,635 rows across 5,382 companies and seven quarterly concepts. Coverage varies by filer.

Why ordinary fundamentals can mislead an AI agent

A conventional fundamentals database may emphasize the latest known value. That can be useful for current analysis, but it may be inappropriate for a historical question.

Suppose an agent is asked:

What revenue and net income were publicly available for a company on June 15 of a past year?

If retrieval returns a figure restated months later, the answer contains information that was unavailable on June 15. The number may now be technically correct while still being historically invalid for that date.

That distinction matters for:

Historical research and event reconstruction
Backtests using fundamental signals
Filing-aware RAG systems
Agents answering “as of” questions
Audit trails for generated financial summaries
Comparing first-reported figures with subsequent revisions

This is lookahead bias at the data layer. A longer explanation is available at https://tradevodata.com/blog/lookahead-bias-fundamental-backtests?utm_content=blog-grounding-a-finance-llm-in-as-reported-fundamentals&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

The fields an agent needs for defensible answers

A useful finance grounding record should do more than return a value. It should establish provenance.

Each Tradevo Data row includes:

first_filed: the date the value became public
original_value: the first-reported, point-in-time-safe value
latest_value: the current revision
restated: a flag for a greater-than-0.5% change under the same XBRL tag, including amendments
qa_status: a quality-assurance status

The dataset labels 37,804 restatements.

For an AI agent, those fields support a stronger response pattern:

Retrieve facts available by the requested date.
Use original_value for an as-reported historical answer.
Compare it with latest_value if revisions matter.
Include first_filed and the requested concept in the response.
State uncertainty or coverage gaps instead of filling them with generated numbers.

An XBRL tag is not a complete semantic guarantee: filers can use extensions, presentation varies, and accounting context still matters. But a dated record containing the original value, revised value, restatement flag, and QA status is more auditable than an unsupported number in model output.

Example query for a finance AI agent

Tradevo Data exposes one JSON fundamentals endpoint with server-side point-in-time filtering:

GET https://tradevodata.com/v1/fundamentals?ticker=AAPL&as_of=2020-06-15&concept=Revenue&period=annual&utm_content=blog-grounding-a-finance-llm-in-as-reported-fundamentals&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate

The server applies:

first_filed <= as_of

The date is inclusive. Annual data is the default, while period=quarterly requests quarterly rows.

A basic agent tool contract might look like this:

{
  "tool": "get_fundamentals",
  "arguments": {
    "ticker": "AAPL",
    "as_of": "2020-06-15",
    "concept": "Revenue",
    "period": "annual"
  }
}

The model should then compose an answer from returned records rather than relying on memorized parameters. A system instruction can require it to include the fiscal period, first_filed, original_value, and concept in every numerical citation.

For example:

Use only retrieved values. If no qualifying row exists by the requested
as-of date, say that the dataset returned no supported value. Do not infer
or interpolate a financial figure.

That final rule is important. Retrieval does not prevent hallucination unless the agent is explicitly required to abstain when evidence is missing.

A practical RAG architecture

A filing agent can separate structured facts from unstructured context:

Use point-in-time fundamentals for supported numerical claims.
Use filing text retrieval for management commentary, accounting policies, and risk disclosures.
Store source metadata alongside every chunk or fact.
Require citations in the final answer.
Validate numerical statements against retrieved structured records before returning them.

Structured fundamentals are not a replacement for the filing. They can serve as a compact numerical layer alongside it.

For bulk workflows, the $29-per-month Pro plan includes /v1/download and /v1/snapshot?as_of, each supporting period=annual|quarterly. These endpoints can support a local retrieval store or reproducible historical snapshot. Parquet format is not included.

More background on the data model is available at https://tradevodata.com/blog/point-in-time-fundamentals-data?utm_content=blog-grounding-a-finance-llm-in-as-reported-fundamentals&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

What Tradevo Data covers—and what it does not

The annual dataset contains these 16 concepts:

Revenue
NetIncome
Assets
StockholdersEquity
OperatingCashFlow
EPSDiluted
DilutedShares
GrossProfit
OperatingIncome
PretaxIncome
IncomeTaxExpense
CapitalExpenditures
CashAndCashEquivalents
CurrentAssets
CurrentLiabilities
NetPPE

Quarterly coverage includes seven concepts. Q4 is reported where tagged or derived and labelled where supported. Derived Q4 EPS and share figures are not provided.

The service does not provide TTM calculations, non-US coverage, delisted-company coverage, or Parquet output. It is not presented as a full filing-text corpus, market-data feed, estimates database, or accounting ontology.

These limits matter. An agent should not present absence as zero, assume every filer reports every concept, or silently substitute one accounting concept for another.

Fair comparison for finance-agent developers

Option	Best fit	Potential strengths	Important considerations
Tradevo Data	Developers wanting a budget-tier API and bulk access for core US fundamentals	Server-side `as_of`, original and latest values, filing dates, restatement labels, and QA status	Limited concept set; no delisted, TTM, non-US, or Parquet coverage
Raw SEC EDGAR/XBRL	Teams needing maximum control and direct primary-source processing	Public-domain source material and direct control over interpretation	Requires ingestion, taxonomy mapping, duplicate handling, amendment processing, QA, and point-in-time logic
Sharadar	Researchers evaluating a broader established data product	A credible alternative with its own coverage and data model	Confirm current scope and licensing; see their pricing page: https://data.nasdaq.com/databases/SF1
Tiingo	Developers evaluating fundamentals alongside other financial APIs	A credible provider with a broader product context	Verify point-in-time semantics and coverage; see their pricing page: https://www.tiingo.com/pricing/overview
QuantConnect	Teams building inside an integrated research and execution platform	Data access within a broader algorithmic environment	Platform fit may matter more than a standalone fundamentals API; see their pricing page: https://www.quantconnect.com/pricing/

No provider should be chosen from a feature label alone. Test amendment handling, filing-date semantics, survivorship assumptions, missing values, concept normalization, redistribution rights, and historical coverage against your actual agent workflow.

When another option wins

Another provider can be the better choice when you need a broader security universe, delisted companies, non-US issuers, more accounting concepts, analyst estimates, TTM figures, market data, or a managed research platform. A specialist vendor may also win when your institution needs enterprise support, contractual service levels, or licensing terms beyond a developer-oriented product.

If your workflow already runs inside QuantConnect, its integrated environment may reduce engineering work. If you need a broader commercial dataset, Sharadar or Tiingo may fit better after you verify current coverage and point-in-time behavior.

Tradevo Data is positioned as an honest budget tier of research-grade point-in-time data, not as the only affordable option and not as a universal financial-data layer.

When to build it yourself

Build directly from SEC EDGAR when provenance control is more important than implementation speed, your required concepts fall outside the supported set, or your team has specialized accounting and data-engineering expertise.

Be prepared to handle:

Filing and amendment chronology
XBRL standard tags and company extensions
Units, periods, contexts, and duplicate facts
Fiscal-year differences
Restatements and comparable-tag logic
Quarterly versus year-to-date facts
Q4 derivation rules
Missing and malformed filings
Reprocessing when extraction rules change
Reproducible historical snapshots

EDGAR is public domain, so self-building can be rational. The practical burden is maintaining consistent interpretation and QA over time.

Inspect the evidence before connecting an agent

A public proof pack is available at https://github.com/christianpichichero-max/pit-fundamentals. It contains five companies, their latest three fiscal years, 225 rows, and the full methodology with no signup.

On reliable-filing rows in that five-company proof pack only, measured lookahead averaged 35.2 days and reached a maximum of 48 days. Those measurements describe the proof pack, not the full dataset.

For API evaluation, https://tradevodata.com/?utm_content=blog-grounding-a-finance-llm-in-as-reported-fundamentals&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate offers a card-backed seven-day free trial covering 10 companies, a rolling three-year history, and 100 requests per day without bulk. Unless canceled, it renews at $29 per month. Pro provides complete available history, 5,000 requests per day, and bulk endpoints. Documentation is at https://tradevodata.com/docs?utm_content=blog-grounding-a-finance-llm-in-as-reported-fundamentals&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

Start with the public sample, test the dates and concepts against source filings, and make abstention part of the agent design. The goal is not to make a model sound certain. It is to make supported financial numbers traceable.

Not investment advice.

A Transaction Timeout Is Not a Transaction Failure

Christian Pichichero — Thu, 08 Oct 2026 20:09:56 +0000

You broadcast a transaction, wait for the RPC response, and get a timeout. What happened?

The honest answer is that you do not know. The node may never have received the request. It may have accepted the transaction into its mempool but lost the response. Another node may already have propagated it. The transaction may even be mined while your executor records a network error.

Treating that timeout as a failure and sending a fresh transaction can execute the same business action twice.

Separate the request from the transaction

An RPC request and a blockchain transaction have different lifecycles. A request can fail at the HTTP layer while the transaction continues through the network.

This distinction produces three outcomes, not two:

Succeeded: chain state shows the intended transaction was included.
Failed: chain state shows a terminal failure, such as a reverted transaction.
Uncertain: the executor cannot yet establish either outcome.

uncertain is not a softer spelling of failed. It is a durable state that needs reconciliation.

Persist the intent before broadcasting

The first durable record should describe the business action, not the RPC attempt. Write it before signing or sending anything.

A minimal record might look like this:

intent_id:      rebalance:account-42:2026-09-03T10
account:        0xabc...
chain_id:       8453
operation:      swap token A for token B
status:         created
nonce:          null
signed_tx:      null
tx_hashes:      []

Give intent_id a uniqueness constraint based on the business operation. If two workers receive the same job, they should load the same intent rather than create independent transactions.

Next, reserve a nonce, construct the transaction, sign it, and persist the signed bytes and hash before broadcast. This ordering lets a restarted worker send the exact same payload.

A database entry marked broadcasting does not prove that any node saw the transaction. Likewise, a missing sent log does not prove that no node saw it. Your database records what the executor attempted; the chain records what took effect.

Use nonces deliberately

For an EVM externally owned account, the nonce is the main duplicate-execution boundary. Two transactions from the same account with the same nonce cannot both become canonical.

That does not mean an executor can derive any nonce from an intent ID. Account nonces are sequential. A practical nonce manager must serialize reservations per account and persist them so concurrent workers do not choose the same nonce for unrelated intents.

After a timeout, rebroadcasting the exact signed bytes is the simplest safe retry. The transaction hash stays the same, and nodes that already know it can treat the broadcast as redundant.

Fee replacement is more complicated. A replacement uses the same nonce but changes fee fields, producing a new hash. Persist every candidate hash under the same intent and monitor all of them. Never respond to a timeout by allocating a fresh nonce to the same action.

If the original transaction has already been mined, a replacement may be rejected because the nonce is too low. That error is a reason to inspect chain state, not to create another transaction.

Smart-contract accounts can provide a stronger application-level boundary by storing an operation ID and rejecting duplicate execution. That handles cases an account nonce cannot express, but it adds contract complexity and execution cost. Sequential EOA nonces are simpler, though they can become a throughput bottleneck.

Reconcile with chain state

A reconciliation worker should examine evidence in roughly this order:

Check receipts for every transaction hash attached to the intent.
Check whether the sending account's nonce has been consumed.
Inspect the destination contract's state or events for the intended operation.
Wait for the confirmation depth required by the application before marking the intent final.

A single RPC node saying “not found” is weak evidence. Mempools differ, nodes can lag, and old pending transactions may disappear from local views. Querying another provider can improve visibility, but it still does not turn absence into proof.

If the account nonce has advanced, determine which transaction consumed it. Your intended transaction may have been included, replaced, or displaced by another process using the same account.

Reorganizations also matter. included and finalized should be separate states. An included transaction can temporarily disappear and need further reconciliation.

The ugly middle state

Suppose the broadcast timed out, no receipt is visible, the transaction is absent from the RPC node, and the account nonce has not moved. The transaction might still be propagating, or every node might have dropped it.

There is no honest terminal answer yet.

Keep the intent uncertain, rebroadcast the same signed payload, and schedule another reconciliation pass. Time alone should not silently convert uncertainty into failure. If the application eventually stops retrying, preserve the unresolved status for operator inspection.

A “cancel” transaction is also only a competing transaction with the same nonce. It can win the race, or the original can be mined first. Model the outcome from chain evidence rather than from the fact that a cancellation request was sent.

Disclosure: I build Tradevo, which compiles plain-English crypto strategies into deterministic rules, runs historical tests and paper trading, lets creators submit strategies for human review and publish approved strategies to a marketplace, and lets subscribers run published rules from wallets they control.

A useful executor invariant

The core invariant is simple:

One business intent may have several broadcast attempts and transaction hashes, but it must not silently become several independent nonce-bearing actions.

That requires coordination across the database, signer, nonce manager, broadcaster, and reconciliation worker. Idempotency is not a retry flag attached to an RPC call. It is a state machine built around the possibility that the network acted while your process learned nothing.

Point-in-Time Data for Academic Finance Research: A Reproducibility Guide

Christian Pichichero — Wed, 07 Oct 2026 20:02:49 +0000

In empirical finance, a backtest should use only information available on the simulated decision date. If it uses financial statements based on fiscal period-end dates, current database values, or later restatements, the model may incorporate facts that had not yet been published.

That is lookahead bias. It can affect estimated predictability, reported coefficients, and reproducibility.

Point-in-time, or PIT, fundamentals record when each value became public and allow researchers to reconstruct the information set available as of a historical date. For accounting-based asset-pricing signals, PIT treatment can be an important part of research design.

Why ordinary fundamentals may not be enough

Suppose a company has a fiscal year ending December 31. Its annual revenue does not necessarily become public on December 31. The filing may arrive weeks later, and an amendment or restatement may subsequently change the value.

A conventional panel might contain:

the fiscal period end;
the latest known value;
a standardized concept;
the company identifier.

That panel may be useful for descriptive tasks, but it does not necessarily answer the historical question: What value could a researcher have observed on the portfolio formation date?

A PIT row needs an availability date or timestamp. Tradevo Data exposes first_filed, defined as the date the value became public. Its API applies first_filed <= as_of, including same-day filings. Each row also contains:

original_value: the first-reported, PIT-safe value;
latest_value: the current revision;
restated: a flag for changes above 0.5% on the same tag, including amendments;
qa_status: a data-quality field.

The distinction between original and latest values matters. Replacing the original observation with a later revision can introduce future information even if the fiscal period itself is correctly lagged.

For a deeper treatment, see https://tradevodata.com/blog/lookahead-bias-fundamental-backtests?utm_content=blog-point-in-time-data-for-academic-finance-research&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

PIT design belongs in the methods section

A reproducible paper should state more than “fundamentals were lagged.” A fixed lag is a convention, not proof that the information was public.

A stronger methodology documents:

Availability rule: Which filing date or timestamp determines eligibility?
Boundary rule: Are filings published on the formation date included?
Revision policy: Does the study use first-reported or latest values?
Fiscal-period policy: How are annual observations aligned across firms?
Concept mapping: Which source tags map to each standardized variable?
Missingness policy: Are missing values dropped, imputed, or interpreted economically?
Universe policy: Does the source include delisted companies and non-US issuers?
Snapshot procedure: Can another researcher reconstruct the same extract later?

Tradevo Data's server-side endpoint is:

/v1/fundamentals?ticker&as_of[&concept][&period=annual|quarterly]

Annual is the default. Bulk access is available through /v1/download and /v1/snapshot?as_of, each accepting period=annual|quarterly. The snapshot route makes the historical cutoff explicit. Parquet format is not included.

What Tradevo Data covers—and what it does not

Tradevo Data provides point-in-time US equity fundamentals derived from public-domain SEC EDGAR filings. Its annual dataset contains 632,466 point-in-time rows across 5,168 US companies, with 16 annual concepts and up to 12 fiscal years. Its quarterly dataset contains 1,089,635 rows across 5,382 companies and seven quarterly concepts. Across the data, 37,804 restatements are labeled.

The annual concepts are Revenue, NetIncome, Assets, StockholdersEquity, OperatingCashFlow, EPSDiluted, DilutedShares, GrossProfit, OperatingIncome, PretaxIncome, IncomeTaxExpense, CapitalExpenditures, CashAndCashEquivalents, CurrentAssets, CurrentLiabilities, and NetPPE.

Quarterly data is also available with seven concepts. Q4 is reported where tagged or derived and labeled where supported. Derived Q4 EPS and shares are excluded.

The limitations are material:

US companies only;
no non-US coverage;
no delisted-company coverage;
no TTM fields;
a focused concept set rather than the breadth of a large institutional platform;
no included Parquet format.

These constraints should appear in a paper's data section. In particular, the absence of delisted companies may create survivorship concerns depending on sample construction and the research question.

Tradevo Data versus WRDS and Compustat

WRDS is an institutional research platform through which universities and other organizations may access datasets such as Compustat, subject to their subscriptions and licenses. It may be the stronger choice when a study requires broader historical coverage, standardized identifiers across databases, delisted securities, international accounting data, or established institutional workflows.

Tradevo Data is a narrower, budget-tier PIT source for researchers who need US filing fundamentals without requiring university-gated access.

Criterion	Tradevo Data	WRDS/Compustat
Access model	Public API and bulk files; card-backed trial	Commonly accessed through subscribing institutions
Source orientation	Public-domain SEC EDGAR filings	Commercial research databases
PIT mechanism	`first_filed`, original and latest values, and server-side `as_of`	Capabilities depend on the subscribed dataset and fields
Coverage	US companies; focused annual and quarterly concepts	Potentially broader options, depending on institutional subscriptions
Delisted companies	Not covered	May be the better route when delisted coverage is required; verify the specific product
International research	Not covered	May be better suited when subscribed products cover the required markets
Reproducibility aids	Public methodology, no-signup sample, and dated snapshots	Institutional workflows subject to licensing and local access
Price	$29/month after the trial unless canceled	See their pricing or institutional-access page: https://wrds-www.wharton.upenn.edu/

This is not an argument that one source universally replaces the other. The appropriate source depends on the identification strategy, required universe, licensing constraints, and replication plan. Sharadar, Tiingo, and QuantConnect are also credible alternatives; compare their current documentation, coverage, PIT methodology, and pricing pages directly.

Public proof before signup

Tradevo Data publishes a no-signup proof pack at:

https://github.com/christianpichichero-max/pit-fundamentals

It contains 225 rows for five companies over their latest three fiscal years, plus the full methodology. Researchers can inspect the schema, test availability logic, and evaluate whether the concept mapping is suitable before entering payment details.

On reliable-filing rows in that five-company proof pack only, measured lookahead averaged 35.2 days and reached a maximum of 48 days. These figures are not evidence about the full dataset. They demonstrate within the sample how filing availability can differ from fiscal period-end dating.

More background on the data model is available at https://tradevodata.com/blog/point-in-time-fundamentals-data?utm_content=blog-point-in-time-data-for-academic-finance-research&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

A practical research workflow

A defensible workflow can be simple:

Pre-register or document the formation-date rule.
Query or snapshot the data with the required as_of date.
Use original_value for the historical information set unless the research question explicitly concerns revised accounts.
Retain first_filed, restated, and qa_status in intermediate files.
Archive query parameters, code, and extraction dates.
Report coverage exclusions, especially delisted and non-US companies.
Run sensitivity checks using alternative availability rules where justified.
Make a small, license-compliant replication extract available when possible.

The card-backed seven-day free trial covers 10 companies, a rolling three-year history, and 100 requests per day, without bulk access. It renews at $29/month unless canceled. Pro provides the complete available history, 5,000 requests per day, and bulk access through /v1/download and /v1/snapshot?as_of. Documentation is at https://tradevodata.com/docs?utm_content=blog-point-in-time-data-for-academic-finance-research&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate, and the trial is available at https://tradevodata.com/?utm_content=blog-point-in-time-data-for-academic-finance-research&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate.

When WRDS or another institutional source wins

Use WRDS, Compustat, or another appropriate institutional dataset when the paper requires:

delisted-company coverage to address survivorship bias;
non-US or cross-country samples;
history beyond Tradevo Data's available horizon;
accounting fields outside Tradevo Data's focused concept set;
linked returns, analyst estimates, ownership, executive, or security-master data;
identifiers and conventions already standard in a target literature;
a licensed source mandated by coauthors, reviewers, or the institution.

Researchers should inspect the exact subscribed product, documentation, PIT fields, and licensing terms rather than assume every dataset exposed through a platform has identical historical properties. For current access and cost information, see each provider's pricing page.

When to build it yourself

Building directly from SEC EDGAR can be the better choice when the extraction pipeline is itself a research contribution, when unusual XBRL tags are central to the hypothesis, or when complete control over transformation rules matters more than implementation time.

A credible build may need to handle filing chronology, amendments, duplicate facts, taxonomy changes, units, dimensions, fiscal calendars, concept mapping, accession-level provenance, and historical snapshots. It may also require tests for revisions and malformed filings.

The advantage is transparency and customization. The disadvantage is the required data-engineering work. One approach is to validate a managed dataset against raw filings for a stratified sample and publish the audit code.

The standard is reconstructability

Using a familiar database does not by itself make academic finance reproducible. The core question is whether another researcher can reconstruct the information set available on each simulated date and understand the transformations applied.

Tradevo Data offers a focused option for annual and quarterly US filing fundamentals, with explicit availability dates, original and revised values, restatement labels, snapshots, and a public methodology. It is not a substitute for broad institutional databases in every research design. It is an inspectable budget-tier option when its coverage matches the paper.

Start by auditing the free sample at https://github.com/christianpichichero-max/pit-fundamentals. If the schema and methodology fit the study, the card-backed seven-day free trial is available at https://tradevodata.com/?utm_content=blog-point-in-time-data-for-academic-finance-research&utm_source=devto&utm_medium=syndication&utm_campaign=syndicate and renews at $29/month unless canceled.

Not investment advice.

Sponsoring Gas Without Holding User Funds: Paymasters and the Trust Model

Christian Pichichero — Thu, 01 Oct 2026 19:50:45 +0000

The awkward edge case in a consumer wallet flow is the first transaction: the user has the asset your application needs, but none of the chain’s native token. Asking them to visit an exchange or bridge before they can use the app is technically honest and often terrible UX.

Account abstraction offers another option. A paymaster can cover the network fee while the user’s assets remain in their own smart account. That removes one prerequisite, but it also introduces a sponsor with policy control, infrastructure costs, and several ways to be abused.

The basic ERC-4337 flow

In ERC-4337, a wallet submits a UserOperation rather than sending a conventional transaction directly. Conceptually, the flow looks like this:

wallet builds and signs UserOperation
        |
        v
sponsor checks operation against its policy
        |
        v
sponsor adds signed paymaster data
        |
        v
bundler simulates and submits the operation
        |
        v
EntryPoint validates wallet and paymaster
        |
        v
call executes; fee is charged to paymaster deposit

The paymaster keeps native tokens deposited with the EntryPoint contract. Its validation logic decides whether it accepts responsibility for a particular operation. The EntryPoint then charges the actual execution cost to that deposit, subject to the operation’s gas fields and protocol rules.

Most consumer applications use a verifying paymaster. An off-chain service inspects the operation and signs sponsorship data if the request satisfies its policy. The on-chain paymaster verifies that signature during validation.

This split matters. The on-chain component checks that the sponsor approved the exact operation, while the off-chain service handles changing concerns such as rate limits, account status, application plans, and allowed contract methods.

What the sponsor can do

A sponsor can decide which operations it is willing to fund. Its policy might inspect:

The destination contract and function selector.
The chain and EntryPoint version.
Maximum gas and fee fields.
Calldata size.
Wallet, user, device, or subscription quotas.
Whether the operation was produced through an expected application flow.
Estimated cost under current network conditions.

The sponsor can also stop issuing approvals, lower limits, or make sponsorship conditional on its backend being available. That is a real dependency. A wallet may remain usable, but the sponsored path can disappear.

The sponsor also learns about an operation before inclusion. Even though the eventual transaction is public, advance visibility and request metadata can matter. Applications should avoid pretending that sponsorship is a privacy mechanism.

What the sponsor cannot do

Paying gas is not the same as authorizing the wallet’s call.

The smart account still validates the user’s signature or whatever authorization scheme the account implements. A paymaster cannot modify the signed destination, calldata, or value without invalidating that authorization. It also cannot spend wallet assets merely because it paid the network fee.

This distinction becomes blurred when one company operates several components. An application might run the sponsor, bundler access, and a delegated wallet signer. Those roles should be analyzed separately:

The paymaster decides whether to pay.
The bundler decides whether to submit a valid operation.
The wallet’s authorization logic decides which calls may execute.

A restricted paymaster policy does not make a broad wallet key safe. Likewise, a narrow wallet authorization does not prevent somebody from wasting an overly permissive sponsorship budget.

Open sponsorship is a griefing surface

A policy that effectively says “pay for any operation from our app” is easy to consume at scale. An attacker does not need to steal the sponsor’s deposit. They only need to create operations that qualify and cost money.

Simple per-wallet limits are weak if wallets are cheap to create. IP limits can block legitimate shared networks and are easy to distribute around. Requiring an application login raises the cost of abuse but moves the problem into account creation and session security.

Failed calls also deserve attention. An operation can pass validation and then revert during execution, consuming gas anyway. Simulation catches many failures, but state can change between simulation and inclusion. Deliberately expensive valid calls may still satisfy a policy that checks only destinations and selectors.

A practical policy therefore combines several controls:

Allow specific contracts and methods rather than arbitrary calls.
Cap gas fields and calldata size.
Check call arguments when their meaning is stable and unambiguous.
Set budgets across wallets, users, and time windows.
Reject stale sponsorship signatures.
Monitor cost per successful operation and cost per failed operation.
Add a global circuit breaker for unexpected spending.

None of these controls is free. More inspection creates more backend coupling and more ways to reject legitimate traffic after a contract upgrade.

Sponsorship is an operating decision

The contract is only part of the system. Someone must keep the paymaster deposit funded, monitor native-token exposure, handle fee spikes, and decide what happens when a daily budget is exhausted.

You also need an outage mode. Can users pay their own gas if the sponsor is unavailable? Does the interface explain that option before asking them to sign? Can another bundler submit the same operation, or have you coupled sponsorship to one provider’s API?

Cost attribution matters too. “Gasless” usually means somebody else receives the bill. A product team needs to know which user action created each charge, whether retries duplicate work, and how much abuse the support team is willing to tolerate. Sponsorship may be cheap during normal traffic but difficult to bound during a fee spike or automated attack.

Alternatives have valid trade-offs. Letting users pay is simpler and makes costs explicit. A faucet is straightforward but attracts automated abuse. Reimbursing users after execution still requires them to hold native tokens first. Application-specific relayers can work well when contracts already support meta-transactions, though they introduce their own trust and compatibility constraints.

Disclosure: I build Tradevo, which lets people build, test, paper-trade, publish, subscribe to, and automatically run crypto strategies from wallets they control; its free tier sponsors gas.

A paymaster improves onboarding by separating transaction authorization from fee payment. It does not remove trust; it moves part of the trust model into sponsorship policy and operations. Treat that policy like a security boundary, and treat its budget like production infrastructure rather than a promotional checkbox.

DEV Community: Christian Pichichero

Why a Lease Is Not Enough to Stop Duplicate Scheduled Jobs

The basic lease

The paused-worker failure

Fencing the side effect

Irreversible execution exposes the gap

What to use when the destination cannot fence

The practical rule

What Each Test Layer Proves When Code Moves Funds

Start with a deterministic execution plan

Use a fork to test contract interactions

Make dry run a capability boundary

Use a canary to test the production path

Production needs continuing controls

Free Financial Data APIs for Backtesting: An Honest Comparison

The short answer

Comparison table

1. SEC EDGAR: a free primary source

When EDGAR wins

2. yfinance: convenient, with limited guarantees

When yfinance wins

3. Stooq: practical market history with validation required

When Stooq wins

4. FRED and ALFRED: consider vintages for macro backtests

When FRED or ALFRED wins

5. Alpha Vantage: packaged APIs for quick prototypes

When Alpha Vantage wins

6. Tradevo Data: inspect the free PIT proof pack first

When Tradevo Data wins

Point-in-time safety checklist

When to build it yourself

A practical selection rule

“Non-Custodial” Is a Spectrum, Not a Boolean

1. A plain externally owned account

2. A smart account with a user-revocable permission

3. A delegated session signer removed by the operator

4. Threshold MPC custody

5. An exchange account

What to ask instead of accepting the label

How to Backtest Value Investing Without Lookahead Bias

Why ordinary fiscal-period joins fail

Step 1: define the strategy before touching the data

Step 2: build a point-in-time universe

Step 3: select fundamentals by first_filed

A small worked example

Pandas example using the free sample

Step 4: choose an explicit restatement policy

Step 5: test the pipeline for leakage

Data options: a fair comparison

When they win: another option may be better

When to build it yourself

A practical starting point

SEC EDGAR Companyfacts API Tutorial

The two SEC endpoints you need

Respect SEC access rules

Resolve a ticker to a CIK

Download companyfacts and submissions

Extract a concept without losing provenance

Gotcha 1: XBRL tag drift

Gotcha 2: annual, quarterly, and Q4 values

Gotcha 3: restatements and duplicate periods

Gotcha 4: filing lag and lookahead bias

Raw EDGAR versus a normalized API

When alternatives win

When to build it yourself

When a maintained layer is more practical

Why a Scheduler Cannot Trust Its Own Database

The database is authoritative about intent, not chain facts

Stale-permissive: the database says yes

Stale-restrictive: the database says no

A simple failure sequence

Reconciliation is a control loop

Decide where freshness is worth its cost

How to Find Financial Restatements in SEC Filings

What counts as a financial restatement?

The reliable method: preserve original and revised values

Why the same-tag requirement matters

A practical SEC EDGAR workflow

1. Identify the original filing

2. Search for amendments

Step 3: select fundamentals by `first_filed`