DEV Community: Walter

Jev Decoded: 67.8% Hit Rate, Still Lost Money — What Real Quant Backtests Show

Walter — Wed, 23 Sep 2026 07:51:20 +0000

67.8% directional accuracy. 339 decisions. Net result after fees: -62.69.

That's the outcome from Waxmell114514/jev-trade, an open-source project that wired Jev directly into NQ futures order book data and traded every decision. A second project, egrm07/jev_bitcoin_backtest, tested ten different data representations on BTC/USD 5-minute bars and ran a five-gate validation framework. Best holdout AUC after 65 days out-of-sample: 0.503 — statistically indistinguishable from random.

Both projects lost money. Neither result is a verdict on Jev's calibration quality. They're a diagnosis of where Jev is being placed in the system.

This post covers three things: what Jev actually does under the hood, why high hit rate doesn't prevent losses at the execution layer, and a specific data infrastructure problem that applies to any AI decision layer — not just Jev.

What Jev Is (and Is Not)

Jev, built by TypeSafe, skips something all standard LLMs do as a matter of course: it never generates text token by token to produce a structured answer.

Standard LLM classification workflow:

Process prompt → build KV cache
Generate {, then ", then s, then e... each token requires moving the cache through GPU memory
Parse the output string → extract "positive"

A three-way classification decision means roughly 12 tokens of autoregressive generation. Each token is a full forward pass, and the bottleneck isn't raw compute — it's memory bandwidth. Moving the KV cache back and forth is the slow part.

Jev's approach: one forward pass, shared KV cache across all batched questions, probabilities read directly from a numeric output head. The answer options aren't text the model writes out — they're dimensions in the computation graph.

Three question types:

Type	How it works	Returns
Multiple choice	Pick from predefined options	option + probability + confidence
Scoring	Evaluate against a rubric	score + probability + confidence
Judgment	Is this statement true or false	0–1 probability

No format validation. No JSON parsing retries. No structured output overhead.

One architectural note worth understanding: Jev's option probabilities are not independent. Add an irrelevant option and the others shift. This is the behavior of a classifier doing joint computation over the full option set — not a model scoring each option in isolation. It maps inputs to a predefined decision space in a single pass, more like a discriminative classifier than a generative model reasoning through an answer.

The Calibration Story: What RLCD Actually Claims

TypeSafe trained Jev with RLCD — Reinforcement Learning for Calibrated Decisions. The stated goal: when the model outputs 70% confidence, the event should actually occur roughly 70% of the time.

This matters more than it sounds, and the difference from standard RLHF is specific:

	Standard Training (RLHF)	Calibration Training (RLCD)
Optimization target	Human preference score	Calibration error
Signal	"Is this answer good?"	"Does 70% confidence → 70% correct?"
What the model learns	Produce outputs humans rate highly	Make probability statements accurate

A standard LLM can output confidence: 0.95 with no statistical basis — because high-confidence-sounding language correlates with human preference ratings. RLCD targets calibration error directly.

Independent test results:

Source	Test	Calibration Error	Accuracy
webofmike	60 tool-call risk cases	0.0712 (latest) / 0.0505 (preview)	91.7%
archerhume.com	MMLU 1,200 questions	0.0313 ECE	84.6% (MMLU-Pro)

5–7 percentage points of calibration error is genuinely better than a vanilla LLM. The direction is right.

One caveat to state plainly: TypeSafe hasn't published the RLCD training methodology. The calibration results are independently measured, but the mechanism isn't verifiable. If you're using Jev's probabilities for position sizing or risk management, that uncertainty needs to be in your model.

The BTC/USD Backtest: Ten Representations, Zero Edge

egrm07/jev_bitcoin_backtest is methodologically careful. Five validation gates:

Permutation testing
Multiple comparison correction
Holdout validation (out-of-sample)
Sharpe ratio confidence intervals
Buy-and-hold comparison

Dataset: Binance BTCUSDT 5-minute bars. Development window: March–July 2026. Out-of-sample holdout: July 15–September 19, 2026 (~65 days).

Ten data representations tested: raw OHLCV, percentage returns, technical indicators, text descriptions, ASCII charts, and combinations.

Results:

Metric	Value
Holdout AUC (all 10 representations)	0.471–0.503
Best strategy return	-15.73%
BTC buy-and-hold (same period)	+25.55%
Total model API cost	$2.58
Statistical significance	None found

Random guessing has AUC 0.5. The range 0.471–0.503 straddles that floor.

The NQ Order Book: High Accuracy, Still Underwater

Waxmell114514/jev-trade used synthetic NQ order book data replayed against real Kraken data. 339 decisions, 67.8% directional accuracy.

Metric	Value
Total decisions	339
Hit rate	67.8%
Gross P&L	+28.90
Fees	-91.60
Net P&L	-62.69
Break-even fee threshold	< 0.316 bps

The break-even threshold — below 0.316 basis points — is lower than what any realistic trading venue offers at this decision frequency. High directional accuracy still loses because transaction costs are a fixed drag that doesn't care how right you are.

The structural problem: forcing a trade on every Jev decision turns a calibrated probability model into a randomized execution engine. Zerve.ai's 2026 quant research report states it directly:

LLMs don't generate alpha. They don't surface research directions that produce real signal.

A recent arXiv paper (2608.20304) goes further: after statistical calibration, LLM feature contributions collapse to zero. A near-zero-cost baseline (headline count) outperformed every LLM feature tested. The paper proposes a calibration viability checkpoint — verify that the LLM feature has real predictive power before building the inference pipeline.

Jev outputs a probability, not a strategy. The signal extraction and position sizing layer is still on you.

The Real Bottleneck: Market Data Infrastructure

Where does Jev belong in a quant system?

Layer	Good fit	Bad fit
Information processing	News sentiment, earnings tone, regime classification, candidate screening	—
Signal execution	—	Per-tick direction calls → orders

But there's a deeper problem the backtests don't surface.

Jev accepts text. Quant systems run on structured data: prices, volumes, session states, order book depth. Jev has no market data interface. It doesn't know whether the open price field is populated during pre-market hours. It doesn't know that US extended-hours data carries different field structures than regular session. It doesn't know that a data snapshot from 09:28 ET has different semantics than one from 09:35 ET.

If your pipeline feeds Jev a raw market snapshot without explicitly encoding session state, the probability you get back has no attributable data context.

The trade_session field: existence vs. value

Here's a concrete example. The US market trading sessions endpoint returns structurally different objects depending on which session is active:

import requests

resp = requests.get(
    "https://api.tickdb.ai/v1/market/trading-sessions",
    params={"market": "US"},
    headers={"X-API-Key": "YOUR_KEY"}
)

# Response:
# {
#   "market": "US",
#   "trading_sessions": [
#     {"begin_time": 400,  "end_time": 930,  "trade_session": 1},   # pre-market
#     {"begin_time": 930,  "end_time": 1600},                        # regular — no trade_session field
#     {"begin_time": 1600, "end_time": 2000, "trade_session": 2}    # after-hours
#   ]
# }

The regular trading session (09:30–16:00) has no trade_session field. Pre-market carries trade_session: 1. After-hours carries trade_session: 2.

Session	trade_session field	Correct detection method
Pre-market 04:00–09:30	= 1	field exists AND value is 1
Regular 09:30–16:00	absent	field does NOT exist
After-hours 16:00–20:00	= 2	field exists AND value is 2

If you write if data.get("trade_session") == 0 to detect regular hours, your code raises a KeyError during regular session because the field isn't there. The correct check is field existence, not field value:

session_info = trading_sessions_response["trading_sessions"]

def get_current_session(current_time_hhmm: int, sessions: list) -> str:
    for s in sessions:
        if s["begin_time"] <= current_time_hhmm < s["end_time"]:
            # Detect by field existence, not value
            if "trade_session" not in s:
                return "regular"
            elif s["trade_session"] == 1:
                return "pre_market"
            elif s["trade_session"] == 2:
                return "after_hours"
    return "closed"

This isn't a quirk of one API. Every market has session-specific field structures. Hong Kong equities have a 60-minute lunch break with a data gap at noon. Futures roll dates change contract liquidity structure overnight. If your AI decision layer doesn't know which session the snapshot came from, it's reasoning from incomplete context.

Building the audit trail with timestamps

resp = requests.get(
    "https://api.tickdb.ai/v1/market/kline",
    params={"symbol": "AAPL", "interval": "1d", "limit": 3},
    headers={"X-API-Key": "YOUR_KEY"}
)

# Each bar includes:
# {
#   "time": 1789531200000,   # Unix milliseconds — this is your audit trail
#   "open": "332.53",
#   "high": "335.48",
#   "low": "330.70",
#   "close": "332.41",
#   "volume": "35981000"
# }

Each bar carries a Unix millisecond timestamp. When Jev returns a 72% probability on a direction call, there's a question you need to be able to answer: which bar, which session state, which fields were populated? If you can't trace the probability back to a specific, complete data state, the decision isn't auditable — and any retrospective backtest you run is disconnected from what actually happened.

This is not a Jev problem. It's a data infrastructure problem that lives upstream of any AI inference layer.

The Speed Numbers in Context

TypeSafe's headline: 193.6x faster, 444.6x cheaper than GPT-5.6 Terra.

Source	Baseline	Speed	Cost
TypeSafe (self-reported)	GPT-5.6 Terra	193.6x	444.6x
PearPages (independent)	Equivalent intelligence baseline	~25x	~76x
Near Here (independent)	Mistral Small 4	~5x	~8.6x
webofmike (measured p50)	—	421.6ms	—

5–25x faster depending on the comparison. TypeSafe's 193x is against the slowest, most expensive possible baseline. The measured p50 of 421ms is the operational number for planning purposes.

TypeSafe's stated end-to-end latency is 70–500ms; 421ms sits in the upper half of that range. For the information processing use case — batch news classification, earnings tone scoring, universe screening — this matters. For the execution layer, speed is irrelevant because the edge doesn't exist there.

If Jev Is in Your Stack: Three Things to Check

1. Where in the system? Information processing → reasonable. Per-tick execution with a forced trade on every decision → expect negative returns regardless of hit rate.

2. Is your data layer session-aware? Before feeding a market snapshot to Jev, encode session state explicitly in the text input: "09:42 ET, regular session, fields present: OHLCV, no pre/post-market quotes". Don't leave the model to infer what data was available.

3. Does your decision log have timestamps? Every Jev call on market data should log: which bar, which session, which fields were active. Without this, the probability output is not auditable, and any backtest is answering a different question than what happened in production.

The calibrated probability is real. The trading strategy still has to be built on top of it — with the full data infrastructure stack that entails.

Sources: TypeSafe Jev documentation; archerhume.com architecture reverse-engineering; APUS open-source replication report; webofmike 60-case tool-call risk benchmark; PearPages speed/cost analysis; Near Here 50-decision content moderation benchmark; GitHub egrm07/jev_bitcoin_backtest; GitHub Waxmell114514/jev-trade; Zerve.ai LLMs in Quant Research (2026); arXiv 2608.20304 LLM Calibration-Induced Degeneracy in Financial Forecasting; arXiv 2501.19047 Understanding Model Calibration; TickDB API (tested 2026-09-21). Free market data API access: tickdb.ai

How to Connect Claude to Real-Time Market Data with MCP

Walter — Tue, 15 Sep 2026 03:23:53 +0000

Meta description: Configure TickDB MCP to give Claude access to live market data with millisecond timestamps. Step-by-step setup for Claude Desktop, real JSON output, and a timestamp verification method so you know exactly when the data is from.

Claude gives you a price. How old is it?

Ask Claude "What's Apple trading at right now?" It answers. The number sounds plausible.

But there's no timestamp on that number. You don't know if it's from today's open, last month's earnings call, or a training dataset from two years ago. If you feed an unverified price into a research workflow, you're building on an unknown foundation — and you won't know it until the strategy breaks.

This isn't a hallucination in the traditional sense. Claude's training has a cutoff date. Without a live data connection, it reconstructs prices from memory, and memory doesn't carry timestamps.

The fix is Model Context Protocol (MCP): configure a real-time data source, and Claude calls a live API instead. Every response includes a timestamp. You can verify it.

What MCP changes

MCP is an open standard for connecting AI tools to external services. Once an MCP server is registered, Claude can call it during a conversation — the same way it might call a calculator or a code interpreter.

The difference in practice:

Without MCP → "Apple is trading around $185." (No source. No timestamp. Unknown date.)
With TickDB MCP → "last_price": "332.27", "timestamp": 1789156801000 (API-sourced. Timestamp verifiable.)

The price is a fact. The timestamp is proof of when that fact was observed.

Setting up TickDB MCP

TickDB provides an MCP server with 13 tools (as of 2026-08-04), covering real-time snapshots, historical candlestick data, order book depth, capital flow, intraday data, and market metrics.

Step 1: Get your API key

Step 2: Register the MCP server in Claude Desktop

Edit the Claude Desktop config file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json

Add the following under mcpServers:

{
  "mcpServers": {
    "tickdb": {
      "command": "uvx",
      "args": ["tickdb-mcp@latest"],
      "env": {
        "TICKDB_API_KEY": "your_api_key_here"
      }
    }
  }
}

Verify the exact command against the current docs at tickdb.ai/docs/mcp.

Step 3: Fully restart Claude Desktop

Quit from the system tray (not just close the window) and relaunch. The TickDB tools should appear in the tool panel.

Verifying the connection: timestamps are the test

Once configured, ask Claude: "Get me a real-time snapshot of AAPL and 000001.SZ."

Claude calls get_ticker. The raw response:

{
  "code": 0,
  "message": "success",
  "data": [
    {
      "symbol": "AAPL",
      "name": "Apple Inc.",
      "last_price": "332.27",
      "open": "327.45",
      "high_24h": "336.22",
      "low_24h": "326.3",
      "volume_24h": "50716865",
      "price_change_percent_24h": "1.75",
      "timestamp": 1789156801000
    },
    {
      "symbol": "000001.SZ",
      "name": "Ping An Bank",
      "last_price": "11.8",
      "open": "11.73",
      "high_24h": "11.9",
      "low_24h": "11.72",
      "price_change_percent_24h": "0.51",
      "timestamp": 1789353765000
    }
  ]
}

Tested 2026-09-14, Claude Sonnet 4.6 via TickDB MCP. Output is from the actual API call — not constructed manually.

Now convert the timestamps to confirm they're valid:

from datetime import datetime, timezone
import pytz

def verify_ts(ts_ms: int, label: str, tz: str = "US/Eastern"):
    dt = datetime.fromtimestamp(ts_ms / 1000, tz=timezone.utc)\
                 .astimezone(pytz.timezone(tz))
    print(f"{label}: {dt.strftime('%Y-%m-%d %H:%M:%S %Z')}")

verify_ts(1789156801000, "AAPL")
# → 2026-09-11 18:00:01 EDT  (Friday after-hours — last US trading day before the weekend)

verify_ts(1789353765000, "000001.SZ", tz="Asia/Shanghai")
# → 2026-09-14 10:43:05 CST  (Monday morning, China A-share session)

The timestamps reveal something the numbers alone don't: AAPL's quote is from Friday's after-hours session because US markets don't trade on weekends. The 000001.SZ quote is from Monday morning because China's A-share market opened today. Without timestamps, you'd have no way to know this distinction.

Three-point acceptance check:

timestamp field is present and is an integer
Converted time falls within a plausible trading session for that market
Field structure matches TickDB documentation (symbol, last_price, timestamp all present)

All three pass → the data is API-sourced, not reconstructed from model memory.

Common issues

Tools don't appear after restart

Most likely cause: JSON syntax error in the config file (JSON disallows comments and trailing commas). Also check that uvx is installed — run uvx --version; if it's missing, run pip install uv. Always quit from the system tray, not just close the window.

Authentication errors

Confirm that TICKDB_API_KEY is set inside the env block, with no extra whitespace or wrapping quotes around the key value itself.

Empty results or wrong symbol

US stocks use the plain ticker (AAPL). China A-shares use 000001.SZ (Shenzhen) or 600036.SH (Shanghai). Hong Kong stocks use 00700.HK. If you're unsure of the format, call get_available_symbols first to look up the correct symbol code.

What else you can do after get_ticker

Once get_ticker works, the other 10 Tier A tools follow the same pattern — describe what you need in plain language, and Claude selects the right tool:

Tool	Example request
`get_kline_latest`	"What's AAPL's current daily candle?"
`get_kline`	"Give me AAPL's last 5 daily bars with volume"
`get_capital_flow`	"Net capital flow for 000001.SZ today"
`get_order_book`	"AAPL order book depth right now"
`get_intraday`	"000001.SZ minute-by-minute data for today"
`get_market_metrics`	"Current A-share market breadth indicators"
`get_trading_sessions`	"What are the US market sessions today?"

Each tool's response includes a time field (timestamp or time). The verification logic is identical to the one above.

The underlying point

Configuring TickDB MCP doesn't solve "can Claude talk about markets." It solves "can you trust what Claude says about markets."

A response with a verifiable timestamp and a documented field structure is something you can act on. A response without a timestamp is a number without provenance — it might be right, but you have no way to confirm it.

TickDB's MCP tools give Claude access to structured market data: symbol, field, and time. That ordering matters. The time is what lets you decide whether the fact is current enough to use.

FAQ

How do I connect MCP to real-time financial data?

Register an MCP server in Claude Desktop's config file, specifying the server command and your API key. After a full restart, Claude can call live market data tools during conversations and return structured JSON responses, each with a timestamp.

How can an AI agent retrieve verifiable market data?

The key is the timestamp field. Every response from TickDB's MCP tools includes a millisecond-precision Unix timestamp. Convert it to local time and cross-check it against the market's trading hours to confirm the data is recent and API-sourced — not reconstructed from training memory.

Does Claude support real-time stock data by default?

No. Without an MCP configuration, Claude has no connection to live data and draws on its training knowledge, which has a cutoff date. Registering an MCP server like TickDB gives Claude access to real-time data with verifiable timestamps.

Does this work with Cursor?

Yes. Cursor supports MCP servers with equivalent configuration. The tool calls and JSON responses are identical to what Claude Desktop produces — only the config file location differs. Refer to Cursor's MCP documentation for the exact setup path.