I Fine-Tuned a 7B Model on 549 A-Share Signal Samples. The Score Didn't Move — the Errors Did.

#ai #python #opensource #stocks

I fine-tuned a 7B model to classify Chinese A-share signals. Setup cost me four minutes of GPU time and 549 training examples. The output looked right. Then I ran the same eval twice with 4x the LoRA rank and 2.7x the epochs — and the score landed in the same place.

This post is the honest version of that experiment: what the data actually looked like, which knobs did nothing, and where the bottleneck really was (spoiler: it was never the model).

The data: 549 samples, 12 stocks, 5 classes

I collect A-share snapshots every trading day at 15:10 with a cron job. No paid API — Tencent's quote endpoint returns GBK-encoded text and needs no key:

import requests

def quote(code):            # code like "sh600519"
    r = requests.get(f"https://qt.gtimg.cn/q={code}", timeout=10)
    f = r.content.decode("gbk").split("~")
    return {
        "price": float(f[3]),
        "prev_close": float(f[4]),
        "open": float(f[5]),
        "high": float(f[33]),
        "low": float(f[34]),
        "volume": float(f[6]),
    }

Each snapshot becomes a chat-style training row (instruction / input / output), where output is a structured verdict: signal label, reasoning, action, risk note. After a few months of collection the file sits at 549 rows over 12 tickers, and the label distribution is already a warning sign:

175  缩量回调寻底      (low-volume pullback)
155  趋势上涨通道      (trend following)
146  横盘震荡          (sideways consolidation)
 43  关键放量突破      (breakout with volume)
 30  高位放量崩塌      (distribution day)

The gap between the most and least frequent class is 5.8:1. The rarest class — the one that tells you to get out — is also the one that matters most. That ratio, not the model size, turned out to be the whole story.

The training run

Base model: Qwen2.5-7B-Instruct, QLoRA 4-bit, Unsloth, on a single RTX 3080. Rank 16, 3 epochs = 78 steps ≈ 4 minutes. That's the part everyone tweets about, and it works exactly as advertised.

Three things broke, and all three are boring infrastructure bugs worth knowing before you burn an evening:

trl version matters more than it should. trl==0.20.0 worked; 0.23/0.24 threw an entropy-related training bug. Pin it.
Chat templates with custom tokens. If your dataset uses <EOS_TOKEN> / <PAD_TOKEN> placeholders, they must be registered in the tokenizer vocabulary first, otherwise loss silently ignores them.
Environment isolation. My agent's Python environment had a different NumPy build than the training venv. Training in the wrong interpreter gave ABI crashes that look like CUDA problems. Use a dedicated venv, not the one your agent runs in.

The result that mattered

First run (rank 16, 3 epochs): the model produced fluent Chinese analysis but didn't reliably follow the output format of the training rows. Classic under-training. So I went to 8 epochs — about 11 minutes — and the format aligned. Good.

Then I did what I should have done first: I held the eval set fixed and changed one thing at a time.

rank 16 → rank 64 (4x the trainable parameters)
3 epochs → 8 epochs

The aggregate score did not move beyond run-to-run noise. What moved was which examples were wrong. The same structural failure persisted in both runs: the 30-sample distribution day class kept collapsing into the far more common pullback class, because a 5.8:1 prior is a stronger signal than the few distinguishing examples. Extra capacity just reshuffled which individual rows got hit.

That's the useful takeaway, and it's not specific to stocks:

When your dataset is coverage-bound, adding capacity only reorders your errors. It doesn't create knowledge that isn't in the examples.

A 7B model with rank-64 adapters still only knows what the 549 rows taught it. In this task, the rows also had a deeper problem: the input is a single day of price and volume, but the label depends on market regime — what the index did that week, whether the sector was rotating, what the volume looked like relative to the last 20 sessions. The model couldn't see the variable that determines the answer. No amount of LoRA fixes a missing feature.

What I'd do differently (in order)

Fix the label distribution before touching hyperparameters. Oversample or synthesise the scarce classes until the ratio is under 2:1. My rare class has 30 examples; that's not a class, it's an anecdote.
Give the model the regime, not just the snapshot. Attach 20-day rolling context (index return, sector breadth, relative volume percentile) to each row.
Split by time, never randomly. Adjacent days of the same ticker are near-duplicates. A random split leaks the answer into your eval set and inflates the score.
Freeze the eval set before the first run. Otherwise "I improved it" means "I picked the checkpoint that scored best on the set I kept looking at."
Route low-confidence output to a re-check instead of trusting it. A model that is right 80% of the time and knows when it's unsure is worth more than one that is confidently wrong on the 30 examples that matter.

The part that actually shipped

The fine-tune is the least interesting artifact from this project. The reusable pieces are:

a free, key-less data pipeline (Tencent quotes + a daily collector) that produces a clean snapshot file,
a regime-split backtest so that "this signal works" is a claim with a date range attached, and
the failure log above, which is what stopped me from shipping a model that was fluent and wrong.

I keep all three in the open here — including the negative results:

→ github.com/Felixwang007/a-share-signal-lab — honest signal replication on real A-share data: 15,900 stock-days, regime-split backtests, and the free-data Python toolkit used above. No API keys, no paid feeds, runnable on a laptop.

If you'd rather have the packaged version as an agent skill (the indicator library + the screening workflow, ready to drop into Claude Code / Cursor / any MCP client), it's also on the skill marketplaces — 虾评 and Agensi — alongside the GitHub Trending aggregator I wrote about earlier.

If you're fine-tuning anything on a few hundred rows: check your class balance and your train/test split before you check your rank. In my case that was worth more than the 4x parameter bump — and it took 4 minutes to test instead of 4 hours.

Comments welcome — especially if you've hit the same coverage ceiling with domain-specific classifiers. I'd like to know what made the difference for you: synthetic data, better features, or a smaller, cleaner label set.

DEV Community