One of the five parts of our stock score is news tone: how positive or negative the recent headlines about a company are. We used FinBERT, a BERT model fine-tuned on financial text, to read them.
How it works
- Collect headlines per company.
- Run each one through FinBERT, which gives probabilities for positive, negative and neutral.
- Turn that into one number from −1 to +1 (positive minus negative).
- Average the recent headlines per stock, then rank against the other stocks.
from transformers import pipeline
clf = pipeline("text-classification", model="ProsusAI/finbert", top_k=None)
def tone(headline):
p = {d["label"]: d["score"] for d in clf(headline)[0]}
return p["positive"] - p["negative"]
The timing detail that matters
A headline published at 5 pm New York time, after the market closed, couldn't have affected that day's trading. So headlines after the US close count for the next trading day. Getting this wrong is a quiet form of look-ahead bias, and it flatters news-based models a lot.
The result
Positive news tone did not help predict which stocks did better over the following month, in our tests.
The likely reason is simple: by the time something is a headline, the price has usually already moved. Markets react in minutes; our horizon is a month.
So why keep it?
Because the score's job is to describe a stock, not predict it. Knowing that the news about a company has been unusually negative this week is useful context. It's just not a forecast, and the site says so.
Takeaway
If you build a news-sentiment model, check two things first: the timestamp rule (before or after the close?) and the horizon. Tone that's already in the price can't tell you much about next month.
From Quantooo Lab, the makers of quantooo.com, a free, open and educational quant lab: every formula, every test and every failure is public. See it here: https://quantooo.com/ticker/AAPL
Educational, not investment advice.
Top comments (1)
Accounting for post-close timestamps is the exact discipline that separates real backtests from paper alpha. The horizon mismatch matches what shows up across market microstructure, where directional drift from public news decays in minutes because market makers adjust quotes before retail flow arrives. The other friction is that FinBERT scores unconditional tone rather than the surprise relative to what was already priced in. An earnings headline can read overwhelmingly positive, but if options implied a four-percent move and the print only justified two, the equity sells off immediately. In that sense, keeping the score as a descriptive metric rather than a predictive signal is the honest architecture.