The five highest-scoring submissions in the six-day comparison all received the same topic: Unknown.
That is an awkward opening for a topic-modeling article. It is also the most useful result. Among 5,997 Hacker News submissions dated October 24–29, 2026, a small model assigned 4,432 to no learned topic. Those unknown titles carried 81.55% of the observed points. A chart showing only the named groups would have hidden most of the conversation—and most of its measured attention.
The question was straightforward: which subjects appear on HN, which attract attention, and how does the picture differ across sources and periods? Answering it required more than finding words such as “AI.” It required keeping the model's vocabulary, the source population and the popularity calculation separate.
This case uses real submissions, a small trained model, and independently callable steps. The examples below run through Splime Public Hub and produce an HTML report with topic summaries, tags, collection details and a linked reading list.
Start with the population.
The collector retrieved 13,282 unique stories created from October 17 at 00:00 UTC to October 30 at 00:00 UTC, with the end excluded. Collection ran on October 31 from 11:01:59 to 11:03:03 UTC. Training uses the earlier seven days; the later six days supply the descriptive comparisons and temporal evaluation.
For historical dates, the implementation queries HN Search, splitting busy time intervals until each query fits within one returned page. This run needed 83 requests and exhausted the requested intervals without reaching the item limit. That establishes retrieval coverage of the search index, not a guarantee that every HN submission is indexed. The returned snapshot records its bounds, limits and collection timestamps.
Current Firebase feeds answer a different question. The official API supplies bounded lists of current top, new, Ask and Show stories; filtering those lists by date does not recreate a past front page. The collector exposes both routes and describes their coverage explicitly. Network failures raise errors; they never produce substitute records. Official HN API documentation, HN Search API.
“Direction” is therefore not a field in the model. A topic is a learned word group. A source is a feed or index membership. In the historical data, story includes ask_hn and show_hn; these sources overlap. Each HN ID counts once overall, while its memberships remain available for comparisons.
Learn a small map of words.
An earlier baseline, analyze_titles@1, applied supplied phrase rules. It was explainable, but it had no training step. The new model starts with titles and learns its own word groups.
First, fixed text cleanup removes common function words and source prefixes such as “Show HN.” TF-IDF represents each title using words and adjacent word pairs. A term gets more weight when it is informative within the training collection. The vocabulary and document-frequency weights are fitted only on the earlier partition.
Next, nonnegative matrix factorization, or NMF, approximates those title vectors with eight reusable word groups. Think of each title as a mixture of a few word lists, with a different weight on each list. This is a standard, inspectable topic-extraction approach; the scikit-learn example illustrates the same TF-IDF/NMF pairing.
The baseline caps the vocabulary at 600 terms. Eight topics and that small vocabulary are demonstration choices, not a tuned optimum. From 7,285 earlier submissions, training removed 122 titles with too little usable text and 255 repeated normalized titles. The remaining 6,908 titles supplied the learned state; 6,175 had a nonzero vector in the selected vocabulary. NMF converged after 37 iterations.
The topic names are the four highest-weight learned terms, printed without editorial polishing. This matters: open / source / open source / alternative is a useful clue, while video / php / world / context is a diffuse group. Singular agent and plural agents became separate components. The model has learned associations in this corpus, not a comprehensive taxonomy of technology.
For a new title, the vocabulary, IDF and components stay fixed. A nonnegative least-squares fit estimates the title's mixture of those components. The strongest component supplies a primary topic when the rejection rules allow it. Very short titles, too few known words, weak matches or ambiguous mixtures become Unknown. Those thresholds are heuristics, not calibrated confidence scores.
Tags are even more literal: they are high-weight terms from the assigned topic that occur in the cleaned title; word pairs are adjacent after stop-word removal. A title may receive agent and skills; it does not inherit every word in its topic's name. Nothing in training or inference uses points, comment counts, source membership or future activity as a feature.
The map covers a recognizable corner of HN.
Here is the later six-day population under that single frozen model:
| Automatic topic name | Stories | Share of all stories | Observed points |
|---|---|---|---|
| Unknown | 4,432 | 73.90% | 95,396 |
| ai / ai agents / ai agent / slop | 558 | 9.30% | 9,423 |
| agents / ai agents / coding / coding agents | 185 | 3.08% | 1,437 |
| code / claude / claude code / codex | 170 | 2.83% | 3,085 |
| open / source / open source / alternative | 155 | 2.58% | 1,782 |
| agent / ai agent / coding / coding agent | 149 | 2.48% | 832 |
| video / php / world / context | 146 | 2.43% | 1,541 |
| browser / data / game / pdf | 133 | 2.22% | 1,721 |
| model / language / world / first | 69 | 1.15% | 1,758 |
The named groups make the corpus look dominated by AI and developer tools. Yet their combined coverage is only 26.10%. For example, the HN submission titled “Anthropic S-1” had 1,708 observed points and remained unknown. So did “We're releasing the weights of Mistral Large 4”, with 1,694 points. The topic map misses even some conspicuous AI-related titles.
The five highest-point records were checked against the official Firebase API: their IDs, titles, creation times and observed scores matched. This verifies the submissions and metadata, not the truth of claims made in their headlines. Counts, shares and engagement totals were also independently recomputed from the collected records.
A rise in submissions is a different result from a rise in attention.
The comparison uses two equal 72-hour windows: October 24–26, with 2,532 stories, and October 27–29, with 3,465. The broad ai / ai agents / ai agent / slop group moved from 223 to 335 stories, or 8.81% to 9.67% of each population: +0.86 percentage points. The singular-agent group rose by 0.68 points. Unknown stayed near three quarters of both windows.
These are changes in the composition of observed submissions. They are not evidence that a topic is gaining users, that sentiment improved, or that its next post will become popular. Both periods use the same model fingerprint. Comparing “topic 3” from two independently trained models would make the numbers look comparable while changing their meaning.
Points and comments provide another view. The broad AI group accumulated 9,423 points and 6,835 comments. Its highly discussed examples include “AI promised less work. I've never worked more”, with 684 points and 662 comments, and “No signs of AI payoff outside the tech sector”, with 653 points and 678 comments. These titles suggest debate about AI's practical consequences; comment counts alone cannot establish approval or sentiment.
Every topic's median story score was just two. Totals are sensitive to a few exceptional posts. Counts measure publishing volume; shares account for different submission totals; points summarize observed net scoring; comments measure discussion. Missing engagement remains missing and the report shows how many values were observed.
Age is another constraint. At collection, the median story was 136.5 hours old in the earlier window and 67.6 hours old in the later one. The older group had more time to accumulate activity. These points and comments were collected after the posting periods, so they are not historical “as of midnight” totals. The case describes accumulated attention and does not claim an age-adjusted trend or a popularity forecast.
Ask and Show reveal different coverage.
Among the later stories, 179 carried ask_hn and 858 carried show_hn. Unknown accounted for 70.39% of Ask HN, 59.32% of Show HN and 73.90% overall. The open/source group represented 7.11% of Show HN, compared with 2.58% overall; the plural-agents group also represented 7.11% of Show HN.
This is a useful source contrast, although it partly reflects vocabulary coverage: Show HN titles more often match these learned groups. It is not proof that the unassigned stories lack themes. The overall column already includes Ask and Show, so adding their counts to it would count some submissions twice.
Inspect errors before celebrating coverage.
The temporal evaluation held out submissions created on or after October 24. It conservatively removed 263 later titles whose normalized text or canonical external URL appeared in training, leaving 5,734 evaluation records. That purge applies to evaluation; descriptive counts retain distinct later HN submissions, even when they discuss the same external article.
On that purged holdout, the model assigned 25.99% and rejected 74.01%. Mean known-token coverage was 37.38%. These figures describe the model's reach, not its semantic accuracy. No independent human topic labels were available, and no rule-generated labels were used to manufacture an accuracy score.
A qualitative review examines deliberately difficult examples, selected by large reconstruction residuals. It is an assistant inspection of titles, not human ground truth or a representative accuracy study. Several errors are easy to understand:
- “Open a local LLM as a 3D graph…” landed in the open/source group because “open” can be a verb.
- An Apple iOS-support title ending in “[video]” entered the video group because a format marker acted like a subject.
- HSBC's data-center debt title matched the browser/data/game/pdf group through “data,” although the title concerns finance.
These are specific reasons to improve the representation. A larger vocabulary, phrase handling, longer text or a better tokenizer may help. The next evaluation should use independent human review of future titles. Raising the assignment rate by weakening rejection would not, on its own, demonstrate improvement.
Repeat the computation one step at a time.
Use Python 3.13 and install these tested package versions in the environment used by your notebook:
python -m pip install splime==0.4.13 httpx==0.28.1 numpy==2.3.5 \
scipy==1.18.0 scikit-learn==1.8.0 ipython==9.18.0
An SPH consumer needs no registration or API key. In embedded mode the published code is downloaded and runs locally; trust=True explicitly permits its execution.
The following version table is generated only from confirmed publication receipts:
| Public object | Confirmed immutable version |
|---|---|
| fetch_snapshot | fetch_snapshot@2 |
| normalize_stories | normalize_stories@2 |
| train_topic_model | train_topic_model@1 |
| predict_topics | predict_topics@1 |
| evaluate_topics_temporally | evaluate_topics_temporally@2 |
| summarize_topics | summarize_topics@2 |
| hn_topic_model | hn_topic_model@1 |
| tag_titles | tag_titles@1 |
| render_digest | render_digest@3 |
| hn_topic_digest | hn_topic_digest@4 |
In a notebook, set up the client and exact references:
from spl import SPLClient
from IPython.display import HTML, display
client = SPLClient.embedded(registry_url="https://splime.io", run_receipts=False)
base = "splime://@splime.project/case4-hn"
Fetch and normalize independently. Collection limits are visible rather than hidden in a utility:
snapshot = client.call(
f"{base}/fetch_snapshot@2",
kwargs=dict(source="algolia", feeds=["story"],
start_utc="2026-10-17T00:00:00Z", end_utc="2026-10-30T00:00:00Z",
max_items=20000, page_size=500, max_requests=200,
timeout=15, attempts=3, budget_seconds=300), trust=True,
).value
assert not snapshot["source_info"]["truncated"], "Increase limits or narrow the period"
snapshot = client.call(
f"{base}/normalize_stories@2",
kwargs=dict(records=snapshot["records"], source_info=snapshot["source_info"]),
trust=True,
).value
Training remains an explicit operation. Its result is a JSON-compatible model containing the learned vocabulary, weights and automatic labels:
earlier = dict(snapshot, records=[r for r in snapshot["records"]
if r["created_at_utc"] < "2026-10-24T00:00:00+00:00"])
model = client.call(
f"{base}/train_topic_model@1",
kwargs=dict(snapshot=earlier, n_topics=8, max_features=600, random_state=42),
trust=True,
).value
The published predictor includes one frozen model as a required native scalar dependency. Its learned state is serialized as JSON text inside the signed release, and a restore step checks the model fingerprint before prediction. The bundle carries the model itself; it does not retrieve an author's file or retrain when called.
The consumer uses tag_titles with only a snapshot; its bound model is mandatory. The model artifact can also be fetched independently through hn_topic_model, while predict_topics accepts an explicit model for experiments. Consumers of the fixed pipeline never search for a local model file, pass one manually or retrain. Repeated live queries can change as the index and scores change. Every comparison here uses the same frozen model, while the reported totals refer to the collection time stated above.
later = dict(snapshot, records=[r for r in snapshot["records"]
if r["created_at_utc"] >= "2026-10-24T00:00:00+00:00"])
predictions = client.call(
f"{base}/tag_titles@1",
kwargs={"snapshot": later}, trust=True,
).value["result"]["default"]
analysis = client.call(
f"{base}/summarize_topics@2",
kwargs={"predictions": predictions, "window_hours": 72,
"anchor_utc": "2026-10-30T00:00:00Z"}, trust=True,
).value
html = client.call(
f"{base}/render_digest@3",
kwargs={"analysis": analysis, "limit": 12}, trust=True,
).value
display(HTML(html))
The HTML is an escaped fragment with container-scoped CSS, so displaying it does not restyle the notebook. Use full_document=True only when saving a standalone page.
Finally, the ready pipeline does collection, preparation, fixed-model inference, popularity calculation and rendering in one call. Training is absent:
html = client.call(
f"{base}/hn_topic_digest@4",
kwargs=dict(source="firebase", feeds=["newstories", "topstories"],
max_items=15, max_requests=50, timeout=15,
attempts=2, budget_seconds=180, limit=10),
trust=True,
).value["result"]["default"]
display(HTML(html))
All ten releases passed an anonymous run in a clean Python 3.13 environment with a fresh cache on 1,210 real HN stories. Its 15 checks exercised the separate stages, the complete one-call pipeline, fixed-model identity and empty-input behavior. The companion notebook also completed all 17 code cells against a fresh 13,282-story historical download and a current-feed pipeline call.
The experiment leaves a practical reading tool and an equally practical warning about its interpretation. The learned groups reveal recurring vocabulary, source differences and modest changes in submission share. Keeping unknowns visible shows exactly where that map stops helping. A useful report makes those boundaries as easy to inspect as its most appealing chart.
Top comments (0)