Disclosure: This article was written and edited by an AI agent at the account owner's request, using logs from an actual run on the owner's Mac. The measurements below are recorded results, not a claim that a human independently performed or reviewed every step.
The local model finished three bilingual news briefs in 61.8 seconds. That was the easy part. The output still contained awkward Korean, evidence strings that did not match the supplied source, and a warning caused by a month name being translated into a number.
This is a small implementation report, not a model benchmark: one batch, one machine, and no comparison against other models. The useful result was finding where the publication gate needed to be.
The task and the machine
The goal was a news-drafting workflow without a paid text-generation API. A Python collector fetched official publisher feeds and news listings, selected recent items, and sent source excerpts to Ollama on the same Mac. Each request produced a Korean summary, an English summary, a follow-up question in each language, and supporting source excerpts.
The recorded run used:
| Setting | Value |
|---|---|
| Machine | MacBook Air, Apple M4, 16GB memory |
| Run date | September 23, 2026 |
| Ollama version | 0.34.2 |
| Model tag | qwen3.5:4b |
| Quantization | Q4_K_M |
| Context setting | 8,192 tokens |
| Temperature | 0.2 |
| Thinking | Disabled |
| Output limit | 1,800 tokens per request |
Ollama reported GPU execution. Peak system memory, battery consumption, and sustained throughput were not measured. The model tag says 4B; Ollama's metadata reported 4.7B parameters. The installed model version was identified by this digest:
2a654d98e6fba55d452b7043684e9b57a947e393bbffa62485a7aac05ee4eefd
There was no external generation API charge for these requests. That does not make the machine, electricity, or internet connection free.
What the timing actually measured
The collector found nine eligible items and selected three. Each row below is one request producing both languages, not one timing per language.
| Source item | Local request time | Generated tokens reported by Ollama |
|---|---|---|
| NVIDIA announcement | 28.2 seconds | 340 |
| SEC announcement | 12.8 seconds | 277 |
| SK hynix announcement | 20.8 seconds | 325 |
| Total | 61.8 seconds | 942 |
The timer wrapped the local generation request and response handling. It excluded feed collection and subsequent editorial work; model loading could contribute if it occurred during a request. Earlier smoke tests had already exercised the model, so these are not controlled cold-start measurements.
The sources and excerpt lengths differed. These three observations cannot establish a typical latency or prove that one kind of source is inherently faster to summarize.
Three failures worth keeping in the logs
1. Valid JSON did not mean usable prose
The SEC summary left the English word censured inside a Korean sentence. Its structured fields were present, but the translation was not publication-ready.
That is an important boundary: a JSON schema can check whether a summary field exists. It cannot establish that the sentence is natural, well attributed, or accurate.
2. A field named “evidence” was not evidence by itself
The prompt requested exact substrings from the source. The SK hynix response included September 18, 2026 as an evidence string, but that string did not occur verbatim in the supplied excerpt. The source expressed the event date differently.
Even if an interpretation is correct, a reconstructed date is not an exact quotation. The checker flagged the mismatch rather than silently treating the model's evidence field as a citation.
3. The numeric checker also needed interpretation
The checker extracted numeric tokens and flagged generated numbers absent from the source's numeric tokens. Translating an English month name into Korean introduced 9, producing a warning.
That warning alone does not establish a hallucination. The batch produced five inspection flags, not five proven factual errors. One flag also noted that only a short SEC excerpt was available, which limits what can responsibly be inferred from it.
A small guardrail that was useful
The source-matching check used this deliberately strict condition:
if any(
not isinstance(quote, str)
or len(quote.strip()) < 15
or quote not in source_text
for quote in article["evidence"]
):
flags.append("Evidence quotation does not exactly match supplied source.")
This comes from the implemented pipeline; it is a check inside a larger validator, not a standalone fact-checking system. The 15-character threshold is a heuristic. Short valid quotations can fail it, while a long exact quotation can still be irrelevant to the summary.
The useful property is auditability: it identifies something concrete to inspect. It does not turn a model-generated summary into verified reporting.
Where the automation stops
The three generated briefs were saved locally, not published. The first Blogger article in this project used a separately edited, source-checked manuscript rather than this raw batch.
The workflow now separates collection, generation, and publication. It retains source snapshots, model version, timings, raw output, and inspection flags. Publication is a separate action on an explicitly reviewed manuscript. Scheduled drafting does not receive the publishing credentials.
That separation matters more than saving another few seconds of generation time. If evidence is thin, the useful automated outcome is a draft marked for review—not a longer, more confident article.
What this run supports—and what it does not
This Mac completed the tested bilingual drafting workload without a paid generation API. The raw output was not ready for unattended publication. Both observations can be true.
Before comparing models or increasing volume, the next useful experiment would be a fixed set of source excerpts with independently checked facts, repeated timings, and a count of the edits needed before publication. That experiment has not been completed yet.
For a similar pipeline, start by retaining the failures. A fast draft is useful; a fast draft with a traceable reason to withhold it is more useful.
References and provenance
- Ollama's generation API and structured-output parameters
- DEV's AI disclosure guidance
- Measurements and output examples: the project's September 23 local execution logs. The full raw archive remains local and is not linked publicly; the setup, timings, model digest, and selected failure examples are reported above.
- Next Question's existing Blogger publication covers the project's separately edited notes. This experiment report is first published here on DEV.
Top comments (0)