DEV Community: LinguaPilot AI

Qwen2.5 7B vs Qwen3 4B & 8B for Writing Correction: 60 Local Ollama Responses on Windows

Sami — Sat, 29 Aug 2026 11:26:17 +0000

I expected Qwen2.5 7B to retain a noticeable advantage over the smaller Qwen3 4B model for writing correction.

In this experiment, it didn't.

Across the same 20 paired writing cases, Qwen2.5 7B and Qwen3 4B produced exactly the same complete-case outcome: both succeeded on the same 18 cases and failed on the same 2.

The difference became much more noticeable in local execution time: in the tested cold-start setup, Qwen3 4B averaged 23.99 seconds, compared with 54.37 seconds for Qwen2.5 7B.

That raised another practical question:

Can a smaller model from a newer generation reproduce the practical writing-correction performance of a larger model from the previous generation?

This benchmark is a follow-up to my earlier Qwen3 4B vs 8B vs 14B writing-correction experiment.

In that first study, Qwen3 8B and 14B both achieved 19/20 complete cases, while the mean cold-start time for Qwen3 8B was roughly half that of Qwen3 14B.

This second experiment approaches the same model-selection question from another direction: instead of comparing sizes within Qwen3, it compares Qwen2.5 7B with Qwen3 4B and Qwen3 8B under the same paired writing-coach evaluation framework.

Benchmark setup

The benchmark uses 20 paired writing cases, including 16 sentences with expected errors and 4 already-correct controls.

Each case was evaluated with the same frozen reference and the same evaluation framework across three locally hosted Qwen models:

qwen2.5:7b
qwen3:4b-q4_K_M
qwen3:8b-q4_K_M

This produced 60 local responses in total.

The models were executed through Ollama on Windows under the same benchmark conditions. The objective was not to identify a universal “best model,” but to compare several dimensions separately:

complete correction of the expected errors;
expected-correction coverage;
localization of the error regions;
preservation of the original facts;
compliance with the requested output structure;
explanation-language and consistency checks;
local cold-start execution time.

For the primary metric discussed here, a case was considered complete only when all expected correction requirements for that case were satisfied without an incorrect replacement, false error, or configured fact loss.

Results

Complete-case correction

The primary correction results were close:

Qwen3 4B: 18/20 complete cases — 90%
Qwen2.5 7B: 18/20 complete cases — 90%
Qwen3 8B: 19/20 complete cases — 95%

On the primary metric alone, Qwen3 8B completed one additional case in this run.

The more distinctive result, however, was the paired comparison between Qwen2.5 7B and Qwen3 4B.

They did not merely finish with the same 18/20 score. They succeeded on the same 18 cases and failed on the same 2 cases.

Across all 20 paired cases:

both succeeded: 18/20
both failed: 2/20
different complete-case outcomes: 0/20

So, for this specific 20-case writing benchmark, moving from Qwen2.5 7B to Qwen3 4B preserved the same primary correction outcome case by case.

This should not be interpreted as evidence that the two models are universally equivalent. It means that no complete-case difference was observed between them within this controlled test set.

Local execution time

The difference became much larger when local cold-start execution time was considered.

In the tested configuration, the mean total cold-start times were:

Qwen3 4B: 23.99 s
Qwen2.5 7B: 54.37 s
Qwen3 8B: 60.24 s

Qwen3 4B therefore required less than half the mean cold-start time of Qwen2.5 7B in this run.

Expressed as a ratio, Qwen3 4B was approximately 2.27× faster than Qwen2.5 7B on this specific cold-start measurement, corresponding to about a 55.9% reduction in mean cold-start time.

This makes the paired correction result more practically interesting: Qwen3 4B preserved the same complete-case outcome as Qwen2.5 7B across all 20 cases while responding substantially faster in the tested local setup.

Qwen2.5 7B and Qwen3 8B, by contrast, were much closer in execution time. Qwen2.5 7B averaged 54.37 s versus 60.24 s for Qwen3 8B, making the 7B model about 9.7% faster on mean cold-start time in this run.

These timing results should be interpreted within the tested configuration. They describe cold-start execution time on this local Windows/Ollama setup, not a universal model-speed ranking.

Error localization and explanation checks

Complete-case correction does not capture every difference between the models.

Across the 21 expected error regions:

Qwen3 4B: 20/21 regions localized — 97.6% F1
Qwen2.5 7B: 20/21 regions localized — 97.6% F1
Qwen3 8B: 21/21 regions localized — 100% F1

Neither Qwen3 4B nor Qwen2.5 7B produced a false-positive error region in this run, while Qwen3 8B localized all 21 expected regions with no false positive.

Expected-correction coverage followed a similar pattern:

Qwen3 4B: 19/21 expected corrections — 90.5%
Qwen2.5 7B: 19/21 expected corrections — 90.5%
Qwen3 8B: 20/21 expected corrections — 95.2%

So Qwen2.5 7B and Qwen3 4B again matched on both correction coverage and error-localization F1, while Qwen3 8B retained a small advantage on these measures.

The explanation checks showed a more nuanced difference.

For explanation-language compliance:

Qwen2.5 7B: 40/40 evaluated fields passed
Qwen3 8B: 41/41 evaluated fields passed
Qwen3 4B: 37 passes and 3 warnings

All three models nevertheless produced parseable JSON, usable core output and full-contract-compliant output in all 20 cases.

The configured fact-preservation check also passed all 20 cases for every model.

Among the 16 error-containing cases, the action–explanation consistency check recorded 16 consistent results for each model, with no contradictory, ambiguous or missing entries.

These automatic checks should not be interpreted as a complete measure of pedagogical quality. They capture selected properties such as language compliance, output discipline and consistency, while deeper explanation quality would require additional linguistic or human evaluation.

What this means for local writing assistants

The main takeaway is not that one model is universally better than the others.

In this benchmark, Qwen2.5 7B and Qwen3 4B produced the same complete-case outcome on all 20 paired cases, while Qwen3 4B required substantially less cold-start time in the tested local setup.

Qwen3 8B, however, completed one additional case and reached 100% error-localization F1.

This creates a practical trade-off rather than a single winner.

For this specific workload:

Qwen3 4B offered the strongest efficiency result: the same paired complete-case outcome as Qwen2.5 7B, with much lower measured cold-start time.
Qwen2.5 7B remained a credible previous-generation baseline: it matched Qwen3 4B on complete cases, expected-correction coverage and error-localization F1, while showing cleaner explanation-language compliance in this run.
Qwen3 8B provided the strongest correction coverage: one additional complete case, one additional expected correction and full error localization.

This suggests that choosing a local writing model should be treated as a multi-criteria decision rather than as a simple parameter-count comparison.

For an interactive writing assistant, the relevant trade-off may involve correction reliability, explanation behavior, latency, model size and the available hardware.

In this run, Qwen3 8B provided a small measurable correction advantage, but that advantage came with substantially higher cold-start time than Qwen3 4B.

Limitations

This benchmark is exploratory and should be interpreted within its experimental scope.

Several limitations are important:

20 paired cases: the dataset is intentionally focused and is not large enough to support a universal ranking of the models.
One language scenario: the benchmark evaluates English writing correction with explanations requested in French. Results may differ for other languages or writing tasks.
One model family: the comparison is limited to Qwen2.5 7B, Qwen3 4B and Qwen3 8B and should not be generalized to unrelated local model families.
One local machine: execution time depends on hardware, background workload, model format, quantization and Ollama configuration.
Cold-start timing: the reported measurements describe this specific cold-start workflow rather than warm-start interaction latency.
Reference-based evaluation: outputs were compared against the same frozen, versioned correction reference and acceptance rules across all three models.
Automatic explanation checks: language compliance, output discipline and consistency were evaluated automatically, but these checks do not constitute a complete linguistic or pedagogical assessment.
No direct resource measurement: RAM, VRAM, CPU load and energy consumption were not measured in this run.

The last point is particularly important for interpreting the efficiency result.

Qwen3 4B has a substantially smaller nominal parameter count than Qwen2.5 7B, but parameter count alone should not be treated as a direct measurement of hardware-resource usage.

The benchmark therefore supports a measured latency comparison, not a claim that Qwen3 4B used a specific percentage less RAM, CPU or energy.

What I plan to test next

This second benchmark leaves several useful directions open for further evaluation.

The next iterations could extend the framework in four directions:

measure peak RAM and VRAM usage alongside correction quality;
compare cold-start and warm-start latency under the same local conditions;
extend the benchmark to a larger and more diverse writing test set;
compare additional local model families under the same evaluation protocol.

Direct resource measurements would be particularly useful.

In this run, Qwen3 4B showed a strong latency advantage over Qwen2.5 7B while preserving the same paired complete-case outcome, but memory and compute usage were not measured directly.

Adding RAM, VRAM and processor-load measurements would make it possible to evaluate the trade-off more completely across:

correction reliability;
explanation behavior;
latency;
memory footprint;
compute load;
and model size.

I am particularly interested in whether the efficiency pattern observed here remains stable when the task becomes more linguistically demanding and when resource usage is measured directly.

Conclusion

This experiment did not identify a universal winner, but it revealed a clear practical trade-off.

For this specific 20-case writing benchmark, Qwen3 4B and Qwen2.5 7B produced the same complete-case outcome on every paired case:

both completed 18/20 cases;
both applied 19/21 expected corrections;
both reached 97.6% error-localization F1.

The difference was much larger in local execution time.

Qwen3 4B averaged 23.99 seconds of total cold-start time, compared with 54.37 seconds for Qwen2.5 7B.

In this tested configuration, that corresponds to approximately 2.27× faster cold-start execution for Qwen3 4B while preserving the same primary paired correction outcome.

Qwen3 8B remained slightly stronger on correction coverage, completing 19/20 cases and reaching 100% error-localization F1, but with a mean cold-start time of 60.24 seconds.

The more useful question, therefore, may not be:

Which model has the most parameters?

but rather:

Which model provides the best balance between correction reliability, explanation behavior, responsiveness and available local hardware for the intended task?

For local interactive applications, model size alone is not enough to answer that question.

Disclosure: AI assistance was used to refine the wording and organization of this article. The benchmark results and measurements reported here come from the documented experimental runs and were reviewed for accuracy before publication.

Qwen3 4B vs 8B vs 14B for Writing Correction: 60 Local Ollama Responses on Windows

Sami — Fri, 07 Aug 2026 12:56:05 +0000

I expected the larger Qwen3 model to show a clear advantage for writing correction. In this experiment, it didn't.

Across 20 paired writing cases, Qwen3 8B and 14B both achieved 19/20 complete-case corrections, while Qwen3 4B reached 18/20.

The difference became much more noticeable in local execution time: in the tested cold-start setup, the mean cold-start time for Qwen3 8B was roughly half that of Qwen3 14B.

That raised a practical question:

For a local writing assistant, how much model size do we actually need?

To explore this more systematically, I built a reproducible benchmark comparing Qwen3 4B, 8B and 14B under the same local conditions with Ollama on Windows.

Benchmark setup

The benchmark uses 20 paired writing cases, including 16 sentences with expected errors and 4 already-correct controls.

Each case was evaluated with the same reference and the same evaluation framework across three locally hosted Qwen3 models:

qwen3:4b-q4_K_M
qwen3:8b-q4_K_M
qwen3:14b-q4_K_M

This produced 60 local responses in total.

The models were executed through Ollama on Windows under the same test conditions. The objective was not to identify a universal “best model,” but to compare several dimensions separately:

complete correction of the expected errors;
localization of the error regions;
preservation of the original meaning;
compliance with the requested output structure;
quality of the explanations;
local execution time.

For the primary metric discussed here, a case was considered complete only when the expected correction requirements for that case were satisfied.

Results

Complete-case correction

The primary correction results were close:

Qwen3 4B: 18/20 complete cases — 90%
Qwen3 8B: 19/20 complete cases — 95%
Qwen3 14B: 19/20 complete cases — 95%

On this metric alone, increasing the model size from 8B to 14B did not improve the number of complete cases in this test.

Local execution time

The difference was much larger in execution time.

In the tested cold-start configuration, the mean cold-start times were approximately:

Qwen3 4B: 26.6 s
Qwen3 8B: 60.0 s
Qwen3 14B: 119.4 s

So, while 8B and 14B achieved the same complete-case score, Qwen3 8B took about half as long as Qwen3 14B on average in this cold-start setup.

The 4B model was substantially faster again, while still completing 18 of the 20 cases.

This does not mean that 8B is universally better than 14B. It means that, for this specific writing-correction workload and hardware configuration, the additional model size did not translate into a better primary correction score, while the execution-time cost increased considerably.

Error localization and explanation quality

Complete-case correction and error localization measure different aspects of model behavior.

Across the 21 expected error regions:

Qwen3 4B missed one expected region, with an error-localization F1 of 97.6%.
Qwen3 8B localized all 21 expected regions, reaching 100% F1.
Qwen3 14B missed one expected region and reported one false-positive region, resulting in 95.2% F1.

This distinction matters for a writing coach: producing an acceptable corrected sentence does not necessarily mean that the model identified the underlying error with the same precision.

Explanation quality was therefore evaluated separately from correction success.

In this run, all 41 evaluated explanation fields produced by Qwen3 8B passed the language check. Qwen3 4B showed more explanation-language and terminology issues.

These automatic checks should not be interpreted as a complete measure of pedagogical quality. They cover selected language patterns and contradictions, while deeper explanation quality remains a separate area for linguistic review.

What this means for local writing assistants

The main takeaway is not that one model is universally better than another.

In this benchmark, the three Qwen3 models produced relatively similar complete-case correction results, but their local execution times differed much more substantially.

For this specific workload:

4B offered the lowest execution time, with a small reduction in correction coverage.
8B matched 14B on complete-case correction and achieved the strongest error-localization result in this run.
14B did not improve the primary correction score over 8B, while requiring substantially more execution time in the tested configuration.

This suggests that model selection for a local writing assistant should be treated as a multi-criteria decision, rather than assuming that a larger model will automatically provide the best practical experience.

Depending on the use case and available hardware, the relevant trade-off may involve:

correction reliability;
explanation quality;
latency;
memory and compute requirements;
and the expected frequency of use.

For a writing coach in particular, responsiveness also matters. A model that is slightly larger or theoretically more capable may not necessarily provide the best user experience if each interaction becomes significantly slower.

Limitations

This benchmark is exploratory and should be interpreted within its experimental scope.

Several limitations are important:

20 paired cases: the dataset is intentionally focused and is not large enough to support a universal ranking of the models.
One language scenario: the benchmark evaluates English writing correction with explanations requested in French. Results may differ for other languages or writing tasks.
One model family: only Qwen3 4B, 8B and 14B were compared. The results should not be generalized to other local model families.
One local machine: execution time depends on hardware, model quantization, available memory and Ollama configuration.
Cold-start timing: Ollama was configured with keep_alive=0, meaning the model was reloaded for each response. The reported timings therefore represent this specific cold-start workflow rather than warm-start interaction latency.
Reference-based evaluation: outputs were compared against a frozen, versioned correction reference prepared for this study. This improves consistency across models, but it does not capture every possible acceptable linguistic formulation.
Explanation-quality checks: automatic checks cover selected language patterns and contradictions. They should not be treated as a complete assessment of pedagogical or linguistic quality.

These limitations are also why the benchmark is presented as a controlled comparison rather than a claim about the absolute capabilities of each model.

What I plan to test next

This first benchmark leaves several useful directions open for further evaluation.

The next iterations could extend the framework in four directions:

compare additional local model families under the same evaluation protocol;
examine explanation quality with deeper linguistic and human review;
compare cold-start and warm-start performance;
evaluate the same workflow on more constrained hardware.

A larger and more diverse test set would also help determine whether the patterns observed here remain stable beyond these 20 cases.

I am particularly interested in whether the 8B/14B trade-off observed in this run persists when the task becomes more linguistically demanding.

Conclusion

This experiment did not produce a universal winner, but it did highlight an important practical point: model size alone is not a sufficient criterion for choosing a local writing model.

For this specific workload, Qwen3 8B matched 14B on complete-case correction while its mean cold-start time was roughly half that of 14B in the tested configuration.

The more useful question, therefore, may not be “Which model is the largest?” but rather:

Which model provides the best balance between correction quality, explanation quality, responsiveness, and available hardware for the intended use case?

If you use local LLMs for writing assistance or other interactive tasks, I would be interested to know which trade-off matters most in your own setup: quality, latency, memory requirements, or something else?

Disclosure: AI assistance was used to refine the wording and organization of this article. The benchmark results and measurements reported here come from the documented experimental runs and were reviewed for accuracy before publication.