DEV Community: Erika

I Compared GPT-6 Luna With Four Flash Models

Erika — Thu, 24 Sep 2026 03:35:56 +0000

There are many new “Flash” models now.

The name usually means lower price and faster inference. But it is not one standard class. The models have different sizes, context windows, reasoning modes, and benchmark settings.

I wanted a simple comparison, especially after GPT-6 Luna became another low-cost option.

One disclosure: I work with Token Harbor. The benchmark data below is not from Token Harbor. I used current Artificial Analysis results and linked the sources. Token Harbor access is listed separately.

Quick scorecard

These are current Artificial Analysis Intelligence Index v4.3.2 results checked on September 24, 2026.

Model	AA Intelligence Index	AutomationBench-AA	Terminal-Bench 4.0	SciCode	AA-LCR v1.1
GLM 5.3 Flash	42	60%	33%	52%	80%
Qwen3.8-Flash-Next	40	56%	25%	51%	80%
DeepSeek V4.1 Flash (max)	39	69%	27%	52%	84%
GPT-6 Luna (xhigh)	34	48%	8%	52%	80%
MiMo V2.6 Flash	Not scored yet	—	—	—	—

GLM has the highest composite score. DeepSeek has the best AutomationBench and long-context result in this group. Qwen stays close to both. Luna is lower on the composite score, but it is very cheap and fast.

MiMo needs a note. Artificial Analysis has a score of 46 for MiMo V2.6 Pro, not Flash. I could not find an independent AA Intelligence Index result for MiMo V2.6 Flash. So I left it blank instead of copying the Pro score.

A second model score

LLM Stats has a separate composite score. It is not the AA Intelligence Index, so I did not mix the two numbers.

Model	LLM Stats Score
DeepSeek V4.1 Flash	48.7
Qwen3.8-Flash-Next	48.6
GLM 5.3 Flash	48.2
MiMo V2.6 Flash	45.7
GPT-6 Luna	44.5

This score tells a similar story for the three open Flash models: DeepSeek, Qwen, and GLM are very close. MiMo and Luna follow, with Luna having the lowest input price in the group.

Price and context

Model	Context in AA	Input / 1M	Output / 1M	Weights
GLM 5.3 Flash	1M	$0.15	$0.50	Open
Qwen3.8-Flash-Next	256K	$0.15	$0.47	Open
DeepSeek V4.1 Flash	1M	$0.30	$1.20	Open
GPT-6 Luna	1M	$0.10	$0.50	Proprietary
MiMo V2.6 Flash	1M*	$0.14*	$0.28*	Open

*MiMo values are from Xiaomi, because AA has not published an independent Flash profile yet.

The Qwen naming is also confusing. Alibaba calls the API model Qwen3.8 Flash. Artificial Analysis tests the related Qwen3.8-Flash-Next checkpoint and reports a 256K context window. A hosted route may expose a different limit, so I would check the provider page before sending a very large prompt.

What the scores mean to me

GLM 5.3 Flash: best overall score

GLM is first here with 42. It also has the best Terminal-Bench 4.0 result at 33%.

It looks like the safest general choice when I want one Flash model for mixed work. The downside is speed. Artificial Analysis measured it much slower than DeepSeek in the tested endpoints.

DeepSeek V4.1 Flash: strongest agent result

DeepSeek is third on the total score, but it leads AutomationBench-AA at 69% and AA-LCR at 84%.

That is more useful than the total score for some workloads. For an agent using tools or reading a large codebase, I would test DeepSeek early.

Qwen3.8 Flash: close and inexpensive

Qwen scores 40, only two points behind GLM. Its list price is also a little lower.

It does not win the rows in this small table, but there is no large gap. It may be a better fit when speed and cost matter more than the last benchmark point.

GPT-6 Luna: fast, cheap, but not the score leader

Luna xhigh scores 34. Its AutomationBench result is 48%, and Terminal-Bench 4.0 is 8%.

This does not make Luna useless. It is a proprietary model made for inexpensive, high-volume work. Artificial Analysis measured around 153 output tokens per second. Its measured cost per Intelligence Index task was only $0.04. I see it more as an implementation and background-task model than the first choice for difficult terminal work.

MiMo V2.6 Flash: interesting, but wait for independent scoring

Xiaomi reports several strong numbers for MiMo V2.6 Flash:

Xiaomi-reported benchmark	Score
CyberGym	95.1%
Terminal-Bench 2.1	87.6%
OSWorld-Verified	80.8%
Toolathlon-Verified	73.6%
DeepSWE 1.1	67.9%
AutomationBench v1.0.6	52.3%
Terminal-Bench 4.0	28.8%

It also has a 1M context window and very low list price.

Those numbers are from Xiaomi, not Artificial Analysis. They are useful, but I would not put them in the same ranking table yet. In particular, AutomationBench v1.0.6 is not the same label as AutomationBench-AA. The rank labels in the collected list also need a defined comparison set, so I did not repeat them here.

A separate vendor-reported comparison

I found two benchmark names reported for all five models. This gives a wider view, but the results are not as clean as the independent AA table.

Vendor-reported benchmark	MiMo V2.6 Flash	DeepSeek V4.1 Flash	Qwen3.8 Flash	GLM 5.3 Flash	GPT-6 Luna
DeepSWE 1.1	67.9%	74%	59%	63%	67%
Agents' Last Exam	27.6%	32%	51%	26%	51%

These are directional, not a strict leaderboard. Each vendor used its own harness, context limit, effort setting, and reporting setup. DeepSeek used maximum reasoning and a 1M-token context for DeepSWE. Qwen reports the best of two harnesses with 256K context. GLM used mini-swe-agent with 400K context. OpenAI reports Luna's highest result across an effort sweep.

The results still show something useful. DeepSeek has the strongest published DeepSWE number in this group. Qwen and Luna have the highest published Agents' Last Exam result. MiMo and Luna are almost tied on DeepSWE.

Other vendor benchmarks do not cover all five models. Some interesting examples are DeepSeek's 91% on Terminal-Bench 2.1, GLM's 78% on Toolathlon, Qwen's 92% on LiveCodeBench v6, MiMo's 95.1% on CyberGym, and Luna's 53% on OSWorld 2.0. I kept them out of the main scorecard because they do not form a complete comparison.

Trying the models on Token Harbor

Three models currently have free routes:

Model	Token Harbor model ID	Access
MiMo V2.6 Flash	`mimo-v2.6-flash:free`	Free allowance
DeepSeek V4.1 Flash	`deepseek-v4.1-flash:free`	Free allowance
Qwen3.8 Flash	`qwen3.8-flash:free`	Free allowance
GLM 5.3 Flash	`glm-5.3-flash`	Agent Pass / paid route
GPT-6 Luna	`gpt-6-luna`	New route; check current plan

So the five models are not all free. I think this should be clear before anyone signs up.

For a quick test, I would use the same prompt, reasoning setting, tool schema, and output limit. I would record success, latency, tokens, and corrections. One leaderboard number is useful, but it cannot tell which model will work best with a specific agent harness.

My short choice

For general quality: GLM 5.3 Flash.
For tool use and long context: DeepSeek V4.1 Flash.
For low-cost fast work: Qwen3.8 Flash.
For a proprietary high-throughput route: GPT-6 Luna.
For a new low-cost open model: MiMo V2.6 Flash, but I want independent AA data first.

The gap between GLM, Qwen, and DeepSeek is small. I would not choose only from the total score. The benchmark row closest to the real task matters more.

Sources

I Tried 5 Free LLM APIs With One Python Script

Erika — Mon, 31 Aug 2026 11:31:04 +0000

I wanted to find a free LLM API for small tools and experiments.

There are many lists online, but free plans change often. Some model names and
limits were already old when I checked them. So I opened new accounts and tried
five providers myself:

Groq
Google AI Studio
OpenRouter
Cloudflare Workers AI
Token Harbor

I did not add a credit card to any account. I used the same Python script and
the same two tasks as much as possible.

One disclosure: I work with Token Harbor. To keep the comparison fair, I used
the same test method for all five providers. I also kept failed requests and
problems in the results. This includes the weak points of Token Harbor.

Quick results

Provider and model	Setup	Free limit in my test	Coding task	Context test	Main problem
Groq — `qwen/qwen3.8-27b`	1–2 min	30 RPM, 1K RPD, 8K TPM, 2M TPD	Passed in 1.85s	7,521 input tokens passed in 2.13s	8K TPM limits the size of one request
Google AI Studio — `gemini-3.5-flash-lite`	3–4 min	15 RPM, 250K input TPM, 500 RPD	Passed in 6.54s	First request failed; 84,424-token retry passed in 24.35s	One temporary 503 error
OpenRouter — `poolside/laguna-s-2.1:free`	A few minutes	50 free requests per day	First request got 429; retry passed	First request got 429; 80,464-token retry passed in 9.35s	Shared free pool was busy
Cloudflare Workers AI — `@cf/google/gemma-4-26b-a4b-it`	Longest	10K neurons per day	Returned JSON, but code had one edge-case bug	20K words failed; 42,268-token smaller test passed in 5.93s	Setup was harder and long calls disconnected
Token Harbor — `deepseek-v4-flash:free`	1–2 min	Rolling 7-day value allowance	Passed in 1.84s	63,887 input tokens passed in 34.62s	Dashboard shows a percentage, not an exact cap

These results are from August 28–29, 2026. They are only a snapshot. Free plans
and models can change.

Test method

I used two small tests. My goal was not to make a full model benchmark. I only
wanted to check if the API was really usable.

Test 1: Python debugging

I gave each model a short Python function with several bugs. The model had to:

remove a mutable default argument;
ignore duplicate order IDs;
exclude cancelled orders;
return the correct count and total;
return the answer as JSON.

I checked both the JSON format and the corrected code.

Test 2: long-context retrieval

The script created a synthetic incident log. I placed three exact values near
the start, middle, and end. The model had to find all three values and return
them as JSON.

The normal test was about 20,000 words. I used a smaller input for Groq because
its free plan had an 8K TPM limit. I also reduced the Cloudflare test after the
20K-word requests failed.

I turned off automatic SDK retries. If I retried by hand, I saved both the
failed request and the successful request.

All requests used the same local HTTP proxy from APAC. The latency may be
different in another region.

The basic request looked like this:

from openai import OpenAI

client = OpenAI(
    api_key=API_KEY,
    base_url=PROVIDER_BASE_URL,
    max_retries=0,
)

response = client.chat.completions.create(
    model=MODEL_ID,
    messages=[{"role": "user", "content": prompt}],
    temperature=0,
    max_completion_tokens=3000,
)

All five providers supported an OpenAI-style Chat Completions request. Cloudflare
also needed an Account ID in the base URL.

Groq

Groq was the easiest provider to start using. I signed in with Google and made
an API key. I did not need another email verification step or a credit card.

For qwen/qwen3.8-27b, my account showed these limits:

30 requests per minute
1,000 requests per day
8,000 tokens per minute
2,000,000 tokens per day

The daily limit is large, but the per-minute token limit is much smaller. This
means I could not send the normal 20K-word test in one request.

I used a smaller prompt with 7,521 input tokens. The model found all three
values in 2.13 seconds. The coding task also returned correct code in 1.85
seconds. One sentence in its bug explanation was not correct, but the fixed
function worked.

Groq was the best fit for short and fast requests in this test. For large
prompts, it is important to check TPM and not only TPD.

Groq rate limits

Google AI Studio

Google AI Studio took about three or four minutes to set up. I signed in with
Google, created a project, and generated an API key. It did not require a card.

My project showed 15 RPM, 250K input TPM, and 500 RPD for
gemini-3.5-flash-lite.

The coding task passed in 6.54 seconds. The code was correct, but one sentence
in the explanation was wrong.

The first long-context request failed with HTTP 503 UNAVAILABLE. I tried again
about one minute later. The second request passed:

84,424 input tokens
67 output tokens
24.35 seconds
all three values were correct

I found one difference in Google's usage numbers. The API response reported
84,424 prompt tokens, but the rate-limit page showed a 64.72K TPM peak. I kept
both numbers in my notes because they may use different accounting methods.

Google's terms for unpaid Gemini API services also need attention. Content may
be used to improve Google products, and the rules can be different by region. I
only used synthetic data in this test.

OpenRouter

OpenRouter did not require a credit card or another email verification step.
There were some signup questions, and it showed an option to add a card. I could
skip it.

I used poolside/laguna-s-2.1:free. I chose a fixed model instead of the
openrouter/free router because I wanted to test the same model again on a
retry.

The first coding request returned 429. The first long-context request also
returned 429. Both errors came from the shared upstream free pool. They were not
caused by using all 50 daily requests.

Both manual retries worked. The long-context retry used 80,464 input tokens,
finished in 9.35 seconds, and found all three values.

The OpenRouter Dashboard showed two successful requests, about 81.1K tokens,
and $0.00 spend. However, I could not find a simple counter showing how many of
the 50 daily free requests were still available.

OpenRouter is useful for trying many free models with one API. I would still add
retry logic because the shared free pool can be busy.

OpenRouter free-model limits

Cloudflare Workers AI

Cloudflare did not require a card, but the setup was harder for me. I had to
find the Workers AI REST API page, create a scoped API token, and copy the
Account ID. This was different from a normal API-key page.

The first coding request used all 3,000 output tokens for model thinking and
returned no visible answer. Cloudflare's example for this Gemma model showed
how to turn thinking off:

extra_body={
    "chat_template_kwargs": {
        "enable_thinking": False
    }
}

After this change, the model returned valid JSON in 4.32 seconds. It used 10.82
neurons. The answer was close, but the fixed function missed one edge case. A
cancelled order ID could be accepted later if the same ID appeared again.

The long-context test had another problem. Three 20K-word requests failed before
the client received an answer. Two connections stayed open for about 120 and
111 seconds. The third failed during TLS after 3.48 seconds.

I reduced the prompt to about 10,000 words. This request passed:

42,268 input tokens
54 output tokens
5.93 seconds
all three values were correct

The Dashboard later showed 2.02K of the daily 10K neurons used. Successful
responses explained only about 482 neurons. The remaining amount was close to
the estimated input cost of the two long requests that disconnected after about
two minutes.

This suggests that Cloudflare processed and counted those requests even though
my client did not receive the answers. This is only an estimate from the total
Dashboard usage. I cannot say if Cloudflare, the proxy, or another network part
caused the disconnects.

Token Harbor

Token Harbor took about two minutes to set up. I registered, verified my email,
and generated an API key. It did not require a card.

I tested deepseek-v4-flash:free. The coding task passed in 1.84 seconds. The
normal long-context request also passed:

20,075 words
63,887 input tokens
293 output tokens
34.62 seconds
all three values were correct

The free limit is different from the other providers. It is a value-based
allowance with a personal rolling seven-day period. The Dashboard shows the
used percentage, but it does not show a fixed token number.

About 70K tokens moved the meter from 0% to 4%. A simple estimate gives around
1.5 to 2 million similar DeepSeek tokens for one period. This is not an official
limit. The Dashboard percentage is rounded, and another model may use the
value-based allowance at a different rate.

Permanent free routes are opt-in. Token Harbor may retain prompts and responses
sent through these routes. Paid routes use a different zero-retention policy. I
would not send private code or customer data through the free route.

My final view

Need	My choice from this test
Short and fast requests	Groq
A large free request with clear project limits	Google AI Studio
Many models through one API	OpenRouter, with retry logic
Existing Cloudflare project	Workers AI, after learning its setup
Rolling allowance without a separate TPM limit	Token Harbor

All five APIs worked without a credit card. They were good enough for testing,
personal tools, and low-volume automation.

I would not depend on a free endpoint alone for a user-facing product. Google
returned one 503. OpenRouter returned two shared-pool 429 errors. Cloudflare's
long requests disconnected, and some of that work appeared in the usage meter.

There was also a model-quality issue. Four providers returned functionally
correct Python fixes, but three of those answers included at least one wrong
explanation. Cloudflare returned valid JSON, but its code still had one edge-case
bug.

For me, a free API still needs timeouts, retry rules, logs, and a paid fallback.
The quota number alone is not enough to choose a provider.

I used synthetic prompts only. Limits and available models may change, so
please check the official pages before using them. I used AI to help organize
and edit this article. I checked the final numbers against my saved test files.