8 Fun Probes, 3 Runs Each: mimo-v2.6-pro vs gpt-6-astra in English — and What Happened When We Repeated Them in Russian, Portuguese and Japanese

This is the English-prompt round of a series we started in Chinese. Same two models, same eight probe types, but every prompt was written natively in English, every call was run fresh, and every drawing below was generated for this article. The Chinese, Russian, Portuguese and Japanese rounds are separate experiments with their own data; we cross-reference them at the end because they change the conclusion.

TL;DR#

• 48 calls (8 probes × 2 models × 3 runs); 44 came back, 4 timed out on the client side and are excluded from grading (see "What broke").
• On every probe that came back, both models scored 100%: candy-box false belief, counting e's, 9.11 vs 9.9, the degenerate river crossing, Alice's sisters, a FAST acrostic with six words per line, and a truthful self-counting sentence.
• Pelican riding a bicycle (SVG): mimo-v2.6-pro drew 3/3, in 36–59 s; gpt-6-astra drew 2/2 received (one run timed out), in 83–87 s. gpt's pelicans sit on the saddle; two of mimo's stand on the frame.
• Cost for the calls that came back: mimo $0.013, [gpt-6-astra](/en/pricing/gpt-6-astra)$0.232 — 18×. A third of the gpt bill is a fixed 4,100-token prefix the default route adds to every request (disclosure below).
• The twist: the identical battery in Russian, Portuguese and Japanese collapsed mimo's pelican to 0/3, 0/3 and 1/3 — it kept thinking until it hit the 16,000-token budget and returned nothing. gpt-6-astra drew 9/9 in those languages. In English you would never see this.

Disclosure first: what you reach over an API is not always the model#

We inspected the routing before running anything, because it decides what the numbers mean.

So every gpt-6-astra number here means "gpt-6-astra as reached through that route on 2026-09-22", not OpenAI's native API. The prefix may shape behaviour, it costs about $0.013 per call before you type anything, and it is invisible in the response. The check is trivial and we recommend it before any benchmark: send "hi" and look at prompt_tokens.

Setup#

curl https://api.crazyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $CRAZYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6-astra",
    "messages": [{"role": "user", "content": "Write a four-line poem. The first letters of the lines must spell FAST, in order. Every line must contain exactly six words. Output only the four lines, no title, no punctuation, no explanation."}],
    "max_tokens": 4000
  }'

No system prompt, default temperature, non-streaming, max_tokens 4,000 for text probes and 16,000 for the SVG. Client read timeout 900 s. Every response was logged with id, usage, finish_reason and wall time; SVGs were rendered with Playwright at 480 px for grading. Prices: mimo-v2.6-pro $0.435 /$0.87 per million input / output tokens; gpt-6-astra $5 /$25 (cached input $0.50). See the pricing page.

The eight probes#

Two things the tallies hide.

gpt-6-astra repeats itself. Its three self-counting sentences were byte-identical: "This sentence contains exactly six words." Two of its three acrostics open with the same line, "Fields awaken beneath the silver dawn". mimo produced three different sentences (five, six, six words) and three unrelated poems. We saw the same near-zero diversity from this route in the Chinese round; whether it is the model or a temperature pinned by the wrapper, we cannot tell from outside.

mimo ignores formatting constraints it was not asked about. The candy-box prompt said "exactly one sentence" per answer; mimo answered correctly but wrapped each in bold Markdown and added a "because…" clause. gpt answered in two plain numbered lines.

Acrostics, for the record — mimo:

Frozen stars begin to fade slowly
As golden light spills across fields
Songs rise and bloom into morning
The world wakes and sings again

gpt-6-astra:

Falling leaves drift across quiet rivers
Amber sunlight warms the sleeping meadow
Soft winds carry dreams beyond mountains
Twilight gathers stars above our home

The pelican#

Raw SVGs, unmodified: mimo #0 · mimo #1 · mimo #2 · gpt #0 · gpt #2

Grading by eye: gpt's two drawings both read as "riding". Of mimo's three, none has the bird on the saddle — they are competent bicycles with a pelican placed on top. mimo is faster and 90% cheaper per drawing; gpt's drawing is the one you would keep.

Latency (calls that came back, mean seconds)#

mimo spends its time thinking before short constrained text (the acrostic cost it 300–630 reasoning tokens per run); gpt spends its time on the long SVG output.

What broke: 4 timeouts in 48 calls, on both models#

Four calls never returned within 900 s: gpt's second pelican, one gpt letter-count, one gpt river-crossing (the answer is "1"), and mimo's first candy box. The platform's usage log has no completed record for most of them — they never finished upstream, they were not billed, and we did not receive an error either; the connection simply stayed open. Two models, two upstreams, trivial prompts: this is transport, not intelligence, so we excluded these from the score and report them here instead. If you run long non-streaming calls in production, put your own deadline on them.

What the other languages did to this picture#

We ran the same eight probe types with natively written prompts in Chinese, Russian, Portuguese and Japanese (each is its own article and its own data). The short probes stayed a tie everywhere. The pelican did not:

In the non-English runs mimo did not draw badly — it did not draw at all. It reasoned for 265–437 s (in English, whatever the prompt language), spent the entire 16,000-token budget planning coordinates and redrafting <svg> inside its own reasoning without ever committing one to the answer, returned finish_reason=length with empty content, and was billed $0.0139 for it. Its Japanese acrostic did the same three times out of three at the 4,000-token budget. Ask it in English and none of this happens. That is the single most useful thing this series found, and an English-only benchmark would have hidden it.

Production advice#

For any reasoning model, guard against the budget being eaten by thinking:

resp = client.chat.completions.create(model="mimo-v2.6-pro", messages=msgs, max_tokens=8000)
c = resp.choices[0]
if c.finish_reason == "length" and not c.message.content:
    # thinking exhausted max_tokens; body is empty. Retry with a tighter budget or fall back —
    # and never store the empty string as an answer.
    ...

Prefer English prompts for mimo on open-ended generation until this is fixed, and set your own client timeout on non-streaming calls. Before benchmarking any overseas flagship through a gateway, send "hi" and read prompt_tokens; Crazyrouter's routing inspector shows which channel a model currently lands on and how many candidates exist, once you have an account, and the models page lists endpoints and prices.

FAQ#

Why these probes rather than MMLU-style benchmarks? Because they cannot be memorised: the pelican forces the model to encode spatial relations as coordinates on the spot, the acrostic and self-counting sentence are checked structurally, and the traps test whether the model pattern-matches to a famous puzzle it was not given. The price is subjective grading of the drawings, so every SVG is published.

Is mimo's cross-language collapse a platform artefact? No. Those runs returned finish_reason=length, reasoning_tokens=16000+ and empty content from Xiaomi's own endpoint with zero injection, and the platform billed the tokens. It is the model's thinking loop, triggered by non-English prompts on open-ended tasks.

Why did gpt-6-astra time out on "farmer and cabbage"? We don't know, and neither does the usage log — the request never completed upstream. Four of 48 calls did this across both models. We excluded them rather than guess.

How much of the 18× cost gap is the injected prefix? About 21 × 4,100 ≈ 86K prompt tokens, mostly cache hits at $0.50/M — roughly$0.08 of the $0.23, so a third. The rest is list price.

Is 3 runs enough? For "which is smarter", no. For "which one stops working in Russian", yes — and that is what we found.

Where is the multimodal test? mimo is omni-modal. Not in this series. Text only, on purpose.

Related#

• gpt-6-astra vs Claude Fable 5.1: 60% fewer output characters — but part of it is answering less
• Claude Fable 5.1 vs Fable 5: output efficiency, correctness, and the breaking changes that never raise an error
• Why choose Crazyrouter over OpenRouter? Three concrete differences
• Same experiment in Chinese · Russian · Portuguese · Japanese
• Pricing · Docs

Verdict#

In English, on 2026-09-22, through these two routes: the eight probes cannot separate mimo-v2.6-pro from gpt-6-astra on correctness; they separate them on drawing quality (gpt's pelican rides, mimo's stands), on consistency (gpt repeats itself, mimo varies), and on price (18× in gpt's disfavour, a third of which is a hidden prefix). Then change the prompt language and one of them stops producing output. Test in the language your users actually write.