mimo-v2.6-pro vs gpt-6-astra: Pelican-on-a-Bicycle, the Candy Box and 6 More Fun Probes — Plus One Case of API-Path "Dumbing Down"

The verdict first, then why the verdict needs a discount.

TL;DR#

• 8 fun probes, 3 runs each, 48 graded calls: mimo-v2.6-pro 23/24, gpt-6-astra 24/24.
• 7 "classic trap" questions (candy-box theory of mind, counting r's, 9.11 vs 9.9, degenerate river crossing, Alice's sisters, Chinese acrostic, self-counting sentence) were answered correctly every time by both models. These probes no longer separate 2026 flagships.
• The only gap was the pelican riding a bicycle SVG: gpt-6-astra produced a drawing 3/3; mimo 2/3, with one run that thought for 331 seconds, burned all 16,000 tokens in reasoning, and returned an empty string. The gap is stability, not ceiling — mimo's best drawing is on par with gpt's.
• Real billed cost: 28 gpt-6-astra calls $0.261**, 28 mimo calls **$0.028 — 9.3×. A good chunk of that is not model pricing, see next point.
• Disclosure: the default route we hit for gpt-6-astra injects roughly 4,100 tokens of hidden prefix into every request (a plain "hi" showed prompt_tokens=4121), and its sampling diversity is abnormally low. Every gpt-6-astra number here describes "gpt-6-astra as reached through that path on 2026-09-22", not OpenAI's native API.

Why the disclosure comes first: what you test over an API is not always the model itself#

Before comparing anything, we inspected the routing for both lines. Most benchmark posts skip this step; it decides how much the numbers are worth.

In other words, every gpt-6-astra request on that line carries an invisible system prompt. That affects three things: the model's behaviour may be shaped by a prompt you cannot see; every call pays about $0.013 of fixed input cost; and usage fields are only partially passed through. The mimo line is an official direct connection with zero injection and complete usage.

This is one concrete form of what people call "API dumbing down": the model did not get worse, but what you are calling sits behind a wrapper between you and the official API. Any gateway, aggregator or resold line can have this — including our own default route. There is exactly one way to check: send "hi" and look at whether prompt_tokens is a single digit.

Two more general caveats: 3 runs per probe is a small sample and only tells you whether a path could do the task reliably that day; the two families use different tokenizers, so token counts are not divisible across families — we compare correctness, output characters, billed dollars and seconds only.

Models and prices#

Prices from the Crazyrouter pricing page; billed amounts from the per-token usage log.

Test setup#

Same OpenAI-compatible endpoint for both, no system prompt, default temperature, non-streaming:

curl https://api.crazyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $CRAZYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mimo-v2.6-pro",
    "messages": [{"role": "user", "content": "Generate an SVG of a pelican riding a bicycle. Output only the SVG code, no explanation, no markdown fences."}],
    "max_tokens": 16000
  }'

Every call logged id, usage (including reasoning_tokens), finish_reason, wall time and the full body. SVGs were rendered to 480px PNG with Playwright for human grading.

The 8 probes and results#

A few samples.

Acrostic, mimo, three runs:

路远山川阔 / 由来天地长 / 更逢春雨后 / 快马踏花香
路途远且长 / 由山入云间 / 更上一重岭 / 快马踏清风
路远行人稀 / 由来秋意深 / 更向孤村去 / 快马踏寒林

Acrostic, gpt-6-astra, three runs:

路远逐晨光 / 由心向远方 / 更登千仞岭 / 快意任翱翔
路远春风近 / 由心逐晓光 / 更登千仞岭 / 快意览朝阳
路远春风近 / 由心逐晓光 / 更登千里岫 / 快意向朝阳

Self-counting sentence — mimo: "This sentence has five words." (5, true); gpt: "This sentence contains exactly seven English words." (7, true).

The pelican: the only probe that separated them#

This is Simon Willison's classic: ask the model to write SVG code for "a pelican riding a bicycle" without seeing any image. It tests whether the model can express the spatial relationship between two complex objects as coordinates.

Raw SVG files (unmodified model output): mimo #0 · mimo #2 · gpt-6-astra #0 · gpt-6-astra #1 · gpt-6-astra #2

Three observations:

1. gpt-6-astra's three drawings are one template with different colours — same beach, clouds, beak shape and scarf. Together with the near-identical acrostics and self-counting sentences, this line's sampling diversity is close to zero, as if the temperature were pinned by the wrapper. Whether that is the path or the model cannot be separated in this round.
2. mimo varies a lot between runs: at its best it matches gpt, at its worst the whole call is wasted. The wasted run cost $0.0139 — mimo's most expensive call of the round — and the caller received an empty string.
3. Wall time: gpt was steady at 82–96 s; mimo ranged from 43 to 331 s.

Latency#

mimo is slightly faster on short answers; gpt is steadier on long outputs.

Production integration advice#

If you integrate mimo-v2.6-pro — or any reasoning model — handle "reasoning ate the budget":

resp = client.chat.completions.create(model="mimo-v2.6-pro", messages=msgs, max_tokens=8000)
choice = resp.choices[0]
if choice.finish_reason == "length" and not choice.message.content:
    # thinking exhausted the budget and the body is empty:
    # retry with a tighter max_tokens or another model; never persist the empty string as a result
    ...

A bigger max_tokens is not automatically safer: mimo's runaway happened precisely because a 16,000-token budget let it keep planning. For deterministic tasks, 4,000–8,000 is often more stable.

Before integrating any overseas flagship, run one "hi" probe: check that prompt_tokens is a single digit, that the id has the official format, and that usage is complete. If any of the three is off, there is a layer in between, and both your benchmarks and your cost estimates need redoing. Crazyrouter's routing inspector shows which channel a model currently lands on and how many candidates it has; after signing up, the models page lists every model's endpoints and prices.

Which one for which job#

• Code or structured output that must come back every time: gpt-6-astra 3/3 vs mimo 2/3 this round — a clear stability edge.
• High-volume short Q&A, cost-sensitive: identical correctness, mimo is 9× cheaper and slightly faster.
• Constrained writing in Chinese: both satisfied the hard constraints; mimo had more variety.
• You need reasoning_tokens for cost control: mimo reports it every time; the gpt-6-astra line only sometimes.

FAQ#

Q: Why fun probes instead of a standard benchmark? Standard benchmarks have most likely leaked into training data. Fun probes — especially the pelican SVG — force the model to encode spatial relationships as coordinates on the spot; there is nothing to memorise. The downside is subjective grading, which is why every raw file is published.

Q: Was mimo's empty output a platform problem? No. finish_reason=length, reasoning_tokens=16003 and content="" came straight from upstream, on an official direct line with zero injection. This is a known risk of reasoning models on open-ended long-output tasks.

Q: What are the 4,100 injected tokens on gpt-6-astra? A prefix added by an intermediate layer: 3,968 tokens cache-hit, identical across 28 calls. We never saw its content (it is not in the response); we only inferred it from prompt_tokens. Ruling out its influence needs a re-run pinned to OpenAI's official channel — that is next round.

Q: How much of the 9.3× cost gap is the injection? About 115K prompt tokens across 28 calls, 111K of them cache hits (cached input is billed at 10% of the input price) — roughly $0.076, or 29$0.261. The rest is the actual model price gap.

Q: Why no multimodal tests? Isn't mimo omni-modal? This round is text only. Image and video input get their own plan.

Q: Are 3 runs enough? Not for "which is smarter". Enough to surface stability signals like "one line answers identically three times" and "one model blows up one run in three". Follow-ups will add 18.7K-token long-context false-premise questions (6 runs each) and heavy-reasoning questions (13 runs each).

Related#

• gpt-6-astra vs Claude Fable 5.1: 60% fewer output characters — but part of it is answering less
• Claude Fable 5.1 vs Fable 5: output efficiency, correctness, and the breaking changes that never raise an error
• Why choose Crazyrouter over OpenRouter? Three concrete differences
• Pricing · Docs

Final verdict#

As of September 2026, on these two specific paths: fun probes cannot separate the two models on intelligence; they do separate them on stability (gpt steady, mimo prone to a runaway) and cost (mimo an order of magnitude cheaper). The more useful takeaway is the one before both: send "hi" before you benchmark, and check who you are actually talking to.

Next round: long-context and heavy-reasoning questions for the same pair; gpt-6-astra re-run pinned to the official channel to test the "identical three times" behaviour; then, per the plan, mimo-v2.6-flash vs claude-sonnet-4-6 and kimi-k2.7-code vs claude-opus-5.