gpt-6-astra vs Claude Fable 5.1: 60% Fewer Output Characters — But Part of That Is Answering Less

gpt-6-astra vs Claude Fable 5.1: 60% Fewer Output Characters — But Part of That Is Answering Less#

Test date: 2026-09-05 (Asia/Shanghai) Base URL: https://cn.crazyrouter.com Endpoint: POST /v1/chat/completions Models: gpt-6-astra, claude-fable-5-1

Bottom line first#

astra emits 0.38–0.77× the output characters of fable-5-1, which looks like an efficiency win. But that number has to be taken apart:

• Group A (both fully correct): character ratio 0.377 — this is clean evidence of concision
• B-q1: character ratio 0.140 — this is not concision. It is 97 characters answering one third of the question versus 866 characters answering all of it
• B-q2: character ratio 0.245 — also not concision. It is a short answer built on a premise that does not hold

Reading group B's character ratio as an efficiency advantage is wrong.

1. First, the part most cross-family comparisons skip#

This matters more than the results themselves, because most cross-family comparisons are published without handling any of it.

1.1 The two models do not share an upstream path#

In the previous round (fable-5.1 vs fable-5, same family) both models could be pinned to a single upstream path, cancelling path differences entirely. That is not possible here — these two models share no upstream path on this gateway.

The best available substitute is pinning each model to one deterministic path, which removes routing randomness. But the fact that the two paths differ cannot be removed; it can only be quantified separately and subtracted.

1.2 Tokenizer density differs by 1.91×, so output token counts cannot be divided across families#

Using one filler passage at increasing lengths, least-squares fit of input_tokens:

Residual 0.00 — both are exactly linear. The slopes differ by 1.91×: the same English text counts as 21 tokens on the Claude side and 11 on the OpenAI side.

Any output-token ratio spanning these two families expresses tokenizer difference, not efficiency. So the primary efficiency metric here is output characters. Token counts are recorded but never divided across families.

One useful side effect: fitting claude-fable-5-1 over /v1/chat/completions reproduced 21.000 / 18.00 — digit for digit the same values obtained in the previous round over the native /v1/messages endpoint. The OpenAI-compatible translation layer does not distort its input accounting, so running this comparison over one unified endpoint is valid.

1.3 Per-request fixed overhead differs by a constant, which has to be removed#

An intercept of 309 is clearly abnormal — the normal range is 8–25. Fitting the same-family model on two other independent paths:

control path 1   slope 11.000   intercept 22.00
control path 2   slope 11.000   intercept 22.00

The slopes match exactly, so 11.000 is the genuine tokenizer density for this family and the accounting itself is sound; normal per-request fixed overhead is 22 tokens. The path used for astra in this round therefore carries roughly 290 extra tokens of fixed content per request that the caller did not send.

That conclusion got an independent cross-check that was not designed for — it simply fell out. The raw input-token ratios across the three workloads have no pattern, but subtracting one single constant of 290 collapses them into a tight band:

One constant explains three ratios that differ by 7×. That can only be an additive fixed block — tokenizer effects are multiplicative, and no single multiplier can send both 4.871 and 0.697 to roughly 0.7.

2. Group A: short single-shot rewrite (3 runs each)#

One-sentence rewrite. No system prompt, no tools, no cache.

claude-fable-5-1   chars 158 / 185 / 142    out_tok 45 / 65 / 47    lat 3.5s
gpt-6-astra        chars  61 /  61 /  61    out_tok 14 / 14 / 14    lat 3.2s

All 6 runs correct. astra is clearly more concise and highly deterministic: 2 of 3 outputs were byte-identical, and the output token count was exactly 14 every time.

astra : "High server traffic caused slower-than-normal response times."

fable : "Because the server had high traffic, users experienced slower
         response times than usual." (plus a shorter alternative)

The 0.377 here is clean evidence of concision, because both sides were fully correct. For classification, extraction, and short rewriting, astra's advantage is real — shorter, faster, and reproducible.

3. Group B: ~18,700-token specification document plus questions#

The specification is 300 programmatically generated rules whose state, threshold, severity, and retention days are each uniquely determined by the index. Ground truth is therefore derivable independently of either model.

q0: list the rule IDs matching two conditions#

Ground truth: i%5==3 (DISPUTED) and i%3==0 (high) → i%15==3 → every 15th rule starting at R003, 20 rules total.

Both models produced the exact 20 — nothing missing, nothing extra. This cell is a tie.

q1: what is the longest retention period, and which rule sets it (6 runs each)#

Ground truth: REVERSED is i%5==2, retention is 30+i%90, maximised at i%90==87 → 117 days, set by a three-way tie of R087 / R177 / R267.

astra got the value 117 right every single time, but attributed it wrongly. And the problem is worse than incompleteness — 2 of the 6 named a rule that is factually wrong:

#1  "117 days, set by Rule R157"
#3  "117 days, set by Rule R157"

R157's retention is 30 + (157 % 90) = 97 days, not 117. That is not "omitted the other two" — it is an attribution that does not hold.

fable-5-1 flagged the tie unprompted every time:

"Longest REVERSED retention: 117 days — and it's a three-way tie,
 not a single rule: R087 / R177 / R267"

Character counts: astra mean 113.8, fable mean 813.8, ratio 0.140. That 0.140 is answering less, not answering efficiently.

q2: a question with a planted false premise (6 runs each)#

The question deliberately embeds a false premise: "Two rules could both fire — name them."

Per the specification, at delay=8h all 60 SETTLED rules fire. The "two" premise simply does not hold.

fable-5-1 rejected it in its opening sentence every time (relative position 0.0 in the answer):

"The premise doesn't hold: it's not two rules, it's sixty."

5 of astra's 6 opened directly with "The two rules are:", went along with the premise, and invented a different arbitrary pair each time:

#0  R006 + R026
#1  R006 + R026
#2  R006 + R011
#3  R006 + R026
#4  R006 + R026

The field values for the rules it picked were themselves accurate (R006 = 36 days, R026 = 56 days, R011 = 41 days). What is wrong is going along with a premise that does not hold and manufacturing a pair to satisfy the word "two." Only 1 of 6 appended "These are not the only matches: all SETTLED rules fire at 8 hours."

6/6 versus 1/6 is the widest gap in this round. For RAG, document Q&A, and support workflows — where the user's question frequently carries a wrong assumption — this matters far more in production than a benchmark score does.

4. Group C: heavy-reasoning scheduling problem (13 runs each)#

Eight jobs with dependencies, five workers. Compute the critical path, the minimum makespan, and identify which single job's duration to cut for the largest makespan reduction.

Ground truth: critical path A→B→E→G→H = 21; with 5 workers maximum concurrency is only 2, so the makespan is also 21; the best single-job cut is G from 6 to 2, bringing the makespan to 17.

astra's failure mode is consistent and legible. It cuts E by 1 second for a makespan of 20, then asserts:

"A reduction of more than 1 second in any one critical-path job cannot
 improve the makespan further, because the alternate path A→C→G→H has
 length 20 s."

That assertion is false. G sits on both long paths — A→B→E→G→H and A→C→G→H — so cutting G lowers both at once. With G at 2: A→B→E→G→H = 17, A→C→G→H = 16, A→C→D→F→H = 17, makespan = 17.

astra looked for the bottleneck along a single critical path and missed that G is a shared node on two. This is a specific, reproducible reasoning blind spot, not noise — all 3 failures out of 13 follow the same pattern.

This is also the only group where fable-5-1 was faster.

5. A unified-endpoint accounting trap#

Over /v1/chat/completions:

claude-fable-5-1   completion_tokens = 2311 / 1990 / 2476
                   reasoning_tokens  = 0 / 0 / 0            ← all zero

gpt-6-astra        reasoning_tokens  = 534 / 740 / 776      ← reported normally

Same model, same problem, over the native /v1/messages endpoint:

claude-fable-5-1   output_tokens   = 2101
                   thinking_tokens = 1243                   ← broken out explicitly

Claude-family thinking tokens are folded into completion_tokens on the OpenAI-compatible endpoint, and reasoning_tokens is reported as 0.

Any system doing cross-model output accounting over a unified endpoint therefore mis-attributes Claude's thinking — it looks as if Claude "did not think," while 1243 thinking tokens were silently merged into the body count. Getting the real split requires the native endpoint.

6. What this round does not prove#

Worth stating explicitly, because these boundaries get dropped more easily than the conclusions do:

Does not prove astra is weaker overall  — only 4 task types, 29 graded items
Does not prove fable-5-1 never fails this way — 29/29 here, but limited samples
Does not prove latency differences come from the models — the two upstream paths differ
Does not license dividing output tokens across families — 1.91x tokenizer density gap
Does not license reading group B's character ratio as efficiency — that brevity is incompleteness

Further limits: group A has 3 samples and B-q0 only 1, so those cells are weaker than the 6- and 13-sample cells; all four workloads are synthetic and do not represent real traffic distribution; and the two models run on different upstream paths, which is the structural limitation this round could not remove.

7. Selection guidance#

Short tasks, classification, extraction, rewriting — pick astra. Equal accuracy, 0.377× the output characters, faster, and highly deterministic (exactly 14 output tokens, byte-reproducible).

Complete enumeration, tied answers, accurate attribution — pick fable-5-1. astra gave incomplete attribution in 5 of 6 runs, and 2 of those attributions were themselves wrong.

User questions that may carry a wrong assumption (RAG / document Q&A / support) — pick fable-5-1. 6/6 versus 1/6, the widest gap measured.

Heavy reasoning, scheduling, path optimization — pick fable-5-1. 13/13 versus 10/13, and astra was slower here.

Cross-model accounting — do not compare output tokens between Claude and non-Claude models on a unified endpoint. Both the tokenizer density and the thinking-token reporting convention differ.

Reproducing this#

Both models are on Crazyrouter over the standard OpenAI-compatible endpoint:

curl https://cn.crazyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6-astra",
    "max_tokens": 8000,
    "messages": [{"role": "user", "content": "..."}]
  }'

Swap model to claude-fable-5-1 for the control. The group C prompt verbatim:

A distributed job queue has 5 workers. Each job has a duration and a dependency list.
Jobs: A(4s, deps=[]), B(3s, deps=[A]), C(7s, deps=[A]), D(2s, deps=[B,C]), E(5s, deps=[B]),
F(1s, deps=[D]), G(6s, deps=[C,E]), H(3s, deps=[F,G]).
Compute the critical path length, the minimum makespan with 5 workers, and identify which single
job's duration you would reduce (and by how much) to shorten the makespan the most.
Show the schedule as a table of (job, start, end, worker). Answer precisely.

Grading: critical path 21, makespan 21, and the best single cut reaching 17 — all three values must be right to pass.

One caution when running repeats: add a unique salt (a timestamp works) to every request, or prompt caching will contaminate the input-token readings. The first fit in this round produced garbage — slope −0.835 — for exactly that reason.

Every figure above was produced against https://cn.crazyrouter.com/v1, where both models are callable directly. Get an API key