Claude Fable 5.1 vs Fable 5: Output Efficiency, Correctness, and the Breaking Changes That Never Throw

Claude Fable 5.1 vs Fable 5: Output Efficiency, Correctness, and the Breaking Changes That Never Throw#

Test date: 2026-09-03 (Asia/Shanghai) Base URL: https://cn.crazyrouter.com Endpoint: POST /v1/messages Models: claude-fable-5-1, claude-fable-5

TL;DR#

Most Fable 5.1 coverage restates the same benchmark table. We took a different angle: pin both models to the same upstream route, run three workload shapes three times each, and record correctness, output tokens, thinking budget, and latency together.

Accuracy is identical in all three groups — we did not measure a capability difference. The difference is efficiency: 5.1 reaches the same correct answer with fewer output tokens and less wall-clock time, and the easier the task, the bigger the gap.

Artificial Analysis reported 5.1 using roughly 1.7x the output tokens at max effort. We measured 0.37–0.89x at default effort. Those are not in conflict — see below.

Where the efficiency comes from: thinking budget#

Fable models think adaptively and always on; there is no off switch. thinking_tokens explains the whole curve:

On the easy task 5.1 does not think at all (thinking=0) while 5 still spends 11 tokens getting started. On the hard task both think at full tilt and the gap narrows to 11%.

5.1's improvement is knowing when to stop, not being uniformly shorter. That also resolves the apparent conflict with Artificial Analysis: effort is Fable 5.1's only depth dial, and AA measured max effort across the Intelligence Index suite. Turn the dial to the top and 5.1's thinking volume overtakes. The multiplier changes sign with effort level and task difficulty — do not port a conclusion from one setting to another.

Method#

• POST /v1/messages, native Anthropic protocol, anthropic-version: 2023-06-01
• 3 runs per cell, up to 4 automatic retries; no conclusion rests on a single call
• Correctness graded on objective criteria, not prose style:
  • A: is the sentence genuinely tighter with meaning preserved
  • B: do the rule IDs and figures match the spec document
  • C: critical-path length, makespan and best single-node optimisation must all be right
• Three workload shapes:
  • A. Short one-shot — one sentence to rewrite, no system prompt, no tools, no caching
  • B. Long cached prefix — an ~18,686-token rules document under cache_control: ephemeral, then 3 questions
  • C. Heavy reasoning — an 8-node job-scheduling problem with dependencies

Pin the route, or you are not measuring the model#

Before comparing tokens across models, you have to prove both run on the same upstream path. Fit input_tokens against increasing filler length by least squares; the intercept is the fixed per-request overhead:

Matching slopes mean the same tokenizer and no body inflation; intercepts of 18.0 vs 20.9 mean equivalent fixed overhead. Different upstream paths can differ by hundreds of tokens in intercept — leave routing unpinned and one model may land somewhere else entirely, and you will read that difference as a model property. Every request in this run was pinned to one route.

Results#

A. Short one-shot: same accuracy, 63% fewer output tokens#

claude-fable-5-1   out 55 / 56 / 60    thinking 0    lat 3.27-3.60s
claude-fable-5     out 171 / 161 / 128 thinking 11   lat 4.16-4.48s

All 6 runs rewrote the sentence correctly, with near-identical core output ("Because the server was experiencing heavy traffic, users saw slower response times than usual.").

The difference is packaging: 5.1 gives the answer plus one shorter alternative and stops; 5 appends an unrequested "Key changes" breakdown every time. That is verbosity, not quality — and if a parser sits downstream, the extra prose is noise.

B. Long cached prefix: figures all correct, false premises handled differently#

Both wrote exactly the same cache volume (18,686 tokens) — the prefix cached consistently, neither model quietly skipped the write.

The third question carries a deliberately false premise. It asks "two rules fire simultaneously — which two?", when per the spec all 60 SETTLED rules fire. Both saw through it:

• 5.1 rejected the premise outright — "not two, all 60 SETTLED rules" — and noted the spec contains no conflict-resolution clause, so "which retention wins" has no answer.
• 5 also said "far more than two", but then picked the pair the question probably meant (R006 and R041) and answered anyway, introducing a "longest retention wins" rule of its own.

Factually neither is wrong; R006 and R041 check out. The difference is whether the model goes along with a false premise: 5.1 refused, 5 flagged it and then complied. For work that has to resist leading questions — compliance review, data verification — that difference is worth knowing.

C. Heavy reasoning: 6/6 correct, no significant difference#

5.1  out 2098 / 2357 / 2304   thinking 1131 / 1310 / 1343   lat 26.81-28.80s
5    out 2720 / 2579 / 2284   thinking 1799 / 1458 / 1437   lat 28.41-34.12s

Ground truth: critical path A->B->E->G->H = 21; maximum DAG width is 2, so 5 workers never bind and makespan is also 21; the best single-node optimisation cuts G from 6 to 2, bringing makespan to 17.

All 6 runs got all three figures right. Both models additionally spotted that cutting A by 4 also yields 17 but is degenerate (zero-duration job), and that past 17 the A->C->D->F->H path binds so further cuts buy nothing. This problem does not separate them.

Treat the token volume as a tie too: 5's lowest run (2284) is below 5.1's highest (2357), so the 3-sample ranges overlap. The 0.891 figure in group C is not a valid difference.

Latency#

5.1 is faster in all three, and the gap narrows in step with the token curve — latency difference is essentially driven by output volume, not by raw inference speed. Latency is network- and load-sensitive; treat it as context, not as a capability score.

Breaking changes that never throw#

Anthropic's migration guide lists several changes that return 400 on Fable 5.1. We sent each one 6 times through the gateway:

All seven pass through silently. Which means:

1. Your migration will not fail on these — but it will not surface them either. The parameter is dropped mid-flight, the request returns 200, and the behaviour differs from what you asked for.
2. temperature / top_p / top_k especially: if production code leans on a low temperature for output stability, those values do nothing on Fable models, with no warning.
3. The docs describe forced tool choice failing loudly with a 400. That holds on a direct connection. It does not hold behind any relay layer.

The right way to validate a migration is not to wait for errors but to assert on response content: check stop_reason, check that the tool you forced was actually called, check that repeated identical requests behave the way your temperature setting implies.

Reproduce it#

Compare output efficiency (substitute your own $KEY):

for M in claude-fable-5-1 claude-fable-5; do
  curl -s https://cn.crazyrouter.com/v1/messages \
    -H "x-api-key: $KEY" \
    -H "anthropic-version: 2023-06-01" \
    -H "content-type: application/json" \
    -d "{\"model\":\"$M\",\"max_tokens\":2048,\"messages\":[{\"role\":\"user\",
        \"content\":\"Rewrite this sentence to be more concise: 'Due to the fact that the server was experiencing an elevated level of traffic, the response times that users experienced were longer than what would normally be expected.'\"}]}" \
    | python -c "import sys,json;d=json.load(sys.stdin);u=d['usage'];print(d['model'],'out',u['output_tokens'],'think',u.get('output_tokens_details',{}).get('thinking_tokens'))"
done

Check whether a breaking change is silent:

curl -s https://cn.crazyrouter.com/v1/messages \
  -H "x-api-key: $KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-fable-5-1","max_tokens":64,"temperature":0.7,
       "messages":[{"role":"user","content":"What is the weather in Tokyo?"}]}' \
  -w "\nHTTP %{http_code}\n"

Fit per-request overhead to confirm both models share a route:

# Send increasing filler to one model, least-squares fit input_tokens.
# An intercept of 8-25 is normal; the two models must be close for the
# comparison to be valid at all.
UNIT = "The reconciliation service records each settlement attempt in the ledger. "
for n in [1, 20, 100, 250]:
    body = {"model": M, "max_tokens": 16,
            "messages": [{"role": "user", "content": UNIT * n + "Reply with the single word: ok"}]}

Choosing between them#

1. Short tasks, classification, extraction, rewriting — switch to 5.1. Same accuracy, 63% fewer output tokens, 21% faster, no code changes needed. This is where the gain is largest.
2. Work that must resist leading questions — lean 5.1. On question 3 it rejected the false premise outright while 5 flagged it and then played along. Small sample; re-test with your own leading questions.
3. Heavy reasoning — indistinguishable (6/6 correct, overlapping token ranges). Whether to switch depends on wanting 5.1's newer knowledge cutoff (June 2026 vs January 2026), not on anything measured here.
4. Re-test before touching effort. Anthropic states explicitly that effort levels do not represent equal thinking volume across models; settings tuned on Fable 5 will not carry over. This is the key variable behind AA's 1.7x versus our 0.37–0.89x.
5. Validate migrations by asserting on content, not by waiting for errors. See the section above.

Limitations#

• 3 samples per cell — enough for orders of magnitude, not for significance testing. Group C's ranges overlap and should not drive a decision.
• All three groups tied on correctness, so this run found no capability difference, only efficiency and behavioural ones. Separating capability needs harder problems with a real failure rate.
• Only default effort was tested. AA's 1.7x is a max-effort result; not reproducing that setting here is not a refutation of it — the two measure different points on the same curve.
• The false-premise finding rests on one question, once per model. "5.1 resists leading questions better" is an observation, not a conclusion.
• The three workload shapes are synthetic and do not represent your traffic mix. Re-run with your own workloads and acceptance criteria.

FAQ#

Q1: Is Fable 5.1 smarter than Fable 5? Across three workload shapes and 18 calls, correctness tied everywhere — no capability difference was measured. What was measured is efficiency: the same correct answer for fewer output tokens and less time. Separating capability requires problems with a real failure rate.

Q2: Other benchmarks say 5.1 uses more output tokens. Why did you measure fewer? effort is Fable 5.1's only thinking-depth dial. Artificial Analysis measured max effort; this run used the default. The multiplier changes sign with the setting and the task — 0.372x on an easy prompt at default, 0.891x on a hard one, and it overtakes at max. Do not port a number across settings.

Q3: Does migrating to Fable 5.1 require code changes? Swapping the model ID from claude-fable-5 to claude-fable-5-1 is enough. But check three things: whether you use forced tool choice, whether you hand-assemble the messages array (thinking blocks now bind to the conversation), and whether you depend on sampling parameters like temperature. None of these throw behind a relay, so assert on responses.

Q4: Why doesn't thinking: {"type": "disabled"} turn thinking off? Fable models have exactly one thinking mode: adaptive, always on, no off switch. The docs say passing disabled returns 400, but we measured 200 six times out of six — the parameter is dropped and thinking proceeds. effort is the only way to influence depth.

Q5: Why does the same model report different token counts in different places? Every upstream path can carry its own fixed overhead. Fit input_tokens linearly against increasing filler; the intercept is that overhead and should land between 8 and 25. Slope reflects the tokenizer — if two paths share a slope but differ sharply in intercept, that is a path difference, not a model difference.

Q6: What context window and knowledge cutoff does Fable 5.1 have? 1M-token context, 128K max output — same as Fable 5. Knowledge cutoff is June 2026 against Fable 5's January 2026.

Q7: Why does the upstream route have to be pinned? With routing unpinned, the two models can land on different upstream paths, and the paths' fixed-overhead difference gets misread as a model difference. Here both measured 18.0 vs 20.9 intercept and 21.0 vs 21.2 slope — near-equivalent, which is what makes the comparison valid.

References#

• Anthropic migration guide
• Anthropic: what's new in Fable 5.1
• Artificial Analysis: Fable 5.1
• Snorkel AI: independent Fable 5.1 coding evaluation

Every figure above was produced against https://cn.crazyrouter.com/v1, where both models are callable directly. Get an API key