Kimi K3 vs Claude Opus 4.8: Graduate-Level Math, Physics, and Coding Benchmarks

Kimi K3 vs. Claude Opus 4.8: Graduate-Level Math, Physics, and Programming Benchmarks#

Large models all look great on simple arithmetic. What really separates them is whether a derivation can finish within the output budget, whether matrices and boundary conditions are written correctly, and whether the generated program actually matches the semantics of the problem.

This time I raised the difficulty to the level of an undergraduate capstone or a graduate course and tested kimi-k3 and claude-opus-4-8 with the same prompt set:

• Math: stationary distribution, first-passage time, and first and second moments for a three-state finite Markov chain;
• Physics: a two-degree-of-freedom oscillator with grounded damping and coupling damping, solving for the undamped modes and complex frequency response;
• Programming: an exact scheduling algorithm with dependency closure, time windows, daily capacity, and a three-level objective function.

First, the conclusions:

• claude-opus-4-8 returned complete answers for all three tasks within this run’s budget, and all three tasks had finish_reason of stop;
• kimi-k3 had to be run with temperature set to 1; under a 6000 completion token cap, all three tasks ended with finish_reason=length, and the visible content length was 0 in all three cases;
• Opus 4.8’s math and physics answers passed independent numerical verification;
• Opus 4.8’s scheduling code passed its own 13 assertions, but after adding an independent probe for “prerequisites must finish first,” it was found to guarantee only that dependent tasks are selected together, not that prerequisites are scheduled before downstream tasks;
• Average latency for successful requests in this run: Opus 4.8 was about 70.59 seconds, while Kimi K3 was about 225.47 seconds. This difference only reflects this run’s routing and prompt conditions and should not be treated as a long-term SLA.

Test Environment and Parameter Constraints#

The tests were run in Beijing time from 2026-07-18 to 2026-07-19. I first confirmed that both model IDs were visible via /v1/models:

kimi-k3
claude-opus-4-8

All requests went through Crazyrouter’s OpenAI-compatible API:

POST https://cn.crazyrouter.com/v1/chat/completions

Shared conditions:

Same system prompt
Same user problem
No tools and no web access

The model parameters could not be identical, because upstream imposes a hard constraint on Kimi K3:

When I first mistakenly called Kimi K3 with temperature=0.1, the API clearly returned:

Parameter temperature must be 1 for this model.

So this article records that parameter constraint as a test fact instead of hiding the failure as “the model was unavailable.” When using these results for production model selection, the model-specific parameter constraints should also be handled in the configuration layer rather than assuming every model accepts the same sampling settings.

Summary Table#

Kimi’s three runs all had the same shape: the API returned the model name and usage, but message.content contained no deliverable final answer. For applications that need to present results directly to users or hand them to a compiler, this should be treated as “task not completed,” not as “the model got it right but the logs were missing.”

Math Problem: First-Passage Time and Variance in a Finite-State Markov Chain#

The problem uses the transition matrix:

with target state 3, and defines:

$\tau=\min\{n\ge 0:X_n=3\}。$

This is not the kind of problem you can solve by plugging probabilities into a formula. At minimum, you have to handle the stationary distribution, the transient submatrix, the first moment of the first-passage time, and the second moment, while making the boundary conditions for the target state explicit: t_3=s_3=0.

Opus 4.8 Result#

It gave the stationary distribution as:

and verified πP=π component by component.

Using {1,2} as the transient states, it obtained:

The first-moment equation is:

so:

For the second moment, it used:

which gives:

I independently solved the equations with Fraction, and the results matched Opus exactly; substituting t and s back into both first-step equations also checked out. The math task is judged as “fully passed.”

Kimi K3 Result#

Kimi K3 ran for about 235 seconds with temperature=1 and max_tokens=6000. The usage showed completion_tokens=6000, reasoning_tokens=5997, finish_reason=length, and the visible content was empty.

That means we cannot tell from Kimi’s output in this run whether it could solve this Markov chain problem. The only thing we can confirm is that, under the current API, prompt, and budget, it did not condense its reasoning into a final visible answer.

Physics Problem: Matrix Formulation and Frequency Response of a Damped Coupled Oscillator#

The second problem is a two-degree-of-freedom system with two masses:

m1=1.5 kg，m2=1.0 kg
k1=120 N/m，k2=80 N/m，k3=150 N/m
c1=1.2 N·s/m，c2=0.8 N·s/m，c3=1.5 N·s/m
F(t)=10 cos(8t) N

k2 and c2 are the coupling elements between the two masses, while c1 and c3 are the grounded damping elements. The problem asks first for the undamped natural modes and then for the complex amplitude at ω=8 rad/s, so you cannot just throw all damping terms into a diagonal matrix.

Correct Matrices#

Opus 4.8 wrote:

The undamped characteristic equation gives natural angular frequencies:

The corresponding modal ratios X2/X1 are 0.6173 and -2.4298, representing in-phase and out-of-phase modes respectively.

Complex Frequency Response at ω=8#

The complex dynamic stiffness matrix is:

Independent complex-number calculations gave:

Opus’s energy check also passed: the average dissipated power was about 1.2574 W, which matched the input power from the external force within rounding error. It also correctly explained that if c2 is incorrectly treated as grounded damping, the off-diagonal damping terms C12=C21=-0.8 would be zeroed out by mistake, and both the energy transfer and phase results would change.

Kimi K3 Result#

After about 204.6 seconds, Kimi K3 exhausted 6000 tokens and still ended with finish_reason=length, with empty content. Since there were no final equations or numerical values, its physical modeling cannot be evaluated here; this run can only be recorded as “not delivered within budget.”

Programming Problem: Dependency Closure, Time Windows, and Exact Scheduling#

The programming task asks for an implementation of:

schedule_jobs(jobs, dependencies, capacity_by_day)

Each job has duration, release_day, deadline_day, and value; every job must be scheduled entirely within a single day, each day has a capacity cap, and dependencies mean that prerequisite jobs must finish before dependent jobs start. The objective is compared in this order:

1. Maximize total value;
2. Minimize total duration;
3. Make the sorted list of job IDs lexicographically smaller.

This problem is harder than a standard “knapsack + sorting” task because it must handle all of the following at the same time:

• direct and indirect dependency closure;
• unknown dependencies and cycle detection;
• daily capacity and job time windows;
• exact bin packing of feasible subsets;
• multi-level tie-breaking.

Opus 4.8 Implementation Behavior#

Opus produced a complete bitmask + memoized DFS implementation, along with 13 assertions. We extracted the two Python code blocks and executed them in Python 3.11; all of the model’s built-in assertions passed:

all assertions passed

It correctly covered empty input, insufficient capacity, direct/indirect dependencies, unknown dependencies, cyclic dependencies, and the three-level objective comparison.

However, the built-in assertions did not check that prerequisites must complete before downstream tasks. I added an independent probe:

jobs = [
    {"id": "A", "duration": 4, "release_day": 1,
     "deadline_day": 1, "value": 1},
    {"id": "B", "duration": 4, "release_day": 0,
     "deadline_day": 0, "value": 10},
]
result = schedule_jobs(jobs, [("A", "B")], {0: 4, 1: 4})

Under the correct semantics, A is a prerequisite of B, but A can only be scheduled on day 1 and B only on day 0, so they should not both be selected. The model code returned:

selected_ids: ['A', 'B']
assignment: {'A': 1, 'B': 0}

This shows that the implementation only checked that “selecting B requires also selecting A,” but did not enforce:

assignment[A] < assignment[B]

So the programming task cannot be judged as fully passed. More precisely, the algorithmic framework and input validation are fairly complete, and the resource packing logic runs, but the core business semantics missed the time-ordering constraint on dependencies. For a production scheduling system, this is a correctness bug that must be fixed, not a formatting issue.

Kimi K3 Result#

Kimi K3 ran for about 236.4 seconds before ending with length; all 6000 tokens were spent on reasoning, content was empty, and no executable code was delivered.

Latency, Output Budget, and Visible Answers#

Average latency across the three tasks:

Kimi K3:       (235.420 + 204.568 + 236.434) / 3 = 225.474 秒
Claude Opus:   (57.870 + 64.966 + 88.934) / 3 = 70.590 秒

Kimi’s three usage records were very similar:

completion_tokens = 6000
reasoning_tokens  = 5997
content.length    = 0
finish_reason     = length

These numbers show that callers should not look only at the HTTP status code and the model field. At minimum, they should also record:

choice.finish_reason
response.usage
response.choices[0].message.content

If finish_reason is length, then even if the HTTP response is 200, the request should go into retry, fallback, or manual review.

How to Reproduce the Experiment#

Below is the minimal OpenAI Python SDK call pattern. In the actual test, you should use the full problem statements from this article and parameters that match the model:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_CRAZYROUTER_API_KEY",
    base_url="https://cn.crazyrouter.com/v1",
)

response = client.chat.completions.create(
    model="claude-opus-4-8",
    temperature=0.1,
    max_tokens=9000,
    messages=[
        {"role": "system", "content": "Strict benchmark testing, only provide verifiable conclusions."},
        {"role": "user", "content": "Replace with the full graduate-level test problem"},
    ],
)

print(response.model)
print(response.choices[0].finish_reason)
print(response.usage)
print(response.choices[0].message.content)

When calling Kimi K3, you need to change temperature to 1 and reserve enough time for longer reasoning; even with 6000 tokens in this run, no visible answer was obtained.

Production Selection Advice#

When it makes sense to prioritize testing Claude Opus 4.8#

• You need a complete derivation or complete code in a single request;
• The task involves matrices, boundary conditions, and numerical verification;
• Your business is more sensitive to first-screen latency and deliverability;
• You can add independent tests and semantic checks to generated code.

When it still makes sense to keep evaluating Kimi K3#

• Several minutes of latency are acceptable;
• Your calling layer can handle model-specific sampling parameters;
• You are willing to split tasks, increase budget, or use an asynchronous workflow;
• You care more about the reasoning process than about immediate delivery of the final text in one request.

But based on this run’s data, Kimi K3 cannot be described as “it solved all three problems but the output was hidden by logging.” What we actually observed was that all three runs had no visible final answer.

FAQ#

1. Why are these problems considered undergraduate capstone or graduate level?#

The math problem requires the stationary distribution, the fundamental matrix, and the second moment of the first-passage time; the physics problem requires matrix modeling, modal analysis, complex frequency response, and energy balance; the programming problem requires dependency closure, exact packing, cycle detection, and multi-objective optimization. These correspond to the integrated difficulty of probability/stochastic processes, classical mechanics and vibrations, and algorithms/system design courses.

2. Does Kimi K3 not support temperature=0.1?#

In this run, the API explicitly required Kimi K3’s temperature to be 1. This is a model-parameter constraint imposed by the current routing layer, and callers should read the error message and retry with the model’s required configuration.

3. Does Kimi K3’s length mean it definitely cannot solve these problems?#

No. It only means that under this prompt, output limit, and routing condition, it did not produce a visible final answer. Without a visible answer, correctness cannot be scored, but that also does not prove the model cannot solve the task under all settings.

4. Can Opus 4.8’s programming answer be used directly in production?#

Not directly. Its built-in assertions all passed, but an independent probe found that the prerequisite time-ordering constraint was missing. Generated code must be validated with business-semantic tests, not just the tests bundled by the model itself.

5. Why did Opus 4.8’s physics solution need an energy-dissipation check?#

Because the off-diagonal terms in coupled damping are easy to get wrong. Agreement between input power and damping dissipation is independent evidence that the complex amplitudes, phases, and damping matrix are all correct at the same time.

6. Can this run’s latency be treated as a performance commitment?#

No. Upstream routing, caching, concurrency, and retries all affect latency. In production, you should sample repeatedly and report P50, P95, P99, truncation rate, and the cost per successful task.

Final Verdict#

What this run really compared was not some abstract “intelligence ranking” detached from calling conditions, but two very different delivery behaviors:

1. Claude Opus 4.8 produced visible answers for all three graduate-level tasks in this run, and the math and physics derivations passed independent verification;
2. Opus’s code framework is strong, but it still missed the key business constraint that dependent tasks must be completed in time order;
3. Under the forced temperature=1 setting, Kimi K3 used all 6000 tokens on all three tasks, ended with finish_reason=length, and produced no visible answer;
4. If the business goal is to get a complete mathematical derivation, physics calculation, or executable code in a single request, Opus 4.8 is clearly more deliverable in the current test window;
5. Regardless of which model you choose, finish_reason, usage, independent numerical verification, and code semantic tests must all be part of acceptance.

Reproduction files:

• Test script: kimi_k3_vs_opus48_graduate_test.py
• Kimi results: kimi-k3-graduate-results.json
• Comparison results: kimi-k3-vs-opus48-graduate-results.json

Recommended reading:

• Kimi K3 vs GPT-5.6-SOL: High-Difficulty Math, Physics, and Programming Benchmarks
• Kimi K3 vs Claude Fable 5: Complex Reasoning Benchmarks
• Crazyrouter Sign Up