25 measured metrics · 14 calculated · 5 studies
Claude Haiku 4.5vsClaude Opus 5.5
Claude Opus 5.5 ahead on 3; 7 ties, 29 unclear. A side is ahead only where the intervals or ranges do not overlap.
The verdict
Claude Haiku 4.5 and Claude Opus 5.5 share 25 measured metrics and 14 list-price calculations from 5 studies. Claude Opus 5.5 leads on 3 rows: Pass rate on eight hard tasks (Strict pass), 100% (24/24) vs 46% (11/24); Pass rate on eight hard tasks (Lenient (format misses counted)), 100% (24/24) vs 67% (16/24); Pass rate on 4 harder tasks (Lenient (format misses counted)), 50% (6/12) vs 0% (0/12). On those rows the 95% intervals do not overlap. The other rows are 7 ties and 29 unclear; each row says why. Calculation rows are derived from list prices and recorded counts; they are not bills or runs. Some rows rest on small samples (n = 3 at the smallest).
Headline metrics
How far apart the two sides are on the headline metrics. Length is the ratio of the two values; it is not a winner.
Bar length is the ratio of the two values on a log scale, pointing to the larger one. Larger is not better for time, tokens or cost. A bar has a side’s color only when that side is ahead in the data; gray means the data does not separate them.
| Metric | Claude Haiku 4.5 | Claude Opus 5.5 | n | Interval or range | Outcome | Basis | Study |
|---|---|---|---|---|---|---|---|
| Pass rate on five validated tasks | 100% (15/15)Claude Code · five short validated tasks | 100% (15/15)Claude Code · five short validated tasks | 15 | 95% CI: 80%–100% vs 80%–100% | Tie | The 95% intervals overlap (Claude Haiku 4.5 80% to 100%; Claude Opus 5.5 80% to 100%), so this sample cannot separate them. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Total time per call | 4.43 sClaude Code · five short validated tasks | 2.75 sClaude Code · five short validated tasks | 15 | range: 3.2 s–23.6 s vs 2.5 s–8.9 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 3.16 s to 23.6 s; Claude Opus 5.5 2.47 s to 8.91 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Time to first useful output | 3.63 sClaude Code · five short validated tasks | 1.92 sClaude Code · five short validated tasks | 15 | range: 2.8 s–22.3 s vs 1.6 s–7.2 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.78 s to 22.3 s; Claude Opus 5.5 1.56 s to 7.23 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Input tokens per call: what the CLI sends (Other input) | 3,790Claude Code · five short validated tasks | 680Claude Code · five short validated tasks | 15 | none recorded | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Output tokens per call (Output tokens) | 367Claude Code · five short validated tasks | 64Claude Code · five short validated tasks | 15 | none recorded | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| List-price cost per passing answer (calculation)Calculation | $0.0084Claude Code · five short validated tasks | $0.010Claude Code · five short validated tasks | 15 | none recorded | Unclear | No interval or range was recorded for either side, so the gap ($0.0084 vs $0.010) is not tested against run-to-run variation. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
Marks: 95% intervals (Wilson for rates); fastest–slowest run ranges (not intervals)n is shown per side on every rowHollow marks: list-price calculations, not runs
6 headline metrics as the ratio of the two values. None of them separates the sides in the data. Widest ratio: Output tokens per call (Output tokens), 5.7x (Claude Haiku 4.5 larger).
Metric by metric
Both values of a row come from the same chart in the same study. Each row has its own axis. The shaded band is where the two intervals or ranges overlap: a side is ahead only when they do not.
- Claude Haiku 4.5
- Claude Opus 5.5
- 95% interval
- fastest–slowest run (not an interval)
- where the two overlap
- hollow: list-price calculation
These calculations hold each rate fixed and assume independent samples. They are not a power calculation or a paired test. Open the sample-size planner.
Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head
- Pass rate on five validated tasks100% (15/15)n 15100% (15/15)n 15TiePass rate on five validated tasks: Claude Haiku 4.5 100% (15/15) (n 15, 95% interval 80%–100%); Claude Opus 5.5 100% (15/15) (n 15, 95% interval 80%–100%). Tie.
- Total time per call4.43 sn 152.75 sn 15UnclearTotal time per call: Claude Haiku 4.5 4.43 s (n 15, run range 3.2 s–23.6 s); Claude Opus 5.5 2.75 s (n 15, run range 2.5 s–8.9 s). Unclear.
- Time to first useful output3.63 sn 151.92 sn 15UnclearTime to first useful output: Claude Haiku 4.5 3.63 s (n 15, run range 2.8 s–22.3 s); Claude Opus 5.5 1.92 s (n 15, run range 1.6 s–7.2 s). Unclear.
- Input tokens per call: what the CLI sends (Cache read)0n 151,401n 15UnclearInput tokens per call: what the CLI sends (Cache read): Claude Haiku 4.5 0 (n 15); Claude Opus 5.5 1,401 (n 15). Unclear.
- Input tokens per call: what the CLI sends (Other input)3,790n 15680n 15UnclearInput tokens per call: what the CLI sends (Other input): Claude Haiku 4.5 3,790 (n 15); Claude Opus 5.5 680 (n 15). Unclear.
- Output tokens per call (Output tokens)367n 1564n 15UnclearOutput tokens per call (Output tokens): Claude Haiku 4.5 367 (n 15); Claude Opus 5.5 64 (n 15). Unclear.
- List-price cost per call (calculation)Calculation$0.0057n 15$0.0069n 15UnclearList-price cost per call (calculation), calculation: Claude Haiku 4.5 $0.0057 (n 15, run range $0.0051–$0.018); Claude Opus 5.5 $0.0069 (n 15, run range $0.0059–$0.022). Unclear.
- List-price cost per passing answer (calculation)Calculation$0.0084n 15$0.010n 15UnclearList-price cost per passing answer (calculation), calculation: Claude Haiku 4.5 $0.0084 (n 15); Claude Opus 5.5 $0.010 (n 15). Unclear.
Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks
- Pass rate on eight hard tasks (Strict pass)46% (11/24)n 24100% (24/24)n 24Claude Opus 5.5 aheadPass rate on eight hard tasks (Strict pass): Claude Haiku 4.5 46% (11/24) (n 24, 95% interval 28%–65%); Claude Opus 5.5 100% (24/24) (n 24, 95% interval 86%–100%). Claude Opus 5.5 ahead.
- Pass rate on eight hard tasks (Lenient (format misses counted))67% (16/24)n 24100% (24/24)n 24Claude Opus 5.5 aheadPass rate on eight hard tasks (Lenient (format misses counted)): Claude Haiku 4.5 67% (16/24) (n 24, 95% interval 47%–82%); Claude Opus 5.5 100% (24/24) (n 24, 95% interval 86%–100%). Claude Opus 5.5 ahead.
- Total time per call on hard tasks (separate batches)39.0 sn 249.18 sn 24UnclearTotal time per call on hard tasks (separate batches): Claude Haiku 4.5 39.0 s (n 24, run range 15.3 s–75.1 s); Claude Opus 5.5 9.18 s (n 24, run range 4.2 s–27.2 s). Unclear.
- Time to first useful output on hard tasks35.5 sn 246.78 sn 24UnclearTime to first useful output on hard tasks: Claude Haiku 4.5 35.5 s (n 24, run range 12.9 s–70.3 s); Claude Opus 5.5 6.78 s (n 24, run range 2.4 s–21.8 s). Unclear.
- Output tokens per call on hard tasks (Output tokens)5,064n 24945n 24UnclearOutput tokens per call on hard tasks (Output tokens): Claude Haiku 4.5 5,064 (n 24); Claude Opus 5.5 945 (n 24). Unclear.
- List-price cost per strict pass on hard tasks (calculation)Calculation$0.067n 24$0.028n 24UnclearList-price cost per strict pass on hard tasks (calculation), calculation: Claude Haiku 4.5 $0.067 (n 24); Claude Opus 5.5 $0.028 (n 24). Unclear.
How much of an AI bill is thinking? Reasoning tokens by model and effort
- Reasoning share of output tokens per call on hard tasks (calculation)Calculation91.7%n 2454.8%n 24UnclearReasoning share of output tokens per call on hard tasks (calculation), calculation: Claude Haiku 4.5 91.7% (n 24, run range 76%–99%); Claude Opus 5.5 54.8% (n 24, run range 30%–96%). Unclear.
- List-price cost per call: reasoning, remaining output and input (calculation) (Reasoning (output tokens))Calculation$0.024n 24$0.013n 24UnclearList-price cost per call: reasoning, remaining output and input (calculation) (Reasoning (output tokens)), calculation: Claude Haiku 4.5 $0.024 (n 24); Claude Opus 5.5 $0.013 (n 24). Unclear.
- List-price cost per call: reasoning, remaining output and input (calculation) (Remaining output (visible-answer estimate))Calculation$0.0018n 24$0.0080n 24UnclearList-price cost per call: reasoning, remaining output and input (calculation) (Remaining output (visible-answer estimate)), calculation: Claude Haiku 4.5 $0.0018 (n 24); Claude Opus 5.5 $0.0080 (n 24). Unclear.
- List-price cost per call: reasoning, remaining output and input (calculation) (Input (prompt, cache priced))Calculation$0.0045n 24$0.0077n 24UnclearList-price cost per call: reasoning, remaining output and input (calculation) (Input (prompt, cache priced)), calculation: Claude Haiku 4.5 $0.0045 (n 24); Claude Opus 5.5 $0.0077 (n 24). Unclear.
- Reasoning share on short tasks vs hard tasks (calculation) (Eight hard tasks)Calculation91.7%n 2454.8%n 24UnclearReasoning share on short tasks vs hard tasks (calculation) (Eight hard tasks), calculation: Claude Haiku 4.5 91.7% (n 24, run range 76%–99%); Claude Opus 5.5 54.8% (n 24, run range 30%–96%). Unclear.
- Reasoning share on short tasks vs hard tasks (calculation) (Five short tasks)Calculation90.2%n 150%n 15UnclearReasoning share on short tasks vs hard tasks (calculation) (Five short tasks), calculation: Claude Haiku 4.5 90.2% (n 15, run range 73%–98%); Claude Opus 5.5 0% (n 15, run range 0%–93%). Unclear.
Where the seconds go: first text, output speed and prompt size for 6 LLMs
- Time to first text: a 250-line answer, six models4.00 sn 41.97 sn 4UnclearTime to first text: a 250-line answer, six models: Claude Haiku 4.5 4.00 s (n 4, run range 2.8 s–6.4 s); Claude Opus 5.5 1.97 s (n 4, run range 1.7 s–2.4 s). Unclear.
- Output speed after the first text: visible tokens per second (calculation)Calculation153n 4156n 4UnclearOutput speed after the first text: visible tokens per second (calculation), calculation: Claude Haiku 4.5 153 (n 4, run range 153–216); Claude Opus 5.5 156 (n 4, run range 155–156). Unclear.
- Output speed in characters per second after the first text (calculation)Calculation547n 3347n 4UnclearOutput speed in characters per second after the first text (calculation), calculation: Claude Haiku 4.5 547 (n 3, run range 546–548); Claude Opus 5.5 347 (n 4, run range 345–349). Unclear.
- Time to first text as the prompt grows: 1kCalculation1.93 sn 31.51 sn 3UnclearTime to first text as the prompt grows: 1k, calculation: Claude Haiku 4.5 1.93 s (n 3, run range 1.9 s–2 s); Claude Opus 5.5 1.51 s (n 3, run range 1.5 s–2 s). Unclear.
- Time to first text as the prompt grows: 16kCalculation2.27 sn 31.74 sn 3UnclearTime to first text as the prompt grows: 16k, calculation: Claude Haiku 4.5 2.27 s (n 3, run range 2.2 s–2.5 s); Claude Opus 5.5 1.74 s (n 3, run range 1.7 s–3 s). Unclear.
- Time to first text as the prompt grows: 64kCalculation2.78 sn 31.79 sn 3UnclearTime to first text as the prompt grows: 64k, calculation: Claude Haiku 4.5 2.78 s (n 3, run range 2.5 s–2.9 s); Claude Opus 5.5 1.79 s (n 3, run range 1.7 s–3.7 s). Unclear.
- Total time per call by prompt size (1k prompt)2.34 sn 31.83 sn 3UnclearTotal time per call by prompt size (1k prompt): Claude Haiku 4.5 2.34 s (n 3, run range 2.2 s–2.5 s); Claude Opus 5.5 1.83 s (n 3, run range 1.8 s–2.4 s). Unclear.
- Total time per call by prompt size (16k prompt)2.79 sn 32.36 sn 3UnclearTotal time per call by prompt size (16k prompt): Claude Haiku 4.5 2.79 s (n 3, run range 2.6 s–2.8 s); Claude Opus 5.5 2.36 s (n 3, run range 2.1 s–3.4 s). Unclear.
- Total time per call by prompt size (64k prompt)3.13 sn 32.35 sn 3UnclearTotal time per call by prompt size (64k prompt): Claude Haiku 4.5 3.13 s (n 3, run range 2.8 s–3.3 s); Claude Opus 5.5 2.35 s (n 3, run range 2.3 s–4.3 s). Unclear.
- Exact lookup answers at the 1k, 16k and 64k prompt-size targets100% (9/9)n 956% (5/9)n 9TieExact lookup answers at the 1k, 16k and 64k prompt-size targets: Claude Haiku 4.5 100% (9/9) (n 9, 95% interval 70%–100%); Claude Opus 5.5 56% (5/9) (n 9, 95% interval 27%–81%). Tie.
Calculation: at these rates, about 13 runs per side would separate them.
GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks
- Pass rate on 4 harder tasks (Strict pass)0% (0/12)n 1242% (5/12)n 12TiePass rate on 4 harder tasks (Strict pass): Claude Haiku 4.5 0% (0/12) (n 12, 95% interval 0%–24%); Claude Opus 5.5 42% (5/12) (n 12, 95% interval 19%–68%). Tie.
Calculation: at these rates, about 16 runs per side would separate them.
- Pass rate on 4 harder tasks (Lenient (format misses counted))0% (0/12)n 1250% (6/12)n 12Claude Opus 5.5 aheadPass rate on 4 harder tasks (Lenient (format misses counted)): Claude Haiku 4.5 0% (0/12) (n 12, 95% interval 0%–24%); Claude Opus 5.5 50% (6/12) (n 12, 95% interval 25%–75%). Claude Opus 5.5 ahead.
- Calls that tried a tool although tools were off8% (1/12)n 1242% (5/12)n 12UnclearCalls that tried a tool although tools were off: Claude Haiku 4.5 8% (1/12) (n 12, 95% interval 1.5%–35%); Claude Opus 5.5 42% (5/12) (n 12, 95% interval 19%–68%). Unclear.
- Strict pass rate by task: 10x10 nonogram0% (0/3)n 3100% (3/3)n 3TieStrict pass rate by task: 10x10 nonogram: Claude Haiku 4.5 0% (0/3) (n 3, 95% interval 0%–56%); Claude Opus 5.5 100% (3/3) (n 3, 95% interval 44%–100%). Tie.
Calculation: at these rates, about 4 runs per side would separate them.
- Strict pass rate by task: Sudoku, 22 givens0% (0/3)n 30% (0/3)n 3TieStrict pass rate by task: Sudoku, 22 givens: Claude Haiku 4.5 0% (0/3) (n 3, 95% interval 0%–56%); Claude Opus 5.5 0% (0/3) (n 3, 95% interval 0%–56%). Tie.
- Strict pass rate by task: 6x6 Skyscrapers0% (0/3)n 333% (1/3)n 3TieStrict pass rate by task: 6x6 Skyscrapers: Claude Haiku 4.5 0% (0/3) (n 3, 95% interval 0%–56%); Claude Opus 5.5 33% (1/3) (n 3, 95% interval 6.2%–79%). Tie.
Calculation: at these rates, about 20 runs per side would separate them.
- Strict pass rate by task: Seeded shuffle output0% (0/3)n 333% (1/3)n 3TieStrict pass rate by task: Seeded shuffle output: Claude Haiku 4.5 0% (0/3) (n 3, 95% interval 0%–56%); Claude Opus 5.5 33% (1/3) (n 3, 95% interval 6.2%–79%). Tie.
Calculation: at these rates, about 20 runs per side would separate them.
- Total time per call on harder tasks109.0 sn 1080.3 sn 9UnclearTotal time per call on harder tasks: Claude Haiku 4.5 109.0 s (n 10, run range 25.7 s–224 s); Claude Opus 5.5 80.3 s (n 9, run range 3.8 s–280 s). Unclear.
- Output tokens per call on harder tasks (Output tokens)12,508n 108,420n 9UnclearOutput tokens per call on harder tasks (Output tokens): Claude Haiku 4.5 12,508 (n 10, run range 2,965–26,532); Claude Opus 5.5 8,420 (n 9, run range 279–40,044). Unclear.
| Metric | Claude Haiku 4.5 | Claude Opus 5.5 | n | Interval or range | Outcome | Basis | Study |
|---|---|---|---|---|---|---|---|
| Pass rate on five validated tasks | 100% (15/15)Claude Code · five short validated tasks | 100% (15/15)Claude Code · five short validated tasks | 15 | 95% CI: 80%–100% vs 80%–100% | Tie | The 95% intervals overlap (Claude Haiku 4.5 80% to 100%; Claude Opus 5.5 80% to 100%), so this sample cannot separate them. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Total time per call | 4.43 sClaude Code · five short validated tasks | 2.75 sClaude Code · five short validated tasks | 15 | range: 3.2 s–23.6 s vs 2.5 s–8.9 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 3.16 s to 23.6 s; Claude Opus 5.5 2.47 s to 8.91 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Time to first useful output | 3.63 sClaude Code · five short validated tasks | 1.92 sClaude Code · five short validated tasks | 15 | range: 2.8 s–22.3 s vs 1.6 s–7.2 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.78 s to 22.3 s; Claude Opus 5.5 1.56 s to 7.23 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Input tokens per call: what the CLI sends (Cache read) | 0Claude Code · five short validated tasks | 1,401Claude Code · five short validated tasks | 15 | none recorded | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Input tokens per call: what the CLI sends (Other input) | 3,790Claude Code · five short validated tasks | 680Claude Code · five short validated tasks | 15 | none recorded | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Output tokens per call (Output tokens) | 367Claude Code · five short validated tasks | 64Claude Code · five short validated tasks | 15 | none recorded | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| List-price cost per call (calculation)Calculation | $0.0057Claude Code · five short validated tasks | $0.0069Claude Code · five short validated tasks | 15 | range: $0.0051–$0.018 vs $0.0059–$0.022 | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 $0.0051 to $0.018; Claude Opus 5.5 $0.0059 to $0.022); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| List-price cost per passing answer (calculation)Calculation | $0.0084Claude Code · five short validated tasks | $0.010Claude Code · five short validated tasks | 15 | none recorded | Unclear | No interval or range was recorded for either side, so the gap ($0.0084 vs $0.010) is not tested against run-to-run variation. | Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head |
| Pass rate on eight hard tasks (Strict pass) | 46% (11/24)Claude Code · eight hard validated tasks | 100% (24/24)Claude Code · eight hard validated tasks | 24 | 95% CI: 28%–65% vs 86%–100% | Claude Opus 5.5 ahead | The 95% intervals do not overlap (Claude Haiku 4.5 28% to 65%; Claude Opus 5.5 86% to 100%). | Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks |
| Pass rate on eight hard tasks (Lenient (format misses counted)) | 67% (16/24)Claude Code · eight hard validated tasks | 100% (24/24)Claude Code · eight hard validated tasks | 24 | 95% CI: 47%–82% vs 86%–100% | Claude Opus 5.5 ahead | The 95% intervals do not overlap (Claude Haiku 4.5 47% to 82%; Claude Opus 5.5 86% to 100%). | Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks |
| Total time per call on hard tasks (separate batches) | 39.0 sClaude Code · eight hard validated tasks | 9.18 sClaude Code · eight hard validated tasks | 24 | range: 15.3 s–75.1 s vs 4.2 s–27.2 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 15.3 s to 75.1 s; Claude Opus 5.5 4.24 s to 27.2 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks |
| Time to first useful output on hard tasks | 35.5 sClaude Code · eight hard validated tasks | 6.78 sClaude Code · eight hard validated tasks | 24 | range: 12.9 s–70.3 s vs 2.4 s–21.8 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 12.9 s to 70.3 s; Claude Opus 5.5 2.39 s to 21.8 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks |
| Output tokens per call on hard tasks (Output tokens) | 5,064Claude Code · eight hard validated tasks | 945Claude Code · eight hard validated tasks | 24 | none recorded | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks |
| List-price cost per strict pass on hard tasks (calculation)Calculation | $0.067Claude Code · eight hard validated tasks | $0.028Claude Code · eight hard validated tasks | 24 | none recorded | Unclear | No interval or range was recorded for either side, so the gap ($0.067 vs $0.028, 2.4x) is not tested against run-to-run variation. | Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks |
| Reasoning share of output tokens per call on hard tasks (calculation)Calculation | 91.7%Claude Code | 54.8%Claude Code | 24 | range: 76%–99% vs 30%–96% | Unclear | More or fewer percent is not better or worse by itself; this row describes behaviour, not a winner. | How much of an AI bill is thinking? Reasoning tokens by model and effort |
| List-price cost per call: reasoning, remaining output and input (calculation) (Reasoning (output tokens))Calculation | $0.024Claude Code | $0.013Claude Code | 24 | none recorded | Unclear | More or fewer usd is not better or worse by itself; this row describes behaviour, not a winner. | How much of an AI bill is thinking? Reasoning tokens by model and effort |
| List-price cost per call: reasoning, remaining output and input (calculation) (Remaining output (visible-answer estimate))Calculation | $0.0018Claude Code | $0.0080Claude Code | 24 | none recorded | Unclear | More or fewer usd is not better or worse by itself; this row describes behaviour, not a winner. | How much of an AI bill is thinking? Reasoning tokens by model and effort |
| List-price cost per call: reasoning, remaining output and input (calculation) (Input (prompt, cache priced))Calculation | $0.0045Claude Code | $0.0077Claude Code | 24 | none recorded | Unclear | More or fewer usd is not better or worse by itself; this row describes behaviour, not a winner. | How much of an AI bill is thinking? Reasoning tokens by model and effort |
| Reasoning share on short tasks vs hard tasks (calculation) (Eight hard tasks)Calculation | 91.7%Claude Code | 54.8%Claude Code | 24 | range: 76%–99% vs 30%–96% | Unclear | More or fewer percent is not better or worse by itself; this row describes behaviour, not a winner. | How much of an AI bill is thinking? Reasoning tokens by model and effort |
| Reasoning share on short tasks vs hard tasks (calculation) (Five short tasks)Calculation | 90.2%Claude Code | 0%Claude Code | 15 | range: 73%–98% vs 0%–93% | Unclear | More or fewer percent is not better or worse by itself; this row describes behaviour, not a winner. | How much of an AI bill is thinking? Reasoning tokens by model and effort |
| Time to first text: a 250-line answer, six models | 4.00 sClaude Code | 1.97 sClaude Code | 4 | range: 2.8 s–6.4 s vs 1.7 s–2.4 s | Unclear | Only 4 runs per side; the run ranges (fastest to slowest) do not overlap (Claude Haiku 4.5 2.84 s to 6.38 s; Claude Opus 5.5 1.70 s to 2.35 s), but 4 runs cannot show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Output speed after the first text: visible tokens per second (calculation)Calculation | 153Claude Code | 156Claude Code | 4 | range: 153–216 vs 155–156 | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Output speed in characters per second after the first text (calculation)Calculation | 547Claude Code | 347Claude Code | 3 / 4 | range: 546–548 vs 345–349 | Unclear | More or fewer count is not better or worse by itself; this row describes behaviour, not a winner. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Time to first text as the prompt grows: 1kCalculation | 1.93 sClaude Code | 1.51 sClaude Code | 3 | range: 1.9 s–2 s vs 1.5 s–2 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 1.85 s to 2.04 s; Claude Opus 5.5 1.46 s to 2.01 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Time to first text as the prompt grows: 16kCalculation | 2.27 sClaude Code | 1.74 sClaude Code | 3 | range: 2.2 s–2.5 s vs 1.7 s–3 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.22 s to 2.47 s; Claude Opus 5.5 1.70 s to 2.97 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Time to first text as the prompt grows: 64kCalculation | 2.78 sClaude Code | 1.79 sClaude Code | 3 | range: 2.5 s–2.9 s vs 1.7 s–3.7 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.45 s to 2.89 s; Claude Opus 5.5 1.72 s to 3.72 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Total time per call by prompt size (1k prompt) | 2.34 sClaude Code | 1.83 sClaude Code | 3 | range: 2.2 s–2.5 s vs 1.8 s–2.4 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.22 s to 2.46 s; Claude Opus 5.5 1.82 s to 2.41 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Total time per call by prompt size (16k prompt) | 2.79 sClaude Code | 2.36 sClaude Code | 3 | range: 2.6 s–2.8 s vs 2.1 s–3.4 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.58 s to 2.84 s; Claude Opus 5.5 2.11 s to 3.40 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Total time per call by prompt size (64k prompt) | 3.13 sClaude Code | 2.35 sClaude Code | 3 | range: 2.8 s–3.3 s vs 2.3 s–4.3 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 2.84 s to 3.28 s; Claude Opus 5.5 2.26 s to 4.29 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Exact lookup answers at the 1k, 16k and 64k prompt-size targets | 100% (9/9)Claude Code | 56% (5/9)Claude Code | 9 | 95% CI: 70%–100% vs 27%–81% | Tie | The 95% intervals overlap (Claude Haiku 4.5 70% to 100%; Claude Opus 5.5 27% to 81%), so this sample cannot separate them. | Where the seconds go: first text, output speed and prompt size for 6 LLMs |
| Pass rate on 4 harder tasks (Strict pass) | 0% (0/12)Claude Code | 42% (5/12)Claude Code | 12 | 95% CI: 0%–24% vs 19%–68% | Tie | The 95% intervals overlap (Claude Haiku 4.5 0% to 24%; Claude Opus 5.5 19% to 68%), so this sample cannot separate them. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Pass rate on 4 harder tasks (Lenient (format misses counted)) | 0% (0/12)Claude Code | 50% (6/12)Claude Code | 12 | 95% CI: 0%–24% vs 25%–75% | Claude Opus 5.5 ahead | The 95% intervals do not overlap (Claude Haiku 4.5 0% to 24%; Claude Opus 5.5 25% to 75%). | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Calls that tried a tool although tools were off | 8% (1/12)Claude Code | 42% (5/12)Claude Code | 12 | 95% CI: 1.5%–35% vs 19%–68% | Unclear | More or fewer rate is not better or worse by itself; this row describes behaviour, not a winner. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Strict pass rate by task: 10x10 nonogram | 0% (0/3)Claude Code | 100% (3/3)Claude Code | 3 | 95% CI: 0%–56% vs 44%–100% | Tie | The 95% intervals overlap (Claude Haiku 4.5 0% to 56%; Claude Opus 5.5 44% to 100%), so this sample cannot separate them. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Strict pass rate by task: Sudoku, 22 givens | 0% (0/3)Claude Code | 0% (0/3)Claude Code | 3 | 95% CI: 0%–56% vs 0%–56% | Tie | The 95% intervals overlap (Claude Haiku 4.5 0% to 56%; Claude Opus 5.5 0% to 56%), so this sample cannot separate them. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Strict pass rate by task: 6x6 Skyscrapers | 0% (0/3)Claude Code | 33% (1/3)Claude Code | 3 | 95% CI: 0%–56% vs 6.2%–79% | Tie | The 95% intervals overlap (Claude Haiku 4.5 0% to 56%; Claude Opus 5.5 6% to 79%), so this sample cannot separate them. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Strict pass rate by task: Seeded shuffle output | 0% (0/3)Claude Code | 33% (1/3)Claude Code | 3 | 95% CI: 0%–56% vs 6.2%–79% | Tie | The 95% intervals overlap (Claude Haiku 4.5 0% to 56%; Claude Opus 5.5 6% to 79%), so this sample cannot separate them. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Total time per call on harder tasks | 109.0 sClaude Code | 80.3 sClaude Code | 10 / 9 | range: 25.7 s–224 s vs 3.8 s–280 s | Unclear | The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 25.7 s to 223.9 s; Claude Opus 5.5 3.82 s to 279.5 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
| Output tokens per call on harder tasks (Output tokens) | 12,508Claude Code | 8,420Claude Code | 10 / 9 | range: 2,965–26,532 vs 279–40,044 | Unclear | More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner. | GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks |
Marks: 95% intervals (Wilson for rates); fastest–slowest run ranges (not intervals)n is shown per side on every rowHollow marks: list-price calculations, not runs
39 rows from 5 studies. Claude Opus 5.5 ahead on 3; 7 ties, 29 unclear. A side is ahead only where the intervals or ranges do not overlap.
When to pick which
Only from the rows above. A tie is not a reason to pick either side.
When to pick Claude Haiku 4.5
- List-price cost per call: reasoning, remaining output and input (calculation) (Remaining output (visible-answer estimate)): $0.0018 vs $0.0080. A list-price calculation, not a measured difference. Calculation
- List-price cost per call: reasoning, remaining output and input (calculation) (Input (prompt, cache priced)): $0.0045 vs $0.0077. A list-price calculation, not a measured difference. Calculation
When to pick Claude Opus 5.5
- Pass rate on eight hard tasks (Strict pass): 100% (24/24) vs 46% (11/24). The 95% intervals do not overlap (Claude Haiku 4.5 28% to 65%; Claude Opus 5.5 86% to 100%).
- Pass rate on eight hard tasks (Lenient (format misses counted)): 100% (24/24) vs 67% (16/24). The 95% intervals do not overlap (Claude Haiku 4.5 47% to 82%; Claude Opus 5.5 86% to 100%).
- List-price cost per strict pass on hard tasks (calculation): $0.028 vs $0.067. A list-price calculation, not a measured difference. Calculation
- List-price cost per call: reasoning, remaining output and input (calculation) (Reasoning (output tokens)): $0.013 vs $0.024. A list-price calculation, not a measured difference. Calculation
- Time to first text as the prompt grows: 64k: 1.79 s vs 2.78 s. A list-price calculation, not a measured difference. Calculation
- Pass rate on 4 harder tasks (Lenient (format misses counted)): 50% (6/12) vs 0% (0/12). The 95% intervals do not overlap (Claude Haiku 4.5 0% to 24%; Claude Opus 5.5 25% to 75%).
Side by side
The study charts, showing only these two. Open a study for every configuration.
| Item | Pass rate | 95% interval | n |
|---|---|---|---|
| Claude Opus 5.5 (high) · Claude Code | 100% | 80%–100% | 15 |
| Claude Haiku 4.5 · Claude Code | 100% | 80%–100% | 15 |
2 rows. All at 100%.
NotesWhiskers: 95% Wilson intervaln = 15 per row
Every call counts; failures and timeouts are non-passes
Whiskers are 95% Wilson intervals. The tasks are short and most configurations pass nearly all of them, so pass rate does not separate the models here.
Source: Provider head-to-head: Claude Code models vs Codex efforts
Every lane’s run range overlaps another lane’s, so the chart shows no finish order.
| Item | Total time per call | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 2.8 s | 2.5 s–8.9 s | 15 |
| Claude Haiku 4.5 · Claude Code | 4.4 s | 3.2 s–23.6 s | 15 |
2 rows. Slowest Claude Haiku 4.5 · Claude Code 4.4 s (range 3.2 s–23.6 s, n 15). Fastest Claude Opus 5.5 · Claude Code 2.8 s (range 2.5 s–8.9 s, n 15). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 15 per row
Median per configuration; whiskers = fastest and slowest call
One host, one network, one day. Whiskers are a range, not a confidence interval.
Source: Provider head-to-head: Claude Code models vs Codex efforts
Every lane’s run range overlaps another lane’s, so the chart shows no finish order.
| Item | Time to first useful output | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 1.9 s | 1.6 s–7.2 s | 15 |
| Claude Haiku 4.5 · Claude Code | 3.6 s | 2.8 s–22.3 s | 15 |
2 rows. Slowest Claude Haiku 4.5 · Claude Code 3.6 s (range 2.8 s–22.3 s, n 15). Fastest Claude Opus 5.5 · Claude Code 1.9 s (range 1.6 s–7.2 s, n 15). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 15 per row
Median per configuration; whiskers = fastest and slowest call
One host, one network, one day. Whiskers are a range, not a confidence interval.
Source: Provider head-to-head: Claude Code models vs Codex efforts
- Cache read
- Other input
Totals are the sum of the parts shown. Shares are calculated from the same values.
| Item | Cache read | Other input | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 1,401 | 680 | 15 |
| Claude Haiku 4.5 · Claude Code | 0 | 3,790 | 15 |
2 rows, 2 series: Cache read, Other input. Cache read: highest Claude Opus 5.5 · Claude Code 1,401 (n 15). Lowest Claude Haiku 4.5 · Claude Code 0 (n 15). Other input: highest Claude Haiku 4.5 · Claude Code 3,790 (n 15). Lowest Claude Opus 5.5 · Claude Code 680 (n 15).
Notesn = 15 per row
Mean per call, split into prompt-cache reads and other input
The prompts are a few hundred tokens; most input is the CLI’s own system prompt and tool context. Input counts include cache reads, as the vendors report them.
Source: Provider head-to-head: Claude Code models vs Codex efforts
- Output tokens
- of which reasoning tokens (inner bar)
| Item | Output tokens | Reasoning tokens | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 64 | 0 | 15 |
| Claude Haiku 4.5 · Claude Code | 367 | 297 | 15 |
2 rows, 2 series: Output tokens, Reasoning tokens. Output tokens: highest Claude Haiku 4.5 · Claude Code 367 (n 15). Lowest Claude Opus 5.5 · Claude Code 64 (n 15). Reasoning tokens: highest Claude Haiku 4.5 · Claude Code 297 (n 15). Lowest Claude Opus 5.5 · Claude Code 0 (n 15).
Notesn = 15 per row
Median per configuration; reasoning tokens where the CLI reports them
Vendors count reasoning differently; compare within a vendor first. A median of 0 can mean the CLI did not report reasoning for most calls.
Source: Provider head-to-head: Claude Code models vs Codex efforts
| Item | Cost per call | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | $0.0069 | $0.0059–$0.022 | 15 |
| Claude Haiku 4.5 · Claude Code | $0.0057 | $0.0051–$0.018 | 15 |
List-price calculation, not a run. 2 rows. Highest Claude Opus 5.5 · Claude Code $0.0069 (range $0.0059–$0.022, n 15). Lowest Claude Haiku 4.5 · Claude Code $0.0057 (range $0.0051–$0.018, n 15). All run ranges overlap.
NotesLines: lowest–highest run (not an interval)n = 15 per row
Reported tokens × list price; the calls ran on subscriptions
Calculation, not a bill: the calls ran on flat subscriptions. Whiskers = cheapest and most expensive call.
Sources: Provider head-to-head: Claude Code models vs Codex efforts, Repricing calculation, Anthropic list prices (Claude models), OpenAI list prices
| Item | Cost per pass | n |
|---|---|---|
| Claude Haiku 4.5 · Claude Code | $0.0084 | 15 |
| Claude Opus 5.5 · Claude Code | $0.01 | 15 |
List-price calculation, not a run. 2 rows. Highest Claude Opus 5.5 · Claude Code $0.01 (n 15). Lowest Claude Haiku 4.5 · Claude Code $0.0084 (n 15).
Notesn = 15 per row
All calls in a configuration, failures included, divided by its passes
Calculation, not a bill: reported tokens × list price; the calls ran on flat subscriptions. A failed call still costs, so a lower pass rate raises the cost per pass. Highlighted bars are on the speed/cost/quality frontier.
Sources: Provider head-to-head: Claude Code models vs Codex efforts, Repricing calculation, Anthropic list prices (Claude models), OpenAI list prices
- Strict pass
- Lenient (format misses counted)
| Item | Strict pass | Lenient (format misses counted) | 95% interval | n |
|---|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 100% | 100% | Strict pass: 86%–100%; Lenient (format misses counted): 86%–100% | 24 |
| Claude Haiku 4.5 · Claude Code | 46% | 67% | Strict pass: 28%–65%; Lenient (format misses counted): 47%–82% | 24 |
2 rows, 2 series: Strict pass, Lenient (format misses counted). Strict pass: highest Claude Opus 5.5 · Claude Code 100% (95% interval 86%–100%, n 24). Lowest Claude Haiku 4.5 · Claude Code 46% (95% interval 28%–65%, n 24). Not all intervals overlap. Lenient (format misses counted): highest Claude Opus 5.5 · Claude Code 100% (95% interval 86%–100%, n 24). Lowest Claude Haiku 4.5 · Claude Code 67% (95% interval 47%–82%, n 24). Not all intervals overlap.
NotesWhiskers: 95% Wilson intervaln = 24 per row
Strict: the reply passes as given. Lenient: a correct answer in the wrong format also counts
Whiskers are 95% Wilson intervals. The strict pass is the result; the lenient reading is shown so a reader can see how much is format and how much is a wrong answer. A format miss never counts as a pass.
Source: Provider head-to-head, hard set: eight hard tasks with strict validators
Every lane’s run range overlaps another lane’s, so the chart shows no finish order.
| Item | Total time per call on hard tasks (separate batches) | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 9.2 s | 4.2 s–27.2 s | 24 |
| Claude Haiku 4.5 · Claude Code | 39 s | 15.3 s–75.1 s | 24 |
2 rows. Slowest Claude Haiku 4.5 · Claude Code 39 s (range 15.3 s–75.1 s, n 24). Fastest Claude Opus 5.5 · Claude Code 9.2 s (range 4.2 s–27.2 s, n 24). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 24 per row
Median per configuration; whiskers = fastest and slowest call
One host and network; the counted Claude and Codex batches ran hours apart. Host load was not controlled. Whiskers are a range, not a confidence interval. Highlighted: configurations that passed every call.
Source: Provider head-to-head, hard set: eight hard tasks with strict validators
Every lane’s run range overlaps another lane’s, so the chart shows no finish order.
| Item | Time to first useful output on hard tasks | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 6.8 s | 2.4 s–21.8 s | 24 |
| Claude Haiku 4.5 · Claude Code | 35.5 s | 12.9 s–70.3 s | 24 |
2 rows. Slowest Claude Haiku 4.5 · Claude Code 35.5 s (range 12.9 s–70.3 s, n 24). Fastest Claude Opus 5.5 · Claude Code 6.8 s (range 2.4 s–21.8 s, n 24). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 24 per row
Median per configuration; whiskers = fastest and slowest call
One host and network; the counted Claude and Codex batches ran hours apart. Host load was not controlled. Whiskers are a range, not a confidence interval. Highlighted: configurations that passed every call.
Source: Provider head-to-head, hard set: eight hard tasks with strict validators
- Output tokens
- of which reasoning tokens (inner bar)
| Item | Output tokens | Reasoning tokens | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 945 | 529 | 24 |
| Claude Haiku 4.5 · Claude Code | 5,064 | 4,556 | 24 |
2 rows, 2 series: Output tokens, Reasoning tokens. Output tokens: highest Claude Haiku 4.5 · Claude Code 5,064 (n 24). Lowest Claude Opus 5.5 · Claude Code 945 (n 24). Reasoning tokens: highest Claude Haiku 4.5 · Claude Code 4,556 (n 24). Lowest Claude Opus 5.5 · Claude Code 529 (n 24).
Notesn = 24 per row
Median per configuration; reasoning tokens as the CLI reports them
Reasoning tokens are part of the output tokens where the CLI reports them. Their content is never captured. More tokens is not better or worse by itself.
Source: Provider head-to-head, hard set: eight hard tasks with strict validators
| Item | Cost per strict pass | n |
|---|---|---|
| Claude Opus 5.5 · Claude Code | $0.028 | 24 |
| Claude Haiku 4.5 · Claude Code | $0.067 | 24 |
List-price calculation, not a run. 2 rows. Highest Claude Haiku 4.5 · Claude Code $0.067 (n 24). Lowest Claude Opus 5.5 · Claude Code $0.028 (n 24).
Notesn = 24 per row
All calls in a configuration, failures and format misses included, divided by its strict passes
Calculation, not a bill: reported tokens × list price; the calls ran on a flat subscription. A failed call still costs, so a lower pass rate raises the cost per pass. Highlighted bars are on the quality-vs-cost frontier.
Sources: Provider head-to-head, hard set: eight hard tasks with strict validators, Repricing calculation, Anthropic list prices (Claude models), OpenAI list prices
- Reasoning (output tokens)
- Remaining output (visible-answer estimate)
- Input (prompt, cache priced)
Totals are the sum of the parts shown. Shares are calculated from the same values.
| Item | Reasoning (output tokens) | Remaining output (visible-answer estimate) | Input (prompt, cache priced) | n |
|---|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | $0.024 | $0.0018 | $0.0045 | 24 |
| Claude Opus 5.5 · Claude Code | $0.013 | $0.008 | $0.0077 | 24 |
List-price calculation, not a run. 2 rows, 3 series: Reasoning (output tokens), Remaining output (visible-answer estimate), Input (prompt, cache priced). Reasoning (output tokens): highest Claude Haiku 4.5 · Claude Code $0.024 (n 24). Lowest Claude Opus 5.5 · Claude Code $0.013 (n 24). Remaining output (visible-answer estimate): highest Claude Opus 5.5 · Claude Code $0.008 (n 24). Lowest Claude Haiku 4.5 · Claude Code $0.0018 (n 24).
Notesn = 24 per row
Mean per call on the hard tasks; the three parts add up to the call
Calculation, not a bill: reported tokens × list price (per million output tokens: Haiku 4.5 $5, Sonnet 5.5 $10, Opus 5.5 $20, Fable 5.1 $50 and GPT-6.1 Sol $10); the calls ran on flat subscriptions. Reasoning and remaining output both use the output price. The remainder estimates visible-answer tokens; its exact meaning depends on the CLI counters. Input is the whole prompt, with cache reads and writes priced as in the hard head-to-head. It includes CLI context; these receipts do not separate task tokens from CLI context. It changes with cache counters. Batch timing and cache behavior were not controlled. Means, not medians, so the parts add up.
Sources: Reasoning token bill (calculation), Provider head-to-head, hard set: eight hard tasks with strict validators, Effort ladder: the hard task set at each effort level, Provider head-to-head: Claude Code models vs Codex efforts, Anthropic list prices (Claude models), OpenAI list prices
- Eight hard tasks
- Five short tasks (square)
Gap labels, Five short tasks vs Eight hard tasks: Five short tasks is x percentage points higher (+) or lower (−) than Eight hard tasks, calculated from the two values shown; lines are the lowest–highest run (not an interval).
| Item | Eight hard tasks | Five short tasks | Range (lowest–highest run) | n |
|---|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | 92% | 90% | Eight hard tasks: 76%–99%; Five short tasks: 73%–98% | 24 |
| Claude Opus 5.5 · Claude Code | 55% | 0% | Eight hard tasks: 30%–96%; Five short tasks: 0%–93% | 24 |
List-price calculation, not a run. 2 rows, 2 series: Eight hard tasks, Five short tasks. Eight hard tasks: highest Claude Haiku 4.5 · Claude Code 92% (range 76%–99%, n 24). Lowest Claude Opus 5.5 · Claude Code 55% (range 30%–96%, n 24). All run ranges overlap. Five short tasks: highest Claude Haiku 4.5 · Claude Code 90% (range 73%–98%, n 15). Lowest Claude Opus 5.5 · Claude Code 0% (range 0%–93%, n 15). All run ranges overlap.
NotesLines: lowest–highest run (not an interval)n 15–24 per row
Median call per configuration; five short tasks and eight hard tasks
Calculation from reported tokens, not a run: the median of per-call reasoning ÷ output. A median of 0% means at least half of the calls reported 0 reasoning tokens in the receipt. On a route that reports reasoning, we retain a numeric 0 as the recorded counter. It does not prove the model did no internal reasoning. The table says how many calls reported 0. The short tasks are five small validated tasks (a bug fix, a JSON extraction, an arithmetic problem, a refactor and a ticket classification). Per-call ranges are in the tables.
Sources: Reasoning token bill (calculation), Provider head-to-head, hard set: eight hard tasks with strict validators, Effort ladder: the hard task set at each effort level, Provider head-to-head: Claude Code models vs Codex efforts, Anthropic list prices (Claude models), OpenAI list prices
1st, 2nd …: place by median, given only to a lane whose run range overlaps no other lane’s. A lane marked ~ overlaps another lane’s range, so it gets no place.
| Item | Time to first text | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | 4 s | 2.8 s–6.4 s | 4 |
| Claude Opus 5.5 · Claude Code | 2 s | 1.7 s–2.4 s | 4 |
2 rows. Slowest Claude Haiku 4.5 · Claude Code 4 s (range 2.8 s–6.4 s, n 4). Fastest Claude Opus 5.5 · Claude Code 2 s (range 1.7 s–2.4 s, n 4). Not all run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 4 per row
Median of 4 calls per model; whiskers = fastest and slowest call
Dot = median; whiskers = fastest and slowest call (a range, not a confidence interval). The clock starts when the CLI starts, so first text includes CLI start-up (see chart cli-startup-tax) and any reasoning before the first word. One Mac, one network, two sessions on one night.
Source: LLM speed anatomy
| Item | Visible tokens per second | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | 153 | 153–216 | 4 |
| Claude Opus 5.5 · Claude Code | 156 | 155–156 | 4 |
List-price calculation, not a run. 2 rows. Highest Claude Opus 5.5 · Claude Code 156 (range 155–156, n 4). Lowest Claude Haiku 4.5 · Claude Code 153 (range 153–216, n 4). All run ranges overlap.
NotesLines: lowest–highest run (not an interval)n = 4 per row
Median of 4 calls per model; whiskers = slowest and fastest call
Calculation, not a measurement: visible output tokens (the CLI’s output token count minus its reported reasoning tokens) divided by the time from the first text to the end of the call. The denominator includes the CLI’s exit overhead; its size is not measured here. Whiskers are a range of calls, not a confidence interval. The calculation removes reported reasoning tokens. These receipts do not locate all reasoning in time.
Source: LLM speed anatomy
| Item | Characters per second | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | 547 | 546–548 | 3 |
| Claude Opus 5.5 · Claude Code | 347 | 345–349 | 4 |
List-price calculation, not a run. 2 rows. Highest Claude Haiku 4.5 · Claude Code 547 (range 546–548, n 3). Lowest Claude Opus 5.5 · Claude Code 347 (range 345–349, n 4). Not all run ranges overlap.
NotesLines: lowest–highest run (not an interval)n 3–4 per row
Only replies that matched all 250 lines; median per model, whiskers = slowest and fastest call
Calculation, not a measurement: the characters of a correct reply (the same text for every model) divided by the time from the first text to the end of the call. Each vendor counts the same text as a different number of tokens, so characters compare across models where tokens do not. The CLI’s exit time is inside that time. Whiskers are a range of calls, not a confidence interval.
Source: LLM speed anatomy
- Claude Haiku 4.5 · Claude Code
- Claude Sonnet 5.5 · Claude Code
- Claude Opus 5.5 · Claude Code
- GPT-6.1 Sol (low) · Codex CLI
| Prompt-size target (approximate Haiku tokens; calibration calculation) | Claude Haiku 4.5 · Claude Code | Claude Sonnet 5.5 · Claude Code | Claude Opus 5.5 · Claude Code | GPT-6.1 Sol (low) · Codex CLI | Range (lowest–highest run) | n |
|---|---|---|---|---|---|---|
| 1k | 1.9 s | 1.5 s | 1.5 s | 3.4 s | Claude Haiku 4.5 · Claude Code: 1.9 s–2 s; Claude Sonnet 5.5 · Claude Code: 1.2 s–1.7 s; Claude Opus 5.5 · Claude Code: 1.5 s–2 s; GPT-6.1 Sol (low) · Codex CLI: 3.4 s–4.8 s | 3 |
| 16k | 2.3 s | 1.8 s | 1.7 s | 4 s | Claude Haiku 4.5 · Claude Code: 2.2 s–2.5 s; Claude Sonnet 5.5 · Claude Code: 1.6 s–2.1 s; Claude Opus 5.5 · Claude Code: 1.7 s–3 s; GPT-6.1 Sol (low) · Codex CLI: 3.3 s–4.3 s | 3 |
| 64k | 2.8 s | 3.1 s | 1.8 s | 3.9 s | Claude Haiku 4.5 · Claude Code: 2.5 s–2.9 s; Claude Sonnet 5.5 · Claude Code: 1.4 s–3.6 s; Claude Opus 5.5 · Claude Code: 1.7 s–3.7 s; GPT-6.1 Sol (low) · Codex CLI: 3.4 s–4.4 s | 3 |
List-price calculation, not a run. 3 rows, 4 series: Claude Haiku 4.5 · Claude Code, Claude Sonnet 5.5 · Claude Code, Claude Opus 5.5 · Claude Code, GPT-6.1 Sol (low) · Codex CLI. Claude Haiku 4.5 · Claude Code: slowest 64k 2.8 s (range 2.5 s–2.9 s, n 3). Fastest 1k 1.9 s (range 1.9 s–2 s, n 3). Not all run ranges overlap. Claude Sonnet 5.5 · Claude Code: slowest 64k 3.1 s (range 1.4 s–3.6 s, n 3). Fastest 1k 1.5 s (range 1.2 s–1.7 s, n 3). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 3 per row
Median of 3 calls per size; whiskers = fastest and slowest call
Each call used a new ledger seed. Cache-read counts stayed within the short-prompt baseline (see the cache table). This does not identify which tokens were cached. Sizes name the text we send; each model’s reported input tokens are in the table and include the CLI’s own prefix. The size calibration subtracts estimated prefixes from probe input counts; these are calculations, not measured prefix counts for each call. Whiskers are a range of calls, not a confidence interval.
Source: LLM speed anatomy
1k prompt
16k prompt
64k prompt
One panel per series, all on the same axis; whiskers are the fastest–slowest run (not an interval).
| Item | 1k prompt | 16k prompt | 64k prompt | Range (lowest–highest run) | n |
|---|---|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | 2.3 s | 2.8 s | 3.1 s | 1k prompt: 2.2 s–2.5 s; 16k prompt: 2.6 s–2.8 s; 64k prompt: 2.8 s–3.3 s | 3 |
| Claude Opus 5.5 · Claude Code | 1.8 s | 2.4 s | 2.4 s | 1k prompt: 1.8 s–2.4 s; 16k prompt: 2.1 s–3.4 s; 64k prompt: 2.3 s–4.3 s | 3 |
2 rows, 3 series: 1k prompt, 16k prompt, 64k prompt. 1k prompt: slowest Claude Haiku 4.5 · Claude Code 2.3 s (range 2.2 s–2.5 s, n 3). Fastest Claude Opus 5.5 · Claude Code 1.8 s (range 1.8 s–2.4 s, n 3). All run ranges overlap. 16k prompt: slowest Claude Haiku 4.5 · Claude Code 2.8 s (range 2.6 s–2.8 s, n 3). Fastest Claude Opus 5.5 · Claude Code 2.4 s (range 2.1 s–3.4 s, n 3). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 3 per row
Median of 3 calls per bar; whiskers = fastest and slowest call
Whole call: CLI start-up, first text and the one-line answer. Whiskers are a range of calls, not a confidence interval. Each call used a new ledger.
Source: LLM speed anatomy
| Item | Exact answer | 95% interval | n |
|---|---|---|---|
| Claude Haiku 4.5 · Claude Code | 100% | 70%–100% | 9 |
| Claude Opus 5.5 · Claude Code | 56% | 27%–81% | 9 |
2 rows. Highest Claude Haiku 4.5 · Claude Code 100% (95% interval 70%–100%, n 9). Lowest Claude Opus 5.5 · Claude Code 56% (95% interval 27%–81%, n 9). All intervals overlap.
NotesWhiskers: 95% Wilson intervaln = 9 per row
All sizes together per model; whiskers = 95% Wilson intervals
Whiskers are 95% Wilson intervals. One lookup question per call; a reply with extra words is a format miss, not a pass. With 9 calls per model, a perfect score still has a wide interval.
Source: LLM speed anatomy
- Strict pass
- Lenient (format misses counted)
| Item | Strict pass | Lenient (format misses counted) | 95% interval | n |
|---|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 42% | 50% | Strict pass: 19%–68%; Lenient (format misses counted): 25%–75% | 12 |
| Claude Haiku 4.5 · Claude Code | 0% | 0% | Strict pass: 0%–24%; Lenient (format misses counted): 0%–24% | 12 |
2 rows, 2 series: Strict pass, Lenient (format misses counted). Strict pass: highest Claude Opus 5.5 · Claude Code 42% (95% interval 19%–68%, n 12). Lowest Claude Haiku 4.5 · Claude Code 0% (95% interval 0%–24%, n 12). All intervals overlap. Lenient (format misses counted): highest Claude Opus 5.5 · Claude Code 50% (95% interval 25%–75%, n 12). Lowest Claude Haiku 4.5 · Claude Code 0% (95% interval 0%–24%, n 12). Not all intervals overlap.
NotesWhiskers: 95% Wilson intervaln = 12 per row
Strict: the reply passes as given. Lenient: a correct answer in the wrong format also counts
Whiskers are 95% Wilson intervals. Strict passes decide the result. The lenient reading shows which failures contain a correct answer in the wrong format. A format miss never counts as a pass. The study picked tasks that Sonnet did not pass twice in a pilot. This selection can lower a Sonnet row. Counted calls are new calls.
Source: Harder tasks head-to-head
| Item | Tool attempt | 95% interval | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 42% | 19%–68% | 12 |
| Claude Haiku 4.5 · Claude Code | 8.3% | 1.5%–35% | 12 |
2 rows. Highest Claude Opus 5.5 · Claude Code 42% (95% interval 19%–68%, n 12). Lowest Claude Haiku 4.5 · Claude Code 8.3% (95% interval 1.5%–35%, n 12). All intervals overlap.
NotesWhiskers: 95% Wilson intervaln = 12 per row
Share of calls whose reply held tool-call markup or whose tool call the CLI could not parse
Every prompt asked for the answer only. Both CLIs disabled tools. The Codex runner adds a developer instruction not to call tools. The Claude Code runner adds no such instruction. The validator decides the score. Tool-call markup is a separate flag; a flagged call can still pass if its whole reply meets the validator. It is a behaviour, not a quality score: more is not better. No call with a tool attempt passed. Whiskers are 95% Wilson intervals. The flag comes from the reply text, which is not published.
Source: Harder tasks head-to-head
GPT-6.1 Sol (medium) · Codex CLI
Claude Opus 5.5 · Claude Code
Claude Sonnet 5.5 · Claude Code
Claude Haiku 4.5 · Claude Code
One panel per series, all on the same axis; whiskers are the 95% Wilson interval.
| Item | GPT-6.1 Sol (medium) · Codex CLI | Claude Opus 5.5 · Claude Code | Claude Sonnet 5.5 · Claude Code | Claude Haiku 4.5 · Claude Code | 95% interval | n |
|---|---|---|---|---|---|---|
| 10x10 nonogram | 75% | 100% | 100% | 0% | GPT-6.1 Sol (medium) · Codex CLI: 30%–95%; Claude Opus 5.5 · Claude Code: 44%–100%; Claude Sonnet 5.5 · Claude Code: 51%–100%; Claude Haiku 4.5 · Claude Code: 0%–56% | 4 |
| Sudoku, 22 givens | 25% | 0% | 0% | 0% | GPT-6.1 Sol (medium) · Codex CLI: 4.6%–70%; Claude Opus 5.5 · Claude Code: 0%–56%; Claude Sonnet 5.5 · Claude Code: 0%–49%; Claude Haiku 4.5 · Claude Code: 0%–56% | 4 |
| 6x6 Skyscrapers | 100% | 33% | 0% | 0% | GPT-6.1 Sol (medium) · Codex CLI: 51%–100%; Claude Opus 5.5 · Claude Code: 6.2%–79%; Claude Sonnet 5.5 · Claude Code: 0%–49%; Claude Haiku 4.5 · Claude Code: 0%–56% | 4 |
3 rows, 4 series: GPT-6.1 Sol (medium) · Codex CLI, Claude Opus 5.5 · Claude Code, Claude Sonnet 5.5 · Claude Code, Claude Haiku 4.5 · Claude Code. GPT-6.1 Sol (medium) · Codex CLI: highest 6x6 Skyscrapers 100% (95% interval 51%–100%, n 4). Lowest Sudoku, 22 givens 25% (95% interval 4.6%–70%, n 4). All intervals overlap. Claude Opus 5.5 · Claude Code: highest 10x10 nonogram 100% (95% interval 44%–100%, n 3). Lowest Sudoku, 22 givens 0% (95% interval 0%–56%, n 3). All intervals overlap.
NotesWhiskers: 95% Wilson intervaln 3–4 per row
One bar per configuration and task; each bar rests on only a few calls, so the 95% intervals are wide
Each bar is 3 to 4 calls, so one call moves a bar by a quarter or a third. Whiskers are 95% Wilson intervals; with this few calls they overlap almost everywhere.
Source: Harder tasks head-to-head
Every lane’s run range overlaps another lane’s, so the chart shows no finish order.
| Item | Total time per call | Range (lowest–highest run) | n |
|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 80.3 s | 3.8 s–280 s | 9 |
| Claude Haiku 4.5 · Claude Code | 109 s | 25.7 s–224 s | 10 |
2 rows. Slowest Claude Haiku 4.5 · Claude Code 109 s (range 25.7 s–224 s, n 10). Fastest Claude Opus 5.5 · Claude Code 80.3 s (range 3.8 s–280 s, n 9). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n 9–10 per row
Median per configuration; whiskers = fastest and slowest call
Median and range over the calls that completed. Completed calls include wrong answers and format misses. Only timeouts and tool-call parse errors are excluded from this run’s timings. Both count as non-passes in the outcomes chart. One Mac, one network, one session. Arena servers shared the Mac during part of the run. Host load was not controlled, so these times cannot isolate model speed. Whiskers are a range, not a confidence interval. Times include the CLI start-up and the CLI’s own system prompt. Highlighted: configurations that passed every call.
Source: Harder tasks head-to-head
- Output tokens
- of which reasoning tokens (inner bar)
| Item | Output tokens | Reasoning tokens | Range (lowest–highest run) | n |
|---|---|---|---|---|
| Claude Opus 5.5 · Claude Code | 8,420 | 8,352 | Output tokens: 279–40,044; Reasoning tokens: 21–9,897 | 9 |
| Claude Haiku 4.5 · Claude Code | 12,508 | 12,483 | Output tokens: 2,965–26,532; Reasoning tokens: 2,924–26,510 | 10 |
2 rows, 2 series: Output tokens, Reasoning tokens. Output tokens: highest Claude Haiku 4.5 · Claude Code 12,508 (range 2,965–26,532, n 10). Lowest Claude Opus 5.5 · Claude Code 8,420 (range 279–40,044, n 9). All run ranges overlap. Reasoning tokens: highest Claude Haiku 4.5 · Claude Code 12,483 (range 2,924–26,510, n 10). Lowest Claude Opus 5.5 · Claude Code 8,352 (range 21–9,897, n 9). All run ranges overlap.
NotesLines: lowest–highest run (not an interval)n 9–10 per row
Median per configuration; reasoning tokens as the CLI reports them
Medians and minimum-to-maximum token ranges cover completed calls only. Ranges are not confidence intervals. The chart omits unknown reasoning counts. Reasoning tokens are part of the output tokens where the CLI reports them. Their content is never captured. Claude Code used an output-token cap setting of 16,000. Some reported totals exceeded it. Codex CLI had no cap. More tokens is not better or worse by itself.
Source: Harder tasks head-to-head
How a row is called
- AheadThe 95% intervals do not overlap, or the run ranges or p50–p95 bands do not overlap with at least 5 runs per side.
- TieThe values match, or both sit at the same ceiling.
- UnclearThe intervals or ranges overlap, too few runs were recorded, no interval was recorded, or more is not better (token counts are never a win).
- CalculationDerived from list prices and recorded counts. Not a bill and not a run.
The dataset compiler makes every call; this page only draws it. No composite score, no rank.
Questions
- Which is better, Claude Haiku 4.5 or Claude Opus 5.5?
- Claude Haiku 4.5 and Claude Opus 5.5 share 25 measured metrics and 14 list-price calculations from 5 studies. Claude Opus 5.5 leads on 3 rows: Pass rate on eight hard tasks (Strict pass), 100% (24/24) vs 46% (11/24); Pass rate on eight hard tasks (Lenient (format misses counted)), 100% (24/24) vs 67% (16/24); Pass rate on 4 harder tasks (Lenient (format misses counted)), 50% (6/12) vs 0% (0/12). On those rows the 95% intervals do not overlap. The other rows are 7 ties and 29 unclear; each row says why. Calculation rows are derived from list prices and recorded counts; they are not bills or runs. Some rows rest on small samples (n = 3 at the smallest).
- How were Claude Haiku 4.5 and Claude Opus 5.5 measured?
- They share 25 measured metrics and 14 list-price calculations from 5 public studies: Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head; Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks; How much of an AI bill is thinking? Reasoning tokens by model and effort; Where the seconds go: first text, output speed and prompt size for 6 LLMs; GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks. Every row names its configuration, its sample size and its interval or range.
- How do Claude Haiku 4.5 and Claude Opus 5.5 compare on pass rate on eight hard tasks (Strict pass)?
- Claude Haiku 4.5: 46% (11/24) (Claude Code · eight hard validated tasks; n = 24; 95% interval 28% to 65%). Claude Opus 5.5: 100% (24/24) (Claude Code · eight hard validated tasks; n = 24; 95% interval 86% to 100%). The 95% intervals do not overlap (Claude Haiku 4.5 28% to 65%; Claude Opus 5.5 86% to 100%).
- How do Claude Haiku 4.5 and Claude Opus 5.5 compare on pass rate on eight hard tasks (Lenient (format misses counted))?
- Claude Haiku 4.5: 67% (16/24) (Claude Code · eight hard validated tasks; n = 24; 95% interval 47% to 82%). Claude Opus 5.5: 100% (24/24) (Claude Code · eight hard validated tasks; n = 24; 95% interval 86% to 100%). The 95% intervals do not overlap (Claude Haiku 4.5 47% to 82%; Claude Opus 5.5 86% to 100%).
- How do Claude Haiku 4.5 and Claude Opus 5.5 compare on pass rate on 4 harder tasks (Lenient (format misses counted))?
- Claude Haiku 4.5: 0% (0/12) (Claude Code; n = 12; 95% interval 0% to 24%). Claude Opus 5.5: 50% (6/12) (Claude Code; n = 12; 95% interval 25% to 75%). The 95% intervals do not overlap (Claude Haiku 4.5 0% to 24%; Claude Opus 5.5 25% to 75%).
- How do Claude Haiku 4.5 and Claude Opus 5.5 compare on pass rate on five validated tasks?
- Claude Haiku 4.5: 100% (15/15) (Claude Code · five short validated tasks; n = 15; 95% interval 80% to 100%). Claude Opus 5.5: 100% (15/15) (Claude Code · five short validated tasks; n = 15; 95% interval 80% to 100%). The 95% intervals overlap (Claude Haiku 4.5 80% to 100%; Claude Opus 5.5 80% to 100%), so this sample cannot separate them.
- How do Claude Haiku 4.5 and Claude Opus 5.5 compare on total time per call?
- Claude Haiku 4.5: 4.43 s (Claude Code · five short validated tasks; n = 15; run range 3.2 s to 23.6 s). Claude Opus 5.5: 2.75 s (Claude Code · five short validated tasks; n = 15; run range 2.5 s to 8.9 s). The run ranges (fastest to slowest) overlap (Claude Haiku 4.5 3.16 s to 23.6 s; Claude Opus 5.5 2.47 s to 8.91 s); the medians alone do not show a reliable difference. A range is not a confidence interval.
The studies behind this page
Haiku vs Sonnet vs Opus vs Fable vs Codex: a timed head-to-head
130 timed calls on five validated tasks: Claude Haiku, Sonnet, Opus and Fable against Codex GPT-6.1 Sol. Pass rate, speed, tokens, cost per pass.
Haiku vs Sonnet vs Opus vs Fable vs GPT-6.1 Sol on 8 hard tasks
152 calls on 8 hard tasks with strict validators. Pass rate with 95% intervals, format misses, speed, tokens and cost per pass.
How much of an AI bill is thinking? Reasoning tokens by model and effort
Reasoning tokens in 378 recorded calls by model and effort: share of output, list-price cost per call and per pass, and time. Calculations.
Where the seconds go: first text, output speed and prompt size for 6 LLMs
60 calls on 6 models: time to first text, tokens per second, and what a 1k, 16k or 64k prompt adds. Claude Code and Codex CLI.
GPT-6.1 Sol vs Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 on 4 harder tasks
56 counted calls on 4 harder tasks with strict validators: GPT-6.1 Sol, Claude Opus 5.5, Sonnet 5.5, Haiku 4.5. Pass rate, intervals, speed, cost.