5 measured metrics · 1 study
GPT-6.1 Sol (OpenAI API)vsGPT-6 Luna (OpenAI API)
No row separates them: 1 tie, 4 unclear.
The verdict
GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) share 5 measured metrics from one study. No row separates them: every interval or run range overlaps, too few runs were recorded, no interval was recorded, or more is not better for that metric. The rows are 1 tie and 4 unclear; each row says why. Some rows rest on small samples (n = 3 at the smallest).
Headline metrics
How far apart the two sides are on the headline metrics. Length is the ratio of the two values; it is not a winner.
Bar length is the ratio of the two values on a log scale, pointing to the larger one. Larger is not better for time, tokens or cost. A bar has a side’s color only when that side is ahead in the data; gray means the data does not separate them.
| Metric | GPT-6.1 Sol (OpenAI API) | GPT-6 Luna (OpenAI API) | n | Interval or range | Outcome | Basis | Study |
|---|---|---|---|---|---|---|---|
| CLI vs API: time for a one-line answer (Total time) | 1.52 sOpenAI API · effort high · fixed exact reply, 5 runs | 0.97 sOpenAI API · effort none · fixed exact reply, 5 runs | 5 | range: 1.4 s–2.2 s vs 0.7 s–1.5 s | Unclear | The run ranges (fastest to slowest) overlap (GPT-6.1 Sol (OpenAI API) 1.35 s to 2.23 s; GPT-6 Luna (OpenAI API) 0.65 s to 1.50 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
| CLI vs API: time for a one-line answer (First useful output) | 1.34 sOpenAI API · effort high · fixed exact reply, 5 runs | 0.82 sOpenAI API · effort none · fixed exact reply, 5 runs | 5 | range: 1.3 s–2.1 s vs 0.5 s–1.4 s | Unclear | The run ranges (fastest to slowest) overlap (GPT-6.1 Sol (OpenAI API) 1.26 s to 2.12 s; GPT-6 Luna (OpenAI API) 0.51 s to 1.37 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
| Hidden prompt: input tokens for the same one-line request | 17OpenAI API · effort high · short fixed tasks | 17OpenAI API · effort none · short fixed tasks | 5 | none recorded | Tie | Same value. More or fewer is not better by itself for this metric. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
Marks: fastest–slowest run ranges (not intervals)n is shown per side on every row
3 headline metrics as the ratio of the two values. None of them separates the sides in the data. Widest ratio: CLI vs API: time for a one-line answer (First useful output), 1.6x (GPT-6.1 Sol (OpenAI API) larger).
Metric by metric
Both values of a row come from the same chart in the same study. Each row has its own axis. The shaded band is where the two intervals or ranges overlap: a side is ahead only when they do not.
- GPT-6.1 Sol (OpenAI API)
- GPT-6 Luna (OpenAI API)
- fastest–slowest run (not an interval)
- where the two overlap
Claude Code CLI vs Codex CLI vs the API: latency and tokens
- CLI vs API: time for a one-line answer (Total time)1.52 sn 50.97 sn 5UnclearCLI vs API: time for a one-line answer (Total time): GPT-6.1 Sol (OpenAI API) 1.52 s (n 5, run range 1.4 s–2.2 s); GPT-6 Luna (OpenAI API) 0.97 s (n 5, run range 0.7 s–1.5 s). Unclear.
- CLI vs API: time for a one-line answer (First useful output)1.34 sn 50.82 sn 5UnclearCLI vs API: time for a one-line answer (First useful output): GPT-6.1 Sol (OpenAI API) 1.34 s (n 5, run range 1.3 s–2.1 s); GPT-6 Luna (OpenAI API) 0.82 s (n 5, run range 0.5 s–1.4 s). Unclear.
- CLI vs API: time for a small coding task (Total time)9.56 sn 34.01 sn 3UnclearCLI vs API: time for a small coding task (Total time): GPT-6.1 Sol (OpenAI API) 9.56 s (n 3, run range 9.4 s–10.9 s); GPT-6 Luna (OpenAI API) 4.01 s (n 3, run range 3.8 s–4.4 s). Unclear.
- CLI vs API: time for a small coding task (First useful output)5.31 sn 30.67 sn 3UnclearCLI vs API: time for a small coding task (First useful output): GPT-6.1 Sol (OpenAI API) 5.31 s (n 3, run range 5 s–6.4 s); GPT-6 Luna (OpenAI API) 0.67 s (n 3, run range 0.6 s–0.8 s). Unclear.
- Hidden prompt: input tokens for the same one-line request17n 517n 5TieHidden prompt: input tokens for the same one-line request: GPT-6.1 Sol (OpenAI API) 17 (n 5); GPT-6 Luna (OpenAI API) 17 (n 5). Tie.
| Metric | GPT-6.1 Sol (OpenAI API) | GPT-6 Luna (OpenAI API) | n | Interval or range | Outcome | Basis | Study |
|---|---|---|---|---|---|---|---|
| CLI vs API: time for a one-line answer (Total time) | 1.52 sOpenAI API · effort high · fixed exact reply, 5 runs | 0.97 sOpenAI API · effort none · fixed exact reply, 5 runs | 5 | range: 1.4 s–2.2 s vs 0.7 s–1.5 s | Unclear | The run ranges (fastest to slowest) overlap (GPT-6.1 Sol (OpenAI API) 1.35 s to 2.23 s; GPT-6 Luna (OpenAI API) 0.65 s to 1.50 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
| CLI vs API: time for a one-line answer (First useful output) | 1.34 sOpenAI API · effort high · fixed exact reply, 5 runs | 0.82 sOpenAI API · effort none · fixed exact reply, 5 runs | 5 | range: 1.3 s–2.1 s vs 0.5 s–1.4 s | Unclear | The run ranges (fastest to slowest) overlap (GPT-6.1 Sol (OpenAI API) 1.26 s to 2.12 s; GPT-6 Luna (OpenAI API) 0.51 s to 1.37 s); the medians alone do not show a reliable difference. A range is not a confidence interval. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
| CLI vs API: time for a small coding task (Total time) | 9.56 sOpenAI API · effort high · small coding task, 3 runs | 4.01 sOpenAI API · effort none · small coding task, 3 runs | 3 | range: 9.4 s–10.9 s vs 3.8 s–4.4 s | Unclear | Only 3 runs per side; the run ranges (fastest to slowest) do not overlap (GPT-6.1 Sol (OpenAI API) 9.44 s to 10.9 s; GPT-6 Luna (OpenAI API) 3.83 s to 4.35 s), but 3 runs cannot show a reliable difference. A range is not a confidence interval. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
| CLI vs API: time for a small coding task (First useful output) | 5.31 sOpenAI API · effort high · small coding task, 3 runs | 0.67 sOpenAI API · effort none · small coding task, 3 runs | 3 | range: 5 s–6.4 s vs 0.6 s–0.8 s | Unclear | Only 3 runs per side; the run ranges (fastest to slowest) do not overlap (GPT-6.1 Sol (OpenAI API) 4.99 s to 6.42 s; GPT-6 Luna (OpenAI API) 0.62 s to 0.81 s), but 3 runs cannot show a reliable difference. A range is not a confidence interval. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
| Hidden prompt: input tokens for the same one-line request | 17OpenAI API · effort high · short fixed tasks | 17OpenAI API · effort none · short fixed tasks | 5 | none recorded | Tie | Same value. More or fewer is not better by itself for this metric. | Claude Code CLI vs Codex CLI vs the API: latency and tokens |
Marks: fastest–slowest run ranges (not intervals)n is shown per side on every row
5 rows from 1 study. No row separates them: 1 tie, 4 unclear.
When to pick which
Only from the rows above. A tie is not a reason to pick either side.
When to pick GPT-6.1 Sol (OpenAI API)
No row in this data puts GPT-6.1 Sol (OpenAI API) ahead of GPT-6 Luna (OpenAI API). Pick on other grounds (price, access, the tasks you run), or measure your own workload.
When to pick GPT-6 Luna (OpenAI API)
No row in this data puts GPT-6 Luna (OpenAI API) ahead of GPT-6.1 Sol (OpenAI API). Pick on other grounds (price, access, the tasks you run), or measure your own workload.
Side by side
The study charts, showing only these two. Open a study for every configuration.
- Total time
- First useful output
| Item | Total time | First useful output | Range (lowest–highest run) | n |
|---|---|---|---|---|
| OpenAI API · GPT-6 Luna · none | 1 s | 0.8 s | Total time: 0.7 s–1.5 s; First useful output: 0.5 s–1.4 s | 5 |
| OpenAI API · GPT-6.1 Sol · high | 1.5 s | 1.3 s | Total time: 1.4 s–2.2 s; First useful output: 1.3 s–2.1 s | 5 |
2 rows, 2 series: Total time, First useful output. Total time: slowest OpenAI API · GPT-6.1 Sol · high 1.5 s (range 1.4 s–2.2 s, n 5). Fastest OpenAI API · GPT-6 Luna · none 1 s (range 0.7 s–1.5 s, n 5). All run ranges overlap. First useful output: slowest OpenAI API · GPT-6.1 Sol · high 1.3 s (range 1.3 s–2.1 s, n 5). Fastest OpenAI API · GPT-6 Luna · none 0.8 s (range 0.5 s–1.4 s, n 5). All run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 5 per row
Matched cohort, fixed exact reply, 5 runs per configuration
Dot = median; whiskers = fastest and slowest run (a range, not a confidence interval). Every run in these cohorts passed its validator.
Source: Provider explorer receipts: CLI vs API
- Total time
- First useful output
| Item | Total time | First useful output | Range (lowest–highest run) | n |
|---|---|---|---|---|
| OpenAI API · GPT-6 Luna · none | 4 s | 0.7 s | Total time: 3.8 s–4.4 s; First useful output: 0.6 s–0.8 s | 3 |
| OpenAI API · GPT-6.1 Sol · high | 9.6 s | 5.3 s | Total time: 9.4 s–10.9 s; First useful output: 5 s–6.4 s | 3 |
2 rows, 2 series: Total time, First useful output. Total time: slowest OpenAI API · GPT-6.1 Sol · high 9.6 s (range 9.4 s–10.9 s, n 3). Fastest OpenAI API · GPT-6 Luna · none 4 s (range 3.8 s–4.4 s, n 3). Not all run ranges overlap. First useful output: slowest OpenAI API · GPT-6.1 Sol · high 5.3 s (range 5 s–6.4 s, n 3). Fastest OpenAI API · GPT-6 Luna · none 0.7 s (range 0.6 s–0.8 s, n 3). Not all run ranges overlap.
NotesLines: fastest–slowest run (not an interval)n = 3 per row
Matched cohort, small coding task, 3 runs per configuration
Dot = median; whiskers = fastest and slowest run (a range, not a confidence interval). Every run in these cohorts passed its validator.
Source: Provider explorer receipts: CLI vs API
| Item | Input tokens | n |
|---|---|---|
| OpenAI API · GPT-6 Luna · none | 17 | 5 |
| OpenAI API · GPT-6.1 Sol · low | 17 | 5 |
2 rows. All at 17.
Notesn = 5 per row
Reported input tokens, matched cohort
The CLI wraps every request in its own system prompt and tool context; the bare API sends only the request. Part of the CLI input is served from cache.
Source: Provider explorer receipts: CLI vs API
How a row is called
- AheadThe 95% intervals do not overlap, or the run ranges or p50–p95 bands do not overlap with at least 5 runs per side.
- TieThe values match, or both sit at the same ceiling.
- UnclearThe intervals or ranges overlap, too few runs were recorded, no interval was recorded, or more is not better (token counts are never a win).
- CalculationDerived from list prices and recorded counts. Not a bill and not a run.
The dataset compiler makes every call; this page only draws it. No composite score, no rank.
Questions
- Which is better, GPT-6.1 Sol (OpenAI API) or GPT-6 Luna (OpenAI API)?
- GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) share 5 measured metrics from one study. No row separates them: every interval or run range overlaps, too few runs were recorded, no interval was recorded, or more is not better for that metric. The rows are 1 tie and 4 unclear; each row says why. Some rows rest on small samples (n = 3 at the smallest).
- How were GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) measured?
- They share 5 measured metrics from 1 public study: Claude Code CLI vs Codex CLI vs the API: latency and tokens. Every row names its configuration, its sample size and its interval or range.
- How do GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) compare on cLI vs API: time for a one-line answer (Total time)?
- GPT-6.1 Sol (OpenAI API): 1.52 s (OpenAI API · effort high · fixed exact reply, 5 runs; n = 5; run range 1.4 s to 2.2 s). GPT-6 Luna (OpenAI API): 0.97 s (OpenAI API · effort none · fixed exact reply, 5 runs; n = 5; run range 0.7 s to 1.5 s). The run ranges (fastest to slowest) overlap (GPT-6.1 Sol (OpenAI API) 1.35 s to 2.23 s; GPT-6 Luna (OpenAI API) 0.65 s to 1.50 s); the medians alone do not show a reliable difference. A range is not a confidence interval.
- How do GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) compare on cLI vs API: time for a one-line answer (First useful output)?
- GPT-6.1 Sol (OpenAI API): 1.34 s (OpenAI API · effort high · fixed exact reply, 5 runs; n = 5; run range 1.3 s to 2.1 s). GPT-6 Luna (OpenAI API): 0.82 s (OpenAI API · effort none · fixed exact reply, 5 runs; n = 5; run range 0.5 s to 1.4 s). The run ranges (fastest to slowest) overlap (GPT-6.1 Sol (OpenAI API) 1.26 s to 2.12 s; GPT-6 Luna (OpenAI API) 0.51 s to 1.37 s); the medians alone do not show a reliable difference. A range is not a confidence interval.
- How do GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) compare on cLI vs API: time for a small coding task (Total time)?
- GPT-6.1 Sol (OpenAI API): 9.56 s (OpenAI API · effort high · small coding task, 3 runs; n = 3; run range 9.4 s to 10.9 s). GPT-6 Luna (OpenAI API): 4.01 s (OpenAI API · effort none · small coding task, 3 runs; n = 3; run range 3.8 s to 4.4 s). Only 3 runs per side; the run ranges (fastest to slowest) do not overlap (GPT-6.1 Sol (OpenAI API) 9.44 s to 10.9 s; GPT-6 Luna (OpenAI API) 3.83 s to 4.35 s), but 3 runs cannot show a reliable difference. A range is not a confidence interval.
- How do GPT-6.1 Sol (OpenAI API) and GPT-6 Luna (OpenAI API) compare on cLI vs API: time for a small coding task (First useful output)?
- GPT-6.1 Sol (OpenAI API): 5.31 s (OpenAI API · effort high · small coding task, 3 runs; n = 3; run range 5 s to 6.4 s). GPT-6 Luna (OpenAI API): 0.67 s (OpenAI API · effort none · small coding task, 3 runs; n = 3; run range 0.6 s to 0.8 s). Only 3 runs per side; the run ranges (fastest to slowest) do not overlap (GPT-6.1 Sol (OpenAI API) 4.99 s to 6.42 s; GPT-6 Luna (OpenAI API) 0.62 s to 0.81 s), but 3 runs cannot show a reliable difference. A range is not a confidence interval.
The studies behind this page
Claude Code CLI vs Codex CLI vs the API: latency and tokens
194 timed runs: how long Claude Code, Codex CLI and the OpenAI API take to answer and to fix code, and how many hidden tokens a CLI adds.