{"i":114,"comparison":{"slug":"claude-sonnet-5-5-vs-gpt-6-1-sol-openai-api","a":"claude-sonnet-5-5","b":"gpt-6-1-sol-openai-api","title":"Claude Sonnet 5.5 vs GPT-6.1 Sol (OpenAI API)","seoTitle":"Claude Sonnet 5.5 vs GPT-6.1 Sol (OpenAI API): benchmarks","description":"Claude Sonnet 5.5 vs GPT-6.1 Sol (OpenAI API): 3 measured metrics from one study, with sample sizes, intervals and every failure counted.","verdict":"Claude Sonnet 5.5 and GPT-6.1 Sol (OpenAI API) share 3 measured metrics from one study. No row separates them: every interval or run range overlaps, too few runs were recorded, no interval was recorded, or more is not better for that metric. The rows are 3 unclear; each row says why. Every row ran the two sides through different routes (for example Claude Code vs OpenAI API), so they compare route + model pairs, not models alone; the contexts name the route. Some rows rest on small samples (n = 3 at the smallest).","rows":{"$k":["metric","aValue","bValue","unit","aDisplay","bDisplay","winner","basis","studySlug","n","chartId","aN","bN","aContext","bContext","rangeKind","spanKind","aRange","bRange"],"$r":[["Repairing a scheduler: Claude Code vs Codex vs API (Total time)",15,17.32,"seconds","15.0 s","17.3 s","unclear","Only 3 runs per side; the run ranges (fastest to slowest) do not overlap (Claude Sonnet 5.5 13.9 s to 15.9 s; GPT-6.1 Sol (OpenAI API) 16.3 s to 18.6 s), but 3 runs cannot show a reliable difference. A range is not a confidence interval.","cli-model-latency-tokens",3,"scheduler-repair-claude-vs-codex",3,3,"Claude Code CLI · effort medium · scheduler repair, 296 checks, 3 runs","OpenAI API · effort medium · scheduler repair, 296 checks, 3 runs","range","minmax",[13.89,15.89],[16.28,18.61]],["Repairing a scheduler: Claude Code vs Codex vs API (First useful output)",7.55,7.46,"seconds","7.55 s","7.46 s","unclear","The run ranges (fastest to slowest) overlap (Claude Sonnet 5.5 6.77 s to 7.63 s; GPT-6.1 Sol (OpenAI API) 6.68 s to 9.05 s); the medians alone do not show a reliable difference. A range is not a confidence interval.","cli-model-latency-tokens",3,"scheduler-repair-claude-vs-codex",3,3,"Claude Code CLI · effort medium · scheduler repair, 296 checks, 3 runs","OpenAI API · effort medium · scheduler repair, 296 checks, 3 runs","range","minmax",[6.77,7.63],[6.68,9.05]],["Output tokens to repair the scheduler (Output tokens)",2227,1313,"tokens","2,227","1,313","unclear","More or fewer tokens is not better or worse by itself; this row describes behaviour, not a winner.","cli-model-latency-tokens",3,"scheduler-repair-output-tokens",3,3,"Claude Code CLI · effort medium · scheduler repair, 296 checks, 3 runs","OpenAI API · effort medium · scheduler repair, 296 checks, 3 runs","\u0001","\u0001","\u0001","\u0001"]]}}}