{"i":1,"slug":"swe-bench-opus-vs-sonnet","chart":{"id":"swebench-opus-sonnet-resolved","title":"Resolved on the same 3 SWE-bench Verified instances (interim)","subtitle":"One attempt per arm per instance, official grader · 95% Wilson intervals","kind":"dot-range","unit":"rate","whisker":"ci95","yLabel":"Resolved","viz":"IntervalDotPlot","series":[{"name":"Resolved","points":[{"label":"Claude Opus 5.5 (Agent, new build)","value":0.6667,"lo":0.2077,"hi":0.9385,"n":3},{"label":"Claude Sonnet 5.5 (Agent, older builds)","value":0.3333,"lo":0.0615,"hi":0.7923,"n":3}]}],"note":"Interim: 3 of 8 declared pairs are graded; 5 were never started; no resumed attempts are included here. With n = 3 the intervals span most of the axis, so this chart supports no ranking. The instances were chosen to hold Sonnet misses and resolves in equal numbers, so neither rate estimates SWE-bench Verified as a whole.","sourceIds":["agent-swebench-opus","agent-swebench-c1","agent-swebench-c2"]}}