{"i":21,"slug":"routing-holdout","chart":{"id":"routing-holdout-tuned-vs-unseen","title":"Tuned case set vs unseen holdout: exact rate per router","subtitle":"Tuned set: the routing study’s 82 cases, revised against Jev answers. Holdout: 56 new cases, frozen before any router call","kind":"grouped-bar","unit":"rate","polarity":"higher","yLabel":"Exact","series":[{"name":"Tuned set (routing-jev-vs-llm)","points":{"$k":["label","value","lo","hi","n","highlight"],"$r":[["Jev 1.13 (TypeSafe)",0.9024,0.8191,0.9497,82,true],["Claude Haiku 4.5 · Claude Code",0.8902,0.8044,0.9412,82,false],["Claude Sonnet 5.5 (low) · Claude Code",0.939,0.8651,0.9737,82,false]]}},{"name":"Unseen holdout","points":{"$k":["label","value","lo","hi","n","highlight"],"$r":[["Jev 1.13 (TypeSafe)",0.8214,0.7016,0.9,56,true],["Claude Haiku 4.5 · Claude Code",0.7857,0.6618,0.8729,56,false],["Claude Sonnet 5.5 (low) · Claude Code",0.875,0.7637,0.9381,56,false]]}}],"note":"Whiskers are 95% Wilson intervals. The two case sets differ in mix and size, so a gap mixes a change of case set with any change in the router, and the data cannot separate them. A gap counts only when the two intervals do not overlap.","whisker":"ci95","sourceIds":["agent-routing-holdout","agent-routing"]}}