{"i":21,"slug":"routing-holdout","chart":{"id":"routing-holdout-by-purpose","title":"Exact rate on unseen decisions, by decision type","subtitle":"14 cases per decision type","kind":"grouped-bar","unit":"rate","polarity":"higher","yLabel":"Exact","series":{"$k":["name","points"],"$r":[["Jev 1.13 (TypeSafe)",{"$k":["label","value","lo","hi","n","highlight"],"$r":[["Failure class",0.9286,0.6853,0.9873,14,true],["Message intent",0.8571,0.6006,0.9599,14,true],["Is it a rule?",0.9286,0.6853,0.9873,14,true],["Context shape",0.5714,0.3259,0.7862,14,true]]}],["Claude Haiku 4.5 · Claude Code",{"$k":["label","value","lo","hi","n","highlight"],"$r":[["Failure class",0.9286,0.6853,0.9873,14,false],["Message intent",0.9286,0.6853,0.9873,14,false],["Is it a rule?",0.9286,0.6853,0.9873,14,false],["Context shape",0.3571,0.1634,0.6124,14,false]]}],["Claude Sonnet 5.5 (low) · Claude Code",{"$k":["label","value","lo","hi","n","highlight"],"$r":[["Failure class",1,0.7847,1,14,false],["Message intent",1,0.7847,1,14,false],["Is it a rule?",0.9286,0.6853,0.9873,14,false],["Context shape",0.5714,0.3259,0.7862,14,false]]}]]},"note":"Whiskers are 95% Wilson intervals. With 14 cases a perfect score has an interval of 78% to 100%, so a decision type where every router scores 14 of 14 is at its ceiling and cannot rank them.","whisker":"ci95","sourceIds":["agent-routing-holdout"]}}