{"i":7,"slug":"caching-consistency","chart":{"id":"consistency-distinct-answers","title":"Same prompt, 10 times: how many different answers","subtitle":"Distinct normalized answers over 10 repetitions (1 = the same answer every time)","kind":"grouped-bar","unit":"count","yLabel":"Distinct answers","series":{"$k":["name","points"],"$r":[["Exact number",{"$k":["label","value","n"],"$r":[["Claude Haiku 4.5 · Claude Code",1,10],["Claude Sonnet 5.5 · Claude Code",1,10],["GPT-6.1 Sol (medium) · Codex CLI",1,10]]}],["JSON object",{"$k":["label","value","n"],"$r":[["Claude Haiku 4.5 · Claude Code",1,10],["Claude Sonnet 5.5 · Claude Code",1,10],["GPT-6.1 Sol (medium) · Codex CLI",1,10]]}],["Code fix",{"$k":["label","value","n"],"$r":[["Claude Haiku 4.5 · Claude Code",6,10],["Claude Sonnet 5.5 · Claude Code",3,10],["GPT-6.1 Sol (medium) · Codex CLI",6,10]]}]]},"note":"Normalized: the last number for the exact-number prompt; key-sorted JSON for the JSON prompt; for the code fix, the code without fences, comments, whitespace, quote style and line-end semicolons. One distinct answer is not the same as a correct answer: an answer can be the same and wrong every time. Different code can be equally correct.","sourceIds":["agent-caching-consistency"]}}