{
  "schema": "agent-public-bench@1",
  "generatedAt": "2026-10-09T00:00:00.000Z",
  "url": "https://agent.sasid.ai/benchmarks/subagent-fan-out-cost",
  "study": {
    "slug": "subagent-fan-out-cost",
    "title": "What does fanning out 100 AI subagents cost? A calculation from measured runs",
    "seoTitle": "CLI subagent cost: calculations for 10, 100 and 500 calls",
    "description": "Calculation from separate CLI calls and Crewshal worker attempts. List-price cost scenarios for 10, 100 and 500 units; no parallel run was measured.",
    "question": "What cost scenarios do recorded CLI calls and worker attempts support for 10, 100 and 500 units?",
    "answer": "Calculation, not a run. We started no parallel subagents. At list price, 100 small Sonnet calls cost $0.36 using the median (15 calls; per-call range $0.0034 to $0.0102). Sonnet passed 12/15 (Wilson 95% interval 54.8% to 93.0%). This is cost per call, not per correct answer.\n\nAt the mean notional cost, 100 full worker attempts cost $280.72 (33 attempts; per-attempt range $0.89 to $4.83).\n\nFor Claude Code minimal replies, 100 × median input tokens gives 676,100 tokens (n = 5; token range unavailable). Input-only Sonnet price scenarios give $0.16, $1.35 or $2.70, with different cache assumptions. The corrected Codex aggregate reports 17,051 inclusive input tokens, including 13,184 cache-read tokens. We add no Codex input or cost projection.\n\nSerial scenarios use N × median duration. We do not know the parallel completion time.",
    "date": "2026-10-06",
    "updated": "2026-10-06",
    "tags": [
      "calculation",
      "subagents",
      "parallel-agents",
      "claude-code",
      "codex-cli",
      "llm-pricing",
      "latency"
    ],
    "method": [
      "The calculation uses separate CLI processes and complete Crewshal worker attempts. It does not measure Claude Code’s built-in subagents within a session.",
      "Minimal replies: 5 calls per CLI on one Mac, one at a time. Claude Code used Haiku 4.5; Codex used its default model. Wall time includes the answer and process exit. Claude input tokens include all cache categories.",
      "The current public Codex aggregate reports 17,051 inclusive input tokens, including 13,184 cache-read tokens. Cached tokens are a subset; they must not be added again. We do not project a common startup cost across different models and prompts.",
      "Small calls: 5 short tasks, 15 calls per selected configuration. We price the reported tokens at list rates, with recorded cache reads and writes. Failed validators remain in cost and time summaries. Pass rates use Wilson 95% intervals.",
      "Worker costs: all 33 SWE-bench attempts, including failures, from 2 campaigns on 2 platform builds. We use recorded notional totals, including compaction, rather than reprice only the main-call tokens.",
      "Cost scenarios multiply per-call medians or the mean worker cost by N. Serial duration scenarios multiply the median duration by N. Neither predicts a new workload.",
      "First-turn cache shares (calculation from reported first-turn counters): Claude 18.7%, n = 6, range 18.7% to 18.7%; Codex 55.5%, n = 3, range 47.7% to 59.3%. These are process-level means, not pooled token shares."
    ],
    "caveats": [
      "Nothing ran in parallel. We did not measure rate limits, shared-host contention, machine limits or orchestration costs. One-unit medians are reference times, not parallel lower bounds.",
      "These are subscription calls priced at API list rates, not invoices. We did not measure subscription capacity or billing at scale.",
      "Minimal-reply calls do not isolate the CLI prefix. No empty-prompt control establishes tokens sent before work. Tools, prompts, flags and models differ across runs.",
      "Claude’s minimal-reply input median is 6,761; whole Sonnet calls have a median 2,081. Times also differ: 2.53 s versus 2.31 s. We did not test the cause. Each small call includes its own CLI prompt, not the separately measured minimal-reply prefix.",
      "Cache costs are scenarios, not bounds. Complete reuse assumes a cache write finishes before later calls start and their prefixes match. Actual writes can exceed plain-input prices.",
      "The 4 later Claude processes read fewer first-turn cache tokens than they wrote. Counters do not identify which ledger tokens they read. Claude used a shortened ledger; Codex used the earlier longer ledger.",
      "Pre-call protocol declaration is unverified from the selected public extracts. File times and final protocol text do not establish what was declared before the calls. This calculation supplies no new experiment or advance-declaration proof.",
      "The linked minimal-reply, head-to-head and cache extracts retain final protocol text with advance-declaration headings. No preserved pre-call protocol versions were supplied for these extracts. Their public summary labels this missing provenance before publication; historical raw records and review verdicts stay unchanged.",
      "The small task set is easy and selected, not a representative coding sample. Sonnet has three exact-text format misses with correct final numbers. Other selected configurations passed every call and hit a ceiling. Small n and overlapping cost ranges do not support a ranking.",
      "Other work shared the Mac and subscription accounts. Timing may include shared load; different dates, models and harnesses prevent causal comparisons.",
      "The worker sample has 33 attempts. Costs range from $0.89 to $4.83 and times from 1.6 min to 54.1 min. Campaigns used different platform builds and include replacement instances. New tasks may require different tokens and outcomes.",
      "These publication-label corrections need a fresh source and text recheck before publication. They do not establish advance declaration or replace the historical failed source review."
    ],
    "hero": {
      "statIds": [
        "fan-out-small-cost-100-sonnet"
      ]
    },
    "sourceIds": [
      "calc-fan-out",
      "agent-routing-overhead",
      "agent-caching-consistency",
      "agent-provider-h2h",
      "agent-swebench-c1",
      "agent-swebench-c2",
      "price-anthropic",
      "price-openai"
    ],
    "stats": [
      {
        "id": "fan-out-startup-tokens-claude",
        "label": "Minimal-reply input tokens, Claude Code (median, Haiku 4.5)",
        "value": 6761,
        "unit": "tokens",
        "display": "6,761",
        "n": 5,
        "note": "Includes the user prompt and cache reads and writes. Token range is absent from the public aggregate; this is not an isolated start-up tax."
      },
      {
        "id": "fan-out-startup-time-claude",
        "label": "Recorded: Claude Code minimal-reply wall time (median)",
        "value": 2.529,
        "unit": "seconds",
        "display": "2.53 s",
        "n": 5,
        "note": "Range 2.27 s to 3.38 s. Different models; timing includes the answer and process exit."
      },
      {
        "id": "fan-out-startup-time-codex",
        "label": "Recorded: Codex CLI minimal-reply wall time (median)",
        "value": 5.999,
        "unit": "seconds",
        "display": "6.00 s",
        "n": 5,
        "note": "Range 5.37 s to 6.51 s. Different models; timing includes the answer and process exit."
      },
      {
        "id": "fan-out-startup-tokens-100-claude",
        "label": "100 minimal replies × median input tokens (calculation)",
        "value": 676100,
        "unit": "tokens",
        "display": "676,100",
        "n": 5,
        "note": "Assumes the same prompt size. Token range is not available in the raw aggregate."
      },
      {
        "id": "fan-out-startup-cost-100-claude-upper",
        "label": "100 minimal replies, plain input (calculation)",
        "value": 1.3522,
        "unit": "usd",
        "display": "$1.35",
        "n": 5,
        "note": "Input-only scenario at Sonnet 5.5 prices; not a measured bound."
      },
      {
        "id": "fan-out-startup-cost-100-claude-lower",
        "label": "100 minimal replies, first writes, later calls read (calculation)",
        "value": 0.16091,
        "unit": "usd",
        "display": "$0.16",
        "n": 5,
        "note": "Input-only scenario at Sonnet 5.5 prices; not a measured bound."
      },
      {
        "id": "fan-out-startup-cost-100-claude-if-written",
        "label": "100 minimal replies, every call writes (calculation)",
        "value": 2.7044,
        "unit": "usd",
        "display": "$2.70",
        "n": 5,
        "note": "Input-only scenario at Sonnet 5.5 prices; not a measured bound."
      },
      {
        "id": "fan-out-cache-cross-session-reuse",
        "label": "Later Claude sessions with first-turn cache reads greater than writes (proxy)",
        "value": 0,
        "unit": "count",
        "display": "0 of 4",
        "n": 4,
        "note": "Proxy only: counters do not identify which ledger tokens were read. Wilson 95% rate interval 0.0% to 49.0%."
      },
      {
        "id": "fan-out-cache-turn1-read-share-claude",
        "label": "Recorded: Claude Code first-turn cache-read share (mean; calculation from reported first-turn counters)",
        "value": 0.18685,
        "unit": "rate",
        "display": "18.7%",
        "n": 6,
        "note": "Per-process range 18.7% to 18.7%; not a confidence interval. Different ledger sizes and CLI prefixes; routes are not matched."
      },
      {
        "id": "fan-out-cache-turn1-read-share-codex",
        "label": "Recorded: Codex CLI first-turn cache-read share (mean; calculation from reported first-turn counters)",
        "value": 0.55451,
        "unit": "rate",
        "display": "55.5%",
        "n": 3,
        "note": "Per-process range 47.7% to 59.3%; not a confidence interval. Different ledger sizes and CLI prefixes; routes are not matched."
      },
      {
        "id": "fan-out-small-task-cost-sonnet",
        "label": "Recorded: Claude Code · Claude Sonnet 5.5, median per-call cost (calculation)",
        "value": 0.0036,
        "unit": "usd",
        "display": "$0.0036",
        "n": 15,
        "note": "Range $0.0034 to $0.0102. Includes failed validators; not cost per correct answer."
      },
      {
        "id": "fan-out-small-task-passed-sonnet",
        "label": "Recorded: Claude Code · Claude Sonnet 5.5, strict validator pass rate",
        "value": 0.8,
        "unit": "rate",
        "display": "12/15",
        "n": 15,
        "note": "The easy task set hits a ceiling for configurations with no misses.",
        "ci": [
          0.5481,
          0.9295
        ]
      },
      {
        "id": "fan-out-small-cost-100-sonnet",
        "label": "100 calls, Claude Code · Claude Sonnet 5.5 (calculation)",
        "value": 0.36006,
        "unit": "usd",
        "display": "$0.36",
        "n": 15,
        "note": "N × median cost; scaled observed range $0.34 to $1.02."
      },
      {
        "id": "fan-out-small-task-cost-opus",
        "label": "Recorded: Claude Code · Claude Opus 5.5, median per-call cost (calculation)",
        "value": 0.00688,
        "unit": "usd",
        "display": "$0.0069",
        "n": 15,
        "note": "Range $0.0059 to $0.0223. Includes failed validators; not cost per correct answer."
      },
      {
        "id": "fan-out-small-task-passed-opus",
        "label": "Recorded: Claude Code · Claude Opus 5.5, strict validator pass rate",
        "value": 1,
        "unit": "rate",
        "display": "15/15",
        "n": 15,
        "note": "The easy task set hits a ceiling for configurations with no misses.",
        "ci": [
          0.7961,
          1
        ]
      },
      {
        "id": "fan-out-small-cost-100-opus",
        "label": "100 calls, Claude Code · Claude Opus 5.5 (calculation)",
        "value": 0.68766,
        "unit": "usd",
        "display": "$0.69",
        "n": 15,
        "note": "N × median cost; scaled observed range $0.59 to $2.23."
      },
      {
        "id": "fan-out-small-task-cost-haiku",
        "label": "Recorded: Claude Code · Claude Haiku 4.5, median per-call cost (calculation)",
        "value": 0.00566,
        "unit": "usd",
        "display": "$0.0057",
        "n": 15,
        "note": "Range $0.0051 to $0.0180. Includes failed validators; not cost per correct answer."
      },
      {
        "id": "fan-out-small-task-passed-haiku",
        "label": "Recorded: Claude Code · Claude Haiku 4.5, strict validator pass rate",
        "value": 1,
        "unit": "rate",
        "display": "15/15",
        "n": 15,
        "note": "The easy task set hits a ceiling for configurations with no misses.",
        "ci": [
          0.7961,
          1
        ]
      },
      {
        "id": "fan-out-small-cost-100-haiku",
        "label": "100 calls, Claude Code · Claude Haiku 4.5 (calculation)",
        "value": 0.5657,
        "unit": "usd",
        "display": "$0.57",
        "n": 15,
        "note": "N × median cost; scaled observed range $0.51 to $1.80."
      },
      {
        "id": "fan-out-small-task-cost-fable",
        "label": "Recorded: Claude Code · Claude Fable 5.1, median per-call cost (calculation)",
        "value": 0.00987,
        "unit": "usd",
        "display": "$0.0099",
        "n": 15,
        "note": "Range $0.0049 to $0.0584. Includes failed validators; not cost per correct answer."
      },
      {
        "id": "fan-out-small-task-passed-fable",
        "label": "Recorded: Claude Code · Claude Fable 5.1, strict validator pass rate",
        "value": 1,
        "unit": "rate",
        "display": "15/15",
        "n": 15,
        "note": "The easy task set hits a ceiling for configurations with no misses.",
        "ci": [
          0.7961,
          1
        ]
      },
      {
        "id": "fan-out-small-cost-100-fable",
        "label": "100 calls, Claude Code · Claude Fable 5.1 (calculation)",
        "value": 0.98675,
        "unit": "usd",
        "display": "$0.99",
        "n": 15,
        "note": "N × median cost; scaled observed range $0.49 to $5.84."
      },
      {
        "id": "fan-out-small-task-cost-sol",
        "label": "Recorded: Codex CLI · GPT-6.1 Sol (medium), median per-call cost (calculation)",
        "value": 0.01018,
        "unit": "usd",
        "display": "$0.0102",
        "n": 15,
        "note": "Range $0.0054 to $0.0269. Includes failed validators; not cost per correct answer."
      },
      {
        "id": "fan-out-small-task-passed-sol",
        "label": "Recorded: Codex CLI · GPT-6.1 Sol (medium), strict validator pass rate",
        "value": 1,
        "unit": "rate",
        "display": "15/15",
        "n": 15,
        "note": "The easy task set hits a ceiling for configurations with no misses.",
        "ci": [
          0.7961,
          1
        ]
      },
      {
        "id": "fan-out-small-cost-100-sol",
        "label": "100 calls, Codex CLI · GPT-6.1 Sol (medium) (calculation)",
        "value": 1.0178,
        "unit": "usd",
        "display": "$1.02",
        "n": 15,
        "note": "N × median cost; scaled observed range $0.54 to $2.69."
      },
      {
        "id": "fan-out-worker-cost-attempt",
        "label": "Full worker attempt, mean notional cost (calculation)",
        "value": 2.8072,
        "unit": "usd",
        "display": "$2.81",
        "n": 33,
        "note": "Range $0.89 to $4.83; 25 resolved; all failures included."
      },
      {
        "id": "fan-out-worker-minutes-median",
        "label": "Full worker attempt, median wall time",
        "value": 9.59,
        "unit": "minutes",
        "display": "9.6 min",
        "n": 33,
        "note": "Range 1.6 min to 54.1 min."
      },
      {
        "id": "fan-out-worker-calls-attempt",
        "label": "Full worker attempt, mean model calls",
        "value": 49.45455,
        "unit": "calls",
        "display": "49.5",
        "n": 33,
        "note": "Range 13 to 71."
      },
      {
        "id": "fan-out-worker-cost-10",
        "label": "10 full worker attempts (calculation)",
        "value": 28.07197,
        "unit": "usd",
        "display": "$28.07",
        "n": 33,
        "note": "N × mean notional cost; scaled observed range $8.90 to $48.27."
      },
      {
        "id": "fan-out-worker-cost-100",
        "label": "100 full worker attempts (calculation)",
        "value": 280.71973,
        "unit": "usd",
        "display": "$280.72",
        "n": 33,
        "note": "N × mean notional cost; scaled observed range $88.98 to $482.74."
      },
      {
        "id": "fan-out-worker-cost-500",
        "label": "500 full worker attempts (calculation)",
        "value": 1403.59864,
        "unit": "usd",
        "display": "$1403.60",
        "n": 33,
        "note": "N × mean notional cost; scaled observed range $444.91 to $2413.72."
      },
      {
        "id": "fan-out-worker-serial-100",
        "label": "100 worker attempts, serial duration scenario (calculation)",
        "value": 959,
        "unit": "minutes",
        "display": "16.0 h",
        "n": 33,
        "note": "N × median; not a measured total."
      },
      {
        "id": "fan-out-worker-model-calls-100",
        "label": "100 worker attempts × mean calls (calculation)",
        "value": 4945.45455,
        "unit": "calls",
        "display": "4,945",
        "n": 33,
        "note": "Scaled observed range 1300 to 7100; provider limits were not tested."
      }
    ],
    "charts": [
      {
        "id": "fan-out-startup-tax",
        "title": "Claude Code minimal-reply input cost: three scenarios (calculation)",
        "subtitle": "N × 6,761 median input tokens, repriced at Sonnet 5.5 rates",
        "kind": "grouped-bar",
        "unit": "usd",
        "yLabel": "USD, input only",
        "series": [
          {
            "name": "Plain input, no cache surcharge",
            "points": [
              {
                "label": "10 subagents",
                "value": 0.1352,
                "n": 5
              },
              {
                "label": "100 subagents",
                "value": 1.3522,
                "n": 5
              },
              {
                "label": "500 subagents",
                "value": 6.761,
                "n": 5
              }
            ]
          },
          {
            "name": "First writes; all later calls read the whole prefix",
            "points": [
              {
                "label": "10 subagents",
                "value": 0.0392,
                "n": 5
              },
              {
                "label": "100 subagents",
                "value": 0.1609,
                "n": 5
              },
              {
                "label": "500 subagents",
                "value": 0.7018,
                "n": 5
              }
            ]
          },
          {
            "name": "Every call writes the whole prefix (1-hour price)",
            "points": [
              {
                "label": "10 subagents",
                "value": 0.2704,
                "n": 5
              },
              {
                "label": "100 subagents",
                "value": 2.7044,
                "n": 5
              },
              {
                "label": "500 subagents",
                "value": 13.522,
                "n": 5
              }
            ]
          }
        ],
        "note": "Calculation, not a bill. n = 5 minimal replies on Haiku 4.5, one at a time. Sonnet 5.5 prices: $2.00 input, $0.20 cache read and $4.00 1-hour write per million tokens (2026-09-21). These scenarios use assumptions, not measured bounds. These scenarios exclude output and orchestration costs. No empty-prompt control isolates the CLI prefix. The corrected Codex aggregate reports 17,051 inclusive input tokens, including 13,184 cache-read tokens. Cached input is a subset, not an additional count. We omit its input and cost projections; different models and a minimal reply do not isolate a common startup prefix.",
        "sourceIds": [
          "calc-fan-out",
          "agent-routing-overhead",
          "price-anthropic"
        ]
      },
      {
        "id": "fan-out-total-cost",
        "title": "N × per-call or per-attempt cost, by workload (calculation)",
        "subtitle": "Small calls use medians; worker attempts use the mean. Each workload comes from a different run.",
        "kind": "grouped-bar",
        "unit": "usd",
        "yLabel": "USD at list price",
        "whisker": "minmax",
        "series": [
          {
            "name": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
            "points": [
              {
                "label": "10 subagents",
                "value": 0.1352,
                "n": 5
              },
              {
                "label": "100 subagents",
                "value": 1.3522,
                "n": 5
              },
              {
                "label": "500 subagents",
                "value": 6.761,
                "n": 5
              }
            ]
          },
          {
            "name": "Claude Code · Claude Sonnet 5.5",
            "points": [
              {
                "label": "10 subagents",
                "value": 0.036,
                "n": 15,
                "lo": 0.0342,
                "hi": 0.1021
              },
              {
                "label": "100 subagents",
                "value": 0.3601,
                "n": 15,
                "lo": 0.3425,
                "hi": 1.0211
              },
              {
                "label": "500 subagents",
                "value": 1.8003,
                "n": 15,
                "lo": 1.7123,
                "hi": 5.1053
              }
            ]
          },
          {
            "name": "Codex CLI · GPT-6.1 Sol (medium)",
            "points": [
              {
                "label": "10 subagents",
                "value": 0.1018,
                "n": 15,
                "lo": 0.054,
                "hi": 0.2686
              },
              {
                "label": "100 subagents",
                "value": 1.0178,
                "n": 15,
                "lo": 0.5404,
                "hi": 2.6858
              },
              {
                "label": "500 subagents",
                "value": 5.089,
                "n": 15,
                "lo": 2.7018,
                "hi": 13.429
              }
            ]
          },
          {
            "name": "Full worker attempt · Crewshal · Sonnet 5.5",
            "points": [
              {
                "label": "10 subagents",
                "value": 28.072,
                "n": 33,
                "lo": 8.8983,
                "hi": 48.2744
              },
              {
                "label": "100 subagents",
                "value": 280.7197,
                "n": 33,
                "lo": 88.9827,
                "hi": 482.7436
              },
              {
                "label": "500 subagents",
                "value": 1403.5986,
                "n": 33,
                "lo": 444.9135,
                "hi": 2413.718
              }
            ]
          }
        ],
        "note": "We ran no parallel workload. Whiskers multiply observed per-unit cost ranges by N; they are scenario spreads, not confidence intervals.\n\nSmall-call costs include non-passes: Sonnet 12/15, Sol 15/15. These are per-call costs, not costs per correct answer. Each call includes its own CLI prompt, at the size used in that run.\n\nModels, flags and prompts differ across these runs. We did not test the cause of the cost difference. Do not add these bars together.\n\nWorker costs include all 33 attempts, including failures.",
        "sourceIds": [
          "calc-fan-out",
          "agent-routing-overhead",
          "agent-caching-consistency",
          "agent-provider-h2h",
          "agent-swebench-c1",
          "agent-swebench-c2",
          "price-anthropic",
          "price-openai"
        ]
      },
      {
        "id": "fan-out-wall-time",
        "title": "Serial duration scenarios and one-unit reference times (calculation)",
        "subtitle": "Serial = N × median. We did not measure parallel completion time.",
        "kind": "grouped-bar",
        "unit": "seconds",
        "yLabel": "Seconds",
        "whisker": "minmax",
        "series": [
          {
            "name": "Serial scenario, 10 units",
            "points": [
              {
                "label": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
                "value": 25.29,
                "n": 5,
                "lo": 22.73,
                "hi": 33.82
              },
              {
                "label": "Claude Code · Claude Sonnet 5.5",
                "value": 23.09,
                "n": 15,
                "lo": 21.73,
                "hi": 77.25
              },
              {
                "label": "Claude Code · Claude Opus 5.5",
                "value": 27.45,
                "n": 15,
                "lo": 24.7,
                "hi": 89.14
              },
              {
                "label": "Claude Code · Claude Haiku 4.5",
                "value": 44.28,
                "n": 15,
                "lo": 31.64,
                "hi": 235.69
              },
              {
                "label": "Claude Code · Claude Fable 5.1",
                "value": 19.44,
                "n": 15,
                "lo": 14.09,
                "hi": 98.28
              },
              {
                "label": "Codex CLI · GPT-6.1 Sol (medium)",
                "value": 56.49,
                "n": 15,
                "lo": 40.95,
                "hi": 254.61
              },
              {
                "label": "Full worker attempt · Crewshal · Sonnet 5.5",
                "value": 5754,
                "n": 33,
                "lo": 978,
                "hi": 32454
              },
              {
                "label": "Codex CLI minimal reply (default model)",
                "value": 59.99,
                "n": 5,
                "lo": 53.67,
                "hi": 65.06
              }
            ]
          },
          {
            "name": "Serial scenario, 100 units",
            "points": [
              {
                "label": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
                "value": 252.9,
                "n": 5,
                "lo": 227.3,
                "hi": 338.2
              },
              {
                "label": "Claude Code · Claude Sonnet 5.5",
                "value": 230.9,
                "n": 15,
                "lo": 217.3,
                "hi": 772.5
              },
              {
                "label": "Claude Code · Claude Opus 5.5",
                "value": 274.5,
                "n": 15,
                "lo": 247,
                "hi": 891.4
              },
              {
                "label": "Claude Code · Claude Haiku 4.5",
                "value": 442.8,
                "n": 15,
                "lo": 316.4,
                "hi": 2356.9
              },
              {
                "label": "Claude Code · Claude Fable 5.1",
                "value": 194.4,
                "n": 15,
                "lo": 140.9,
                "hi": 982.8
              },
              {
                "label": "Codex CLI · GPT-6.1 Sol (medium)",
                "value": 564.9,
                "n": 15,
                "lo": 409.5,
                "hi": 2546.1
              },
              {
                "label": "Full worker attempt · Crewshal · Sonnet 5.5",
                "value": 57540,
                "n": 33,
                "lo": 9780,
                "hi": 324540
              },
              {
                "label": "Codex CLI minimal reply (default model)",
                "value": 599.9,
                "n": 5,
                "lo": 536.7,
                "hi": 650.6
              }
            ]
          },
          {
            "name": "Serial scenario, 500 units",
            "points": [
              {
                "label": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
                "value": 1264.5,
                "n": 5,
                "lo": 1136.5,
                "hi": 1691
              },
              {
                "label": "Claude Code · Claude Sonnet 5.5",
                "value": 1154.5,
                "n": 15,
                "lo": 1086.5,
                "hi": 3862.5
              },
              {
                "label": "Claude Code · Claude Opus 5.5",
                "value": 1372.5,
                "n": 15,
                "lo": 1235,
                "hi": 4457
              },
              {
                "label": "Claude Code · Claude Haiku 4.5",
                "value": 2214,
                "n": 15,
                "lo": 1582,
                "hi": 11784.5
              },
              {
                "label": "Claude Code · Claude Fable 5.1",
                "value": 972,
                "n": 15,
                "lo": 704.5,
                "hi": 4914
              },
              {
                "label": "Codex CLI · GPT-6.1 Sol (medium)",
                "value": 2824.5,
                "n": 15,
                "lo": 2047.5,
                "hi": 12730.5
              },
              {
                "label": "Full worker attempt · Crewshal · Sonnet 5.5",
                "value": 287700,
                "n": 33,
                "lo": 48900,
                "hi": 1622700
              },
              {
                "label": "Codex CLI minimal reply (default model)",
                "value": 2999.5,
                "n": 5,
                "lo": 2683.5,
                "hi": 3253
              }
            ]
          },
          {
            "name": "One-unit median reference (not a parallel bound)",
            "points": [
              {
                "label": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
                "value": 2.529,
                "n": 5,
                "lo": 2.273,
                "hi": 3.382
              },
              {
                "label": "Claude Code · Claude Sonnet 5.5",
                "value": 2.309,
                "n": 15,
                "lo": 2.173,
                "hi": 7.725
              },
              {
                "label": "Claude Code · Claude Opus 5.5",
                "value": 2.745,
                "n": 15,
                "lo": 2.47,
                "hi": 8.914
              },
              {
                "label": "Claude Code · Claude Haiku 4.5",
                "value": 4.428,
                "n": 15,
                "lo": 3.164,
                "hi": 23.569
              },
              {
                "label": "Claude Code · Claude Fable 5.1",
                "value": 1.944,
                "n": 15,
                "lo": 1.409,
                "hi": 9.828
              },
              {
                "label": "Codex CLI · GPT-6.1 Sol (medium)",
                "value": 5.649,
                "n": 15,
                "lo": 4.095,
                "hi": 25.461
              },
              {
                "label": "Full worker attempt · Crewshal · Sonnet 5.5",
                "value": 575.4,
                "n": 33,
                "lo": 97.8,
                "hi": 3245.4
              },
              {
                "label": "Codex CLI minimal reply (default model)",
                "value": 5.999,
                "n": 5,
                "lo": 5.367,
                "hi": 6.506
              }
            ]
          }
        ],
        "note": "A serial total is a sum of durations. N × the median is a scenario, not its measured total or expected value. Whiskers show observed per-unit ranges, scaled by N for serial scenarios. They are not confidence intervals. Simultaneous completion depends on the maximum duration, not the median. The slowest recorded worker attempt took 54.1 min. We did not measure rate limits, machine limits or contention.",
        "sourceIds": [
          "calc-fan-out",
          "agent-routing-overhead",
          "agent-provider-h2h",
          "agent-swebench-c1",
          "agent-swebench-c2"
        ]
      }
    ],
    "tables": [
      {
        "id": "fan-out-measured-inputs",
        "title": "Per-unit inputs, costs and durations",
        "columns": [
          {
            "key": "workload",
            "label": "Workload",
            "unit": "text"
          },
          {
            "key": "n",
            "label": "n",
            "unit": "count"
          },
          {
            "key": "passed",
            "label": "Strict passes, with Wilson 95% interval",
            "unit": "text"
          },
          {
            "key": "inputTokens",
            "label": "Input tokens (median; worker mean)",
            "unit": "tokens"
          },
          {
            "key": "cost",
            "label": "List-price cost (calculation; worker mean, others median)",
            "unit": "usd"
          },
          {
            "key": "costRange",
            "label": "Observed cost range (calculation)",
            "unit": "text"
          },
          {
            "key": "seconds",
            "label": "Median wall time",
            "unit": "seconds"
          },
          {
            "key": "secondsRange",
            "label": "Observed time range",
            "unit": "text"
          }
        ],
        "rows": [
          {
            "workload": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
            "n": 5,
            "passed": "not scored here",
            "inputTokens": 6761,
            "cost": 0.01352,
            "costRange": "Token range absent; input-only scenario",
            "seconds": 2.53,
            "secondsRange": "2.27 s to 3.38 s"
          },
          {
            "workload": "Claude Code · Claude Sonnet 5.5",
            "n": 15,
            "passed": "12/15; 54.8% to 93.0%",
            "inputTokens": 2081,
            "cost": 0.0036,
            "costRange": "$0.0034 to $0.0102",
            "seconds": 2.31,
            "secondsRange": "2.17 s to 7.72 s"
          },
          {
            "workload": "Claude Code · Claude Opus 5.5",
            "n": 15,
            "passed": "15/15; 79.6% to 100.0%",
            "inputTokens": 2077,
            "cost": 0.00688,
            "costRange": "$0.0059 to $0.0223",
            "seconds": 2.75,
            "secondsRange": "2.47 s to 8.91 s"
          },
          {
            "workload": "Claude Code · Claude Haiku 4.5",
            "n": 15,
            "passed": "15/15; 79.6% to 100.0%",
            "inputTokens": 3789,
            "cost": 0.00566,
            "costRange": "$0.0051 to $0.0180",
            "seconds": 4.43,
            "secondsRange": "3.16 s to 23.57 s"
          },
          {
            "workload": "Claude Code · Claude Fable 5.1",
            "n": 15,
            "passed": "15/15; 79.6% to 100.0%",
            "inputTokens": 3229,
            "cost": 0.00987,
            "costRange": "$0.0049 to $0.0584",
            "seconds": 1.94,
            "secondsRange": "1.41 s to 9.83 s"
          },
          {
            "workload": "Codex CLI · GPT-6.1 Sol (medium)",
            "n": 15,
            "passed": "15/15; 79.6% to 100.0%",
            "inputTokens": 12124,
            "cost": 0.01018,
            "costRange": "$0.0054 to $0.0269",
            "seconds": 5.65,
            "secondsRange": "4.09 s to 25.46 s"
          },
          {
            "workload": "Full worker attempt · Crewshal · Sonnet 5.5",
            "n": 33,
            "passed": "not scored here",
            "inputTokens": 4935189,
            "cost": 2.8072,
            "costRange": "$0.89 to $4.83",
            "seconds": 575.4,
            "secondsRange": "1.6 min to 54.1 min"
          }
        ]
      },
      {
        "id": "fan-out-scaled",
        "title": "N × per-unit scenarios (calculation)",
        "columns": [
          {
            "key": "n",
            "label": "Assumed units (not sample n)",
            "unit": "count"
          },
          {
            "key": "workload",
            "label": "Workload",
            "unit": "text"
          },
          {
            "key": "sampleN",
            "label": "Measured sample n",
            "unit": "count"
          },
          {
            "key": "cost",
            "label": "List-price cost scenario",
            "unit": "usd"
          },
          {
            "key": "costRange",
            "label": "N × observed per-unit cost range",
            "unit": "text"
          },
          {
            "key": "serialSeconds",
            "label": "Serial scenario: N × median",
            "unit": "seconds"
          }
        ],
        "rows": [
          {
            "n": 10,
            "workload": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
            "sampleN": 5,
            "cost": 0.1352,
            "costRange": "Token range absent",
            "serialSeconds": 25.29
          },
          {
            "n": 10,
            "workload": "Claude Code · Claude Sonnet 5.5",
            "sampleN": 15,
            "cost": 0.036,
            "costRange": "$0.0342 to $0.10",
            "serialSeconds": 23.09
          },
          {
            "n": 10,
            "workload": "Claude Code · Claude Opus 5.5",
            "sampleN": 15,
            "cost": 0.0688,
            "costRange": "$0.0592 to $0.22",
            "serialSeconds": 27.45
          },
          {
            "n": 10,
            "workload": "Claude Code · Claude Haiku 4.5",
            "sampleN": 15,
            "cost": 0.0566,
            "costRange": "$0.0513 to $0.18",
            "serialSeconds": 44.28
          },
          {
            "n": 10,
            "workload": "Claude Code · Claude Fable 5.1",
            "sampleN": 15,
            "cost": 0.0987,
            "costRange": "$0.0490 to $0.58",
            "serialSeconds": 19.44
          },
          {
            "n": 10,
            "workload": "Codex CLI · GPT-6.1 Sol (medium)",
            "sampleN": 15,
            "cost": 0.1018,
            "costRange": "$0.0540 to $0.27",
            "serialSeconds": 56.49
          },
          {
            "n": 10,
            "workload": "Full worker attempt · Crewshal · Sonnet 5.5",
            "sampleN": 33,
            "cost": 28.072,
            "costRange": "$8.90 to $48.27",
            "serialSeconds": 5754
          },
          {
            "n": 100,
            "workload": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
            "sampleN": 5,
            "cost": 1.3522,
            "costRange": "Token range absent",
            "serialSeconds": 252.9
          },
          {
            "n": 100,
            "workload": "Claude Code · Claude Sonnet 5.5",
            "sampleN": 15,
            "cost": 0.3601,
            "costRange": "$0.34 to $1.02",
            "serialSeconds": 230.9
          },
          {
            "n": 100,
            "workload": "Claude Code · Claude Opus 5.5",
            "sampleN": 15,
            "cost": 0.6877,
            "costRange": "$0.59 to $2.23",
            "serialSeconds": 274.5
          },
          {
            "n": 100,
            "workload": "Claude Code · Claude Haiku 4.5",
            "sampleN": 15,
            "cost": 0.5657,
            "costRange": "$0.51 to $1.80",
            "serialSeconds": 442.8
          },
          {
            "n": 100,
            "workload": "Claude Code · Claude Fable 5.1",
            "sampleN": 15,
            "cost": 0.9867,
            "costRange": "$0.49 to $5.84",
            "serialSeconds": 194.4
          },
          {
            "n": 100,
            "workload": "Codex CLI · GPT-6.1 Sol (medium)",
            "sampleN": 15,
            "cost": 1.0178,
            "costRange": "$0.54 to $2.69",
            "serialSeconds": 564.9
          },
          {
            "n": 100,
            "workload": "Full worker attempt · Crewshal · Sonnet 5.5",
            "sampleN": 33,
            "cost": 280.7197,
            "costRange": "$88.98 to $482.74",
            "serialSeconds": 57540
          },
          {
            "n": 500,
            "workload": "Claude Code minimal reply (Haiku 4.5; Sonnet 5.5 price)",
            "sampleN": 5,
            "cost": 6.761,
            "costRange": "Token range absent",
            "serialSeconds": 1264.5
          },
          {
            "n": 500,
            "workload": "Claude Code · Claude Sonnet 5.5",
            "sampleN": 15,
            "cost": 1.8003,
            "costRange": "$1.71 to $5.11",
            "serialSeconds": 1154.5
          },
          {
            "n": 500,
            "workload": "Claude Code · Claude Opus 5.5",
            "sampleN": 15,
            "cost": 3.4383,
            "costRange": "$2.96 to $11.13",
            "serialSeconds": 1372.5
          },
          {
            "n": 500,
            "workload": "Claude Code · Claude Haiku 4.5",
            "sampleN": 15,
            "cost": 2.8285,
            "costRange": "$2.56 to $9.02",
            "serialSeconds": 2214
          },
          {
            "n": 500,
            "workload": "Claude Code · Claude Fable 5.1",
            "sampleN": 15,
            "cost": 4.9338,
            "costRange": "$2.45 to $29.22",
            "serialSeconds": 972
          },
          {
            "n": 500,
            "workload": "Codex CLI · GPT-6.1 Sol (medium)",
            "sampleN": 15,
            "cost": 5.089,
            "costRange": "$2.70 to $13.43",
            "serialSeconds": 2824.5
          },
          {
            "n": 500,
            "workload": "Full worker attempt · Crewshal · Sonnet 5.5",
            "sampleN": 33,
            "cost": 1403.5986,
            "costRange": "$444.91 to $2413.72",
            "serialSeconds": 287700
          }
        ]
      }
    ],
    "related": [
      "routing-overhead",
      "model-head-to-head",
      "swe-bench-verified"
    ]
  },
  "sources": [
    {
      "id": "agent-swebench-c1",
      "title": "Crewshal on SWE-bench Verified, campaign 1 (25 instances)",
      "kind": "run",
      "date": "2026-10-04",
      "note": "Stratified sample of 25 Verified instances (seed 20261004), one attempt each, official grading harness. Fixed model claude-sonnet-5-5, platform build f0ac3a8a.",
      "data": [
        "/benchmarks/raw/swebench/attempts.json",
        "/benchmarks/raw/swebench/exclusions.json"
      ]
    },
    {
      "id": "agent-swebench-c2",
      "title": "Crewshal on SWE-bench Verified, campaign 2 (8 compiled-extension instances)",
      "kind": "run",
      "date": "2026-10-05",
      "note": "The 6 compiled-extension instances that campaign 1 could not run, plus 2 replacement candidates. One attempt each, platform build 236c0d3f.",
      "data": [
        "/benchmarks/raw/swebench/attempts.json"
      ]
    },
    {
      "id": "agent-provider-h2h",
      "title": "Provider head-to-head: Claude Code models vs Codex efforts",
      "kind": "run",
      "date": "2026-10-05",
      "note": "Five short tasks with deterministic validators, declared protocol, every attempt kept.",
      "data": [
        "/benchmarks/raw/provider-h2h/receipts.json"
      ]
    },
    {
      "id": "agent-caching-consistency",
      "title": "Caching sessions and repeated prompts (Claude Code and Codex CLI)",
      "kind": "run",
      "date": "2026-10-06",
      "note": "Part 1: 5-turn CLI sessions over a fixed synthetic ledger, with the cache counters each provider reports per turn. Part 2: three prompts with deterministic validators, 10 repetitions per model. Declared protocols, validator controls before inference, every attempt kept; answers are published as ordinal ids, never as text.",
      "data": [
        "/benchmarks/raw/caching-consistency/caching.json",
        "/benchmarks/raw/caching-consistency/consistency.json"
      ]
    },
    {
      "id": "price-anthropic",
      "title": "Anthropic list prices (Claude models)",
      "kind": "price-list",
      "date": "2026-09-21",
      "url": "https://platform.claude.com/docs/en/about-claude/pricing",
      "note": "Prices as listed by the vendor on 2026-09-21 and recorded in the product price table. Cache reads at the listed rate, one-hour cache writes at twice the input price."
    },
    {
      "id": "price-openai",
      "title": "OpenAI list prices",
      "kind": "price-list",
      "date": "2026-10-03",
      "url": "https://developers.openai.com/api/docs/pricing",
      "note": "Token prices as listed by the vendor on 2026-10-03."
    },
    {
      "id": "agent-routing-overhead",
      "title": "Routing overhead runs: policy microbenchmark and CLI start-up",
      "kind": "run",
      "date": "2026-10-06",
      "note": "In-process timing of the deterministic routing policy (20,000 timed decisions), CLI start-up with a one-word prompt (5 runs per CLI), and decision counts read from recorded bench runs. LLM router timings are reused from the routing runs.",
      "data": [
        "/benchmarks/raw/routing-overhead/results.json"
      ]
    },
    {
      "id": "calc-fan-out",
      "title": "Fan-out cost and serial time scenarios (calculation)",
      "kind": "calculation",
      "date": "2026-10-06",
      "note": "N times recorded per-call median or worker mean cost. Serial duration uses N times median. Cache scenarios are assumptions, not bounds; one-unit medians are references, not parallel lower bounds. No parallel workload or capacity was measured.",
      "data": [
        "/benchmarks/raw/routing-overhead/results.json",
        "/benchmarks/raw/caching-consistency/caching.json",
        "/benchmarks/raw/provider-h2h/receipts.json",
        "/benchmarks/raw/swebench/attempts.json"
      ]
    }
  ]
}