3 measured metrics · 4 calculated · 1 study
Jev 1.13vsDeterministic routing policy
Deterministic routing policy ahead on 1; 1 tie, 5 unclear. A side is ahead only where the intervals or ranges do not overlap.
The verdict
Jev 1.13 and Deterministic routing policy share 3 measured metrics and 4 list-price calculations from one study. Deterministic routing policy leads on 1 row: Time to make one routing decision, 1.42 µs vs 137 ms. On those rows the p50–p95 bands do not overlap; only a 95% interval is a confidence interval. The other rows are 1 tie and 5 unclear; each row says why. Calculation rows are derived from list prices and recorded counts; they are not bills or runs.
Headline metrics
How far apart the two sides are on the headline metrics. Length is the ratio of the two values; it is not a winner.
Bar length is the ratio of the two values on a log scale, pointing to the larger one. Larger is not better for time, tokens or cost. A bar has a side’s color only when that side is ahead in the data; gray means the data does not separate them.
| Metric | Jev 1.13 | Deterministic routing policy | n | Interval or range | Outcome | Basis | Study |
|---|---|---|---|---|---|---|---|
| Time to make one routing decision | 137 msrouting overhead per decision · TypeSafe API | 1.42 µsAgent · in process · routing overhead per decision | 246 / 20000 | p50–p95: 137 ms–196 ms vs 1.42 µs–2.33 µs | Deterministic routing policy ahead | Jev 1.13’s median is above Deterministic routing policy’s 95th percentile (p50–p95 bands: Jev 1.13 137 ms to 196 ms; Deterministic routing policy 1.42 µs to 2.33 µs); not a confidence interval. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Routing calls that returned a decision | 100% (246/246)routing overhead per decision · TypeSafe API | 100% (20000/20000)Agent · in process · routing overhead per decision | 246 / 20000 | 95% CI: 98%–100% vs 100%–100% | Tie | The 95% intervals overlap (Jev 1.13 98% to 100%; Deterministic routing policy 100% to 100%), so this sample cannot separate them. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Added routing delay per task (calculation) (Every model call routed (49.5 per task))Calculation | 6.76 scalculation per task from recorded decision counts, decisions in line · TypeSafe API | 70.3 µsAgent · in process · calculation per task from recorded decision counts, decisions in line | — | none recorded | Unclear | No interval or range was recorded for either side, so the gap (6.76 s vs 70.3 µs, 96,114x) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Added routing delay per task (calculation) (Only System One decisions (7 per task))Calculation | 0.96 scalculation per task from recorded decision counts, decisions in line · TypeSafe API | 9.9 µsAgent · in process · calculation per task from recorded decision counts, decisions in line | — | none recorded | Unclear | No interval or range was recorded for either side, so the gap (0.96 s vs 9.9 µs, 96,515x) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
Marks: 95% intervals (Wilson for rates); median to p95 bands (not intervals)n is shown per side on every rowHollow marks: list-price calculations, not runs
4 headline metrics as the ratio of the two values. 1 of them separate the sides in the data. Widest ratio: Added routing delay per task (calculation) (Only System One decisions (7 per task)), 97 thousand x (Jev 1.13 larger).
Metric by metric
Both values of a row come from the same chart in the same study. Each row has its own axis. The shaded band is where the two intervals or ranges overlap: a side is ahead only when they do not.
- Jev 1.13
- Deterministic routing policy
- 95% interval
- median to p95 (not an interval)
- where the two overlap
- hollow: list-price calculation
Routing overhead: deterministic policy vs LLM routers vs Jev
- Time to make one routing decision137 msn 2461.42 µsn 20000Deterministic routing policy aheadTime to make one routing decision: Jev 1.13 137 ms (n 246, median to p95 137 ms–196 ms); Deterministic routing policy 1.42 µs (n 20000, median to p95 1.42 µs–2.33 µs). Deterministic routing policy ahead.
- Routing calls that returned a decision100% (246/246)n 246100% (20000/20000)n 20000TieRouting calls that returned a decision: Jev 1.13 100% (246/246) (n 246, 95% interval 98%–100%); Deterministic routing policy 100% (20000/20000) (n 20000, 95% interval 100%–100%). Tie.
- Cost per 1,000 routing decisions: no model call vs provider-reported$0.034n 82$0.00n 20000UnclearCost per 1,000 routing decisions: no model call vs provider-reported: Jev 1.13 $0.034 (n 82); Deterministic routing policy $0.00 (n 20000). Unclear.
- Added routing cost per 1,000 tasks (calculation) (Every model call routed (49.5 per task))Calculation$1.67$0.00UnclearAdded routing cost per 1,000 tasks (calculation) (Every model call routed (49.5 per task)), calculation: Jev 1.13 $1.67; Deterministic routing policy $0.00. Unclear.
- Added routing cost per 1,000 tasks (calculation) (Only System One decisions (7 per task))Calculation$0.24$0.00UnclearAdded routing cost per 1,000 tasks (calculation) (Only System One decisions (7 per task)), calculation: Jev 1.13 $0.24; Deterministic routing policy $0.00. Unclear.
- Added routing delay per task (calculation) (Every model call routed (49.5 per task))Calculation6.76 s70.3 µsUnclearAdded routing delay per task (calculation) (Every model call routed (49.5 per task)), calculation: Jev 1.13 6.76 s; Deterministic routing policy 70.3 µs. Unclear.
- Added routing delay per task (calculation) (Only System One decisions (7 per task))Calculation0.96 s9.9 µsUnclearAdded routing delay per task (calculation) (Only System One decisions (7 per task)), calculation: Jev 1.13 0.96 s; Deterministic routing policy 9.9 µs. Unclear.
| Metric | Jev 1.13 | Deterministic routing policy | n | Interval or range | Outcome | Basis | Study |
|---|---|---|---|---|---|---|---|
| Time to make one routing decision | 137 msrouting overhead per decision · TypeSafe API | 1.42 µsAgent · in process · routing overhead per decision | 246 / 20000 | p50–p95: 137 ms–196 ms vs 1.42 µs–2.33 µs | Deterministic routing policy ahead | Jev 1.13’s median is above Deterministic routing policy’s 95th percentile (p50–p95 bands: Jev 1.13 137 ms to 196 ms; Deterministic routing policy 1.42 µs to 2.33 µs); not a confidence interval. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Routing calls that returned a decision | 100% (246/246)routing overhead per decision · TypeSafe API | 100% (20000/20000)Agent · in process · routing overhead per decision | 246 / 20000 | 95% CI: 98%–100% vs 100%–100% | Tie | The 95% intervals overlap (Jev 1.13 98% to 100%; Deterministic routing policy 100% to 100%), so this sample cannot separate them. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Cost per 1,000 routing decisions: no model call vs provider-reported | $0.034routing overhead per decision · TypeSafe API | $0.00Agent · in process · routing overhead per decision | 82 / 20000 | none recorded | Unclear | No interval or range was recorded for either side, so the gap ($0.034 vs $0.00) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Added routing cost per 1,000 tasks (calculation) (Every model call routed (49.5 per task))Calculation | $1.67calculation per 1,000 tasks from recorded decision counts · TypeSafe API | $0.00Agent · in process · calculation per 1,000 tasks from recorded decision counts | — | none recorded | Unclear | No interval or range was recorded for either side, so the gap ($1.67 vs $0.00) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Added routing cost per 1,000 tasks (calculation) (Only System One decisions (7 per task))Calculation | $0.24calculation per 1,000 tasks from recorded decision counts · TypeSafe API | $0.00Agent · in process · calculation per 1,000 tasks from recorded decision counts | — | none recorded | Unclear | No interval or range was recorded for either side, so the gap ($0.24 vs $0.00) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Added routing delay per task (calculation) (Every model call routed (49.5 per task))Calculation | 6.76 scalculation per task from recorded decision counts, decisions in line · TypeSafe API | 70.3 µsAgent · in process · calculation per task from recorded decision counts, decisions in line | — | none recorded | Unclear | No interval or range was recorded for either side, so the gap (6.76 s vs 70.3 µs, 96,114x) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
| Added routing delay per task (calculation) (Only System One decisions (7 per task))Calculation | 0.96 scalculation per task from recorded decision counts, decisions in line · TypeSafe API | 9.9 µsAgent · in process · calculation per task from recorded decision counts, decisions in line | — | none recorded | Unclear | No interval or range was recorded for either side, so the gap (0.96 s vs 9.9 µs, 96,515x) is not tested against run-to-run variation. | Routing overhead: deterministic policy vs LLM routers vs Jev |
Marks: 95% intervals (Wilson for rates); median to p95 bands (not intervals)n is shown per side on every rowHollow marks: list-price calculations, not runs
7 rows from 1 study. Deterministic routing policy ahead on 1; 1 tie, 5 unclear. A side is ahead only where the intervals or ranges do not overlap.
When to pick which
Only from the rows above. A tie is not a reason to pick either side.
When to pick Jev 1.13
No row in this data puts Jev 1.13 ahead of Deterministic routing policy. Pick on other grounds (price, access, the tasks you run), or measure your own workload.
When to pick Deterministic routing policy
- Time to make one routing decision: 1.42 µs vs 137 ms. Jev 1.13’s median is above Deterministic routing policy’s 95th percentile (p50–p95 bands: Jev 1.13 137 ms to 196 ms; Deterministic routing policy 1.42 µs to 2.33 µs); not a confidence interval.
- Added routing cost per 1,000 tasks (calculation) (Every model call routed (49.5 per task)): $0.00 vs $1.67. A list-price calculation, not a measured difference. Calculation
- Added routing cost per 1,000 tasks (calculation) (Only System One decisions (7 per task)): $0.00 vs $0.24. A list-price calculation, not a measured difference. Calculation
- Added routing delay per task (calculation) (Every model call routed (49.5 per task)): 70.3 µs vs 6.76 s. A list-price calculation, not a measured difference. Calculation
- Added routing delay per task (calculation) (Only System One decisions (7 per task)): 9.9 µs vs 0.96 s. A list-price calculation, not a measured difference. Calculation
Side by side
The study charts, showing only these two. Open a study for every configuration.
1st, 2nd …: place by median, given only to a lane whose run range overlaps no other lane’s. A lane marked ~ overlaps another lane’s range, so it gets no place.
Time per decision · log scale: each gridline is 10 times the one before
| Item | Decision time | Median to p95 | n |
|---|---|---|---|
| Deterministic routing policy (Agent, in process) | 1.42 µs | 1.42 µs–2.33 µs | 20000 |
| Jev 1.13 (TypeSafe) | 137 ms | 137 ms–196 ms | 246 |
2 rows. Slowest Jev 1.13 (TypeSafe) 137 ms (median to p95 137 ms–196 ms, n 246). Fastest Deterministic routing policy (Agent, in process) 1.42 µs (median to p95 1.42 µs–2.33 µs, n 20000). Not all run ranges overlap.
NotesLines: median to p95 (not an interval)n 246–20000 per row
Median; whiskers = median to 95th percentile
The policy is the pure in-process decision (median 1.42 µs, p95 2.33 µs), timed over 20,000 decisions after 5,000 warm-up calls. The Claude routers are wall time per call through the Claude Code CLI, one call at a time, from the recorded routing runs. Jev is wall time of a direct HTTPS call from the same Mac over a home network (246 calls in a 35-second window; its API reports no server time), so the network is inside it. A different route from the CLI, so the chart shows what a caller waits, not model compute time. The whisker is the median to the 95th percentile, not a confidence interval.
Sources: Routing overhead runs: policy microbenchmark and CLI start-up, Routing runs: Jev router vs LLM routing, Jev live run: 246 timed calls on the 82 routing decisions
| Item | Completed | 95% interval | n |
|---|---|---|---|
| Deterministic routing policy (Agent, in process) | 100% | 100%–100% | 20000 |
| Jev 1.13 (TypeSafe) | 100% | 98%–100% | 246 |
2 rows. All at 100%.
NotesWhiskers: 95% Wilson intervaln 246–20000 per row
Completed calls ÷ calls; whiskers = 95% Wilson interval
A completed call returned a decision, right or wrong (accuracy is in the routing study). Whiskers are 95% Wilson intervals.
Sources: Routing overhead runs: policy microbenchmark and CLI start-up, Routing runs: Jev router vs LLM routing, Jev live run: 246 timed calls on the 82 routing decisions
| Item | Cost per 1,000 decisions | n |
|---|---|---|
| Deterministic routing policy (Agent, in process) | $0 | 20000 |
| Jev 1.13 (TypeSafe) | $0.034 | 82 |
2 rows. Highest Jev 1.13 (TypeSafe) $0.034 (n 82). Lowest Deterministic routing policy (Agent, in process) $0 (n 20000).
Notesn 82–20000 per row
USD per 1,000 decisions
The policy makes no model call, so it costs nothing per decision. Jev’s figure is the cost its provider reported for the recorded production run (82 decisions). The input tokens of the live run give the same figure at the published price. The Claude routers are in the next chart: their cost is derived from list prices.
Sources: Routing overhead runs: policy microbenchmark and CLI start-up, Routing runs: Jev router vs LLM routing, Jev 1.13 list price
- Every model call routed (49.5 per task)
- Only System One decisions (7 per task) (square)
Gap labels, Only System One decisions (7 per task) vs Every model call routed (49.5 per task): Only System One decisions (7 per task) is x% higher (+) or lower (−) than Every model call routed (49.5 per task), calculated from the two values shown (the change counted from Every model call routed (49.5 per task)’s value).
| Item | Every model call routed (49.5 per task) | Only System One decisions (7 per task) |
|---|---|---|
| Deterministic routing policy (Agent, in process) | $0 | $0 |
| Jev 1.13 (TypeSafe) | $1.67 | $0.24 |
List-price calculation, not a run. 2 rows, 2 series: Every model call routed (49.5 per task), Only System One decisions (7 per task). Every model call routed (49.5 per task): highest Jev 1.13 (TypeSafe) $1.67. Lowest Deterministic routing policy (Agent, in process) $0. Only System One decisions (7 per task): highest Jev 1.13 (TypeSafe) $0.24. Lowest Deterministic routing policy (Agent, in process) $0.
Notes
Decisions per task from recorded runs × cost per decision
A calculation. Decisions per task: the median of 48 recorded bench runs (routing was off in them, so every model call counts as one decision a router would make). Median recorded work cost per task: $3.03. Claude router costs are list-price calculations; Jev’s is a list-price calculation too (its recorded run’s provider-reported cost is the same).
Sources: Routing overhead runs: policy microbenchmark and CLI start-up, Routing overhead per 1,000 tasks (calculation), Routing runs: Jev router vs LLM routing, Anthropic list prices (Claude models), Jev 1.13 list price, Jev live run: 246 timed calls on the 82 routing decisions
- Every model call routed (49.5 per task)
- Only System One decisions (7 per task) (square)
Gap labels, Only System One decisions (7 per task) vs Every model call routed (49.5 per task): Only System One decisions (7 per task) is x% higher (+) or lower (−) than Every model call routed (49.5 per task), calculated from the two values shown (the change counted from Every model call routed (49.5 per task)’s value).
| Item | Every model call routed (49.5 per task) | Only System One decisions (7 per task) |
|---|---|---|
| Deterministic routing policy (Agent, in process) | 70.3 µs | 9.9 µs |
| Jev 1.13 (TypeSafe) | 6.8 s | 1 s |
List-price calculation, not a run. 2 rows, 2 series: Every model call routed (49.5 per task), Only System One decisions (7 per task). Every model call routed (49.5 per task): slowest Jev 1.13 (TypeSafe) 6.8 s. Fastest Deterministic routing policy (Agent, in process) 70.3 µs. Only System One decisions (7 per task): slowest Jev 1.13 (TypeSafe) 1 s. Fastest Deterministic routing policy (Agent, in process) 9.9 µs.
Notes
Decisions per task × median decision time, if every decision waits in line
A calculation and an upper bound: it assumes each decision waits for the one before. Median recorded task wall time: 10.3 min. Jev’s delay uses its live median over the API from one Mac (network included); the Claude routers’ includes the CLI.
Sources: Routing overhead runs: policy microbenchmark and CLI start-up, Routing overhead per 1,000 tasks (calculation), Routing runs: Jev router vs LLM routing, Anthropic list prices (Claude models), Jev 1.13 list price, Jev live run: 246 timed calls on the 82 routing decisions
How a row is called
- AheadThe 95% intervals do not overlap, or the run ranges or p50–p95 bands do not overlap with at least 5 runs per side.
- TieThe values match, or both sit at the same ceiling.
- UnclearThe intervals or ranges overlap, too few runs were recorded, no interval was recorded, or more is not better (token counts are never a win).
- CalculationDerived from list prices and recorded counts. Not a bill and not a run.
The dataset compiler makes every call; this page only draws it. No composite score, no rank.
Questions
- Which is better, Jev 1.13 or Deterministic routing policy?
- Jev 1.13 and Deterministic routing policy share 3 measured metrics and 4 list-price calculations from one study. Deterministic routing policy leads on 1 row: Time to make one routing decision, 1.42 µs vs 137 ms. On those rows the p50–p95 bands do not overlap; only a 95% interval is a confidence interval. The other rows are 1 tie and 5 unclear; each row says why. Calculation rows are derived from list prices and recorded counts; they are not bills or runs.
- How were Jev 1.13 and Deterministic routing policy measured?
- They share 3 measured metrics and 4 list-price calculations from 1 public study: Routing overhead: deterministic policy vs LLM routers vs Jev. Every row names its configuration, its sample size and its interval or range.
- How do Jev 1.13 and Deterministic routing policy compare on time to make one routing decision?
- Jev 1.13: 137 ms (routing overhead per decision · TypeSafe API; n = 246; p50 to p95 137 ms to 196 ms). Deterministic routing policy: 1.42 µs (Agent · in process · routing overhead per decision; n = 20000; p50 to p95 1.42 µs to 2.33 µs). Jev 1.13’s median is above Deterministic routing policy’s 95th percentile (p50–p95 bands: Jev 1.13 137 ms to 196 ms; Deterministic routing policy 1.42 µs to 2.33 µs); not a confidence interval.
- How do Jev 1.13 and Deterministic routing policy compare on routing calls that returned a decision?
- Jev 1.13: 100% (246/246) (routing overhead per decision · TypeSafe API; n = 246; 95% interval 98% to 100%). Deterministic routing policy: 100% (20000/20000) (Agent · in process · routing overhead per decision; n = 20000; 95% interval 100% to 100%). The 95% intervals overlap (Jev 1.13 98% to 100%; Deterministic routing policy 100% to 100%), so this sample cannot separate them.
- How do Jev 1.13 and Deterministic routing policy compare on cost per 1,000 routing decisions: no model call vs provider-reported?
- Jev 1.13: $0.034 (routing overhead per decision · TypeSafe API; n = 82). Deterministic routing policy: $0.00 (Agent · in process · routing overhead per decision; n = 20000). No interval or range was recorded for either side, so the gap ($0.034 vs $0.00) is not tested against run-to-run variation.
The studies behind this page
Routing overhead: deterministic policy vs LLM routers vs Jev
How much delay and cost a router adds per decision: an in-process policy, Claude routers through a CLI, and Jev. Plus CLI start-up tax and per-task totals.