Feature · Admin
Benchmarks
Compare providers and inspect benchmark evidence.
How it works
Admin overview
Admin overview links every administrator tool: benchmarks and evaluation, people and access, models and routing, governance and spend, and, for platform operators, platform operations.
Benchmarks
Benchmarks has two tabs: Provider benchmarks compares model providers and CLIs on the same cases; Agent harness shows harness runs. Paid runs need consent from an owner or admin.
Who can open it
Only owners and admins. Others see "Admin access required".
From the help article The Admin area, checked against the product on .
What you can do
The operations this feature's screens call. Each one is POST /api/<name>, and the badge says who may call it.
AdminNo API tokensharnessBenchmarks/listCasesHarness benchmarks: lists cases.
Types:
HarnessBenchmarkListCasesRequestreturnsHarnessBenchmarkListCasesResponse
AdminNo API tokensharnessBenchmarks/listRunsHarness benchmarks: lists runs.
Types:
HarnessBenchmarkListRunsRequestreturnsHarnessBenchmarkListRunsResponse
AdminNo API tokensharnessBenchmarks/saveCaseHarness benchmarks: saves the case.
Types:
HarnessBenchmarkSaveCaseRequestreturnsHarnessBenchmarkCase
AdminNo API tokensharnessBenchmarks/compareRunsHarness benchmarks: compares runs.
Types:
HarnessBenchmarkCompareRunsRequestreturnsHarnessBenchmarkComparison
AdminNo API tokensharnessBenchmarks/startRunHarness benchmarks: starts the run.
Types:
HarnessBenchmarkStartRunRequestreturnsHarnessBenchmarkStartRunResponse
AdminNo API tokensharnessBenchmarks/recordRunHarness benchmarks: records a run.
Types:
HarnessBenchmarkRecordRunRequestreturnsHarnessBenchmarkRun
Screens
/admin/benchmarks/*/admin/benchmarks/admin/benchmarks/provider/*/admin/benchmarks/harness
What it records
The feature writes these events to the audit log, the permanent record of what happened.
- Harness benchmark case version created
- Harness benchmark run recorded