Feature · Work
PR trials
Compare past pull requests by result, checks, time and estimated cost.
How it works
What a trial does
- You pick a merged GitHub.com pull request and give the original requirement.
- The runner on your computer rebuilds the repository's history up to before the change. The model does not see the merged change.
- Your own Claude Code or Codex builds a candidate within the turn, time and cost caps you set.
- When the candidate is frozen, the checks run without a model on separate copies of the base, the candidate and the reference.
- You compare the two diffs and record a verdict: Candidate preferred, Reference preferred, Comparable or Inconclusive.
What you need
- Agent runner 0.5 or newer on macOS, with a working sandbox;
- a merged pull request on GitHub.com;
- checks that run offline with the tools you already have.
Statuses
Ready to run, Queued, Running, Verifying, Completed, Failed and Cancelled.
Cost
The trial uses your existing CLI login on your computer. Agent does not give model credits for it. Costs shown are estimates, not bills.
From the help article PR trials, checked against the product on .
What you can do
The operations this feature's screens call. Each one is POST /api/<name>, and the badge says who may call it.
MemberNo API tokensprTrials/preparePR trials: prepare.
Types:
PreparePrTrialRequestreturns{ trial: PrTrialDto }
MemberNo API tokensprTrials/startPR trials: start.
Types:
StartPrTrialRequestreturns{ trial: PrTrialDto }
MembergetRunnerTaskGets the runner task.
Types:
GetRunnerTaskRequestreturnsRunnerTaskDetailDto
MemberNo API tokensprTrials/reviewPR trials: review.
Types:
{ id: string; verdict: PrTrialVerdict; rationale: string; humanMinutes: number }returns{ trial: PrTrialDto }
MemberNo API tokensprTrials/cancelPR trials: cancel.
Types:
{ id: string }returns{ trial: PrTrialDto }
MemberprTrials/exportPR trials: export.
Types:
{ id: string }returns{ report: PrTrialDto; protocolVersion: string }
MemberprTrials/readinessPR trials: readiness.
Types:
Record<string, never>returnsPrTrialReadiness
MemberprTrials/listPR trials: list.
Types:
Record<string, never>returns{ trials: PrTrialDto[] }
MemberNo API tokensprTrials/deletePR trials: delete.
Types:
{ id: string }returns{ deleted: true }
MemberprTrials/getPR trials: get.
Types:
{ id: string }returns{ trial: PrTrialDto }
Screens
What it records
The feature writes these events to the audit log, the permanent record of what happened.
- Integration changed