9 / 9
Scripted valid alternatives
Nine exercised alternative solutions earned full credit.
Evidence
Results you can
trace back to the work.
Inspect recorded actions, business records and outcome checks. Start with the original finance practice batch.
20original practice runs
11passed
9failed
Original R5 practice batch
High-effort runs across cash application, month-end close and full-quarter work.
| TASK / MODEL | PASSED | ATTEMPTS |
|---|---|---|
| Cash application · GPT-6.1 Sol | 5 | 5 |
| Month-end close · GPT-6.1 Sol | 3 | 5 |
| Quarter & audit · GPT-6.1 Sol | 1 | 5 |
| Quarter & audit · GPT-6 Astra | 2 | 5 |
Evaluation details
Original R5 practice seeds 50000–50004, at high effort. These are the retained results from before the harness repair; the model batch has not been rerun on the repaired harness. Cohorts differ by task and model.
SELECTED CASE / FULL QUARTER
PASSING CASE / QUARTER & AUDIT
A correct invoice left unpaid.
The trace follows the supplier notice, the invoice and the blocked payment.
PASSING CASE / QUARTER & AUDIT
The books closed. The checks held.
Follow reconciliations, attestations, quarter close and audit replies.
Environment checks
The scoring needs
to earn trust, too.
0
No work, no credit
The completed no-work control scored zero.
26
Repair checks
Targeted tests checked a repaired grading loophole.
Check populations and retained failures
The alternative-answer work retained 16 local attempts: 15 completed and one incomplete technical attempt. Completed attempts included five baselines, nine exercised alternatives and one historical zero-target case. The no-work control is separate. The 26 passing repair regressions are tests, not model runs. The underlying repair evidence also retains five Harbor control trials, four expected outcomes and one initial setup failure, alongside a separate reader mistake.
Bring your evaluation question.
Define what your agent needs to do, the tools it uses, and what a correct outcome looks like.