Match bank payments to open invoices, with card fees, short wires, and a wrong reference
The request
The task request supplied to the adapter, alongside the workspace below. Runtime instructions, authorized tools and skills also affect the system. Containerized runs withhold host-side references; local mode is not an isolation boundary.
Match the payments in this morning's bank export to our open invoices. Give me unpaid.csv with the invoices still open (invoice_id, customer, amount_outstanding), matches.csv showing what got applied where (invoice_id, line_id, amount_applied), and unapplied.csv for any credit that is not a payment on an open invoice (line_id, amount, reason). My note in the folder has the house rules.
Workspace
- bank_export.csv
- note.txt
- open_invoices.csv
Expected deliverables
- unapplied.csv
- unpaid.csv
This task also has a task-specific grader (check.py).
Recorded acceptance checks
The table uses check names and outcomes from the frozen campaign ledger. Acceptance requires all required predicates to pass; a predicate may itself contain a tolerance or coverage threshold. Its scope is part of the measurement contract.
| Recorded check | Type | Required | Failed / observed |
|---|---|---|---|
| unpaid.csv columns | csv_columns | yes | 0 / 6 |
| unpaid invoice ids | csv_set_equal | yes | 0 / 6 |
| amount outstanding per open invoice (partial remainder required) | csv_values_match | yes | 0 / 6 |
| matches.csv applies payments correctly | custom | yes | 0 / 6 |
| unapplied.csv columns | csv_columns | yes | 0 / 6 |
| unapplied line ids (refund, interest, closed-invoice payment; nothing else) | csv_set_equal | yes | 0 / 6 |
Current task declaration (6 checks)
From task.yaml. Original runner grading and frozen retrospective scoring are separate. Read the frozen source for the rules behind the reported results.
- unpaid.csv columns —
csv_columns, required - unpaid invoice ids —
csv_set_equal, required - amount outstanding per open invoice (partial remainder required) —
csv_values_match, required - matches.csv applies payments correctly —
custom, required - unapplied.csv columns —
csv_columns, required - unapplied line ids (refund, interest, closed-invoice payment; nothing else) —
csv_set_equal, required
Recorded results, desk release
| System | Repetitions | Frozen / 3 | Original / 3 | Median minutes | Mean model estimate (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 2.2 | 0.0268 | |
| Codex / GPT-5.6 sol | 3 | 3 | 2.4 | 0.2407 |
These are repeated executions on the released fixture, not independent new business cases. Cost is captured model usage priced under the recorded assumptions, excluding infrastructure and human review. Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "payments-match-v2".
Recorded results, desk comparison, 28 September 2026
| System | Repetitions | Frozen / 3 | Original / 3 | Median minutes |
|---|---|---|---|---|
| Proto / GPT-6 sol | 2 | 2 | 1.9 | |
| Codex / GPT-6 sol | 1 | 1 | 1.3 | |
| Proto / DeepSeek V4.1 Flash | 1 | 1 | 1.2 |
A separate campaign under the same frozen scorer, not pooled with the release. Conditions and data.