Monthly revenue report by region from a messy sales export
The request
The task request supplied to the adapter, alongside the workspace below. Runtime instructions, authorized tools and skills also affect the system. Containerized runs withhold host-side references; local mode is not an isolation boundary.
Build me a monthly revenue report by region from this export - regions down the side, months across - as an Excel file with live formulas (report.xlsx), plus a short memo.md with the three things I should know.
Workspace
- sales_export.csv
Expected deliverables
- memo.md
- report.xlsx
Recorded acceptance checks
The table uses check names and outcomes from the frozen campaign ledger. Acceptance requires all required predicates to pass; a predicate may itself contain a tolerance or coverage threshold. Its scope is part of the measurement contract.
| Recorded check | Type | Required | Failed / observed |
|---|---|---|---|
| report.xlsx exists | file_exists | yes | 0 / 6 |
| report uses live formulas | xlsx_has_formulas | yes | 0 / 6 |
| no formula evaluates to an error | xlsx_no_errors | yes | 1 / 6 |
| North half-year total (five months, no invented March) | xlsx_value_present | yes | 1 / 6 |
| South half-year total | xlsx_value_present | yes | 1 / 6 |
| East half-year total | xlsx_value_present | yes | 1 / 6 |
| West half-year total | xlsx_value_present | yes | 1 / 6 |
| South February (contains a duplicate and a refund) | xlsx_value_present | yes | 1 / 6 |
| West June (contains a duplicate and a refund) | xlsx_value_present | yes | 1 / 6 |
| one sentence says North has no March data (a gap, not zero sales) | text_sentence_matches | yes | 0 / 6 |
| memo mentions duplicates or refunds | text_contains_any | yes | 0 / 6 |
Current task declaration (11 checks)
From task.yaml. Original runner grading and frozen retrospective scoring are separate. Read the frozen source for the rules behind the reported results.
- report.xlsx exists —
file_exists, required - report uses live formulas —
xlsx_has_formulas, required - no formula evaluates to an error —
xlsx_no_errors, required - North half-year total (five months, no invented March) —
xlsx_value_present, required - South half-year total —
xlsx_value_present, required - East half-year total —
xlsx_value_present, required - West half-year total —
xlsx_value_present, required - South February (contains a duplicate and a refund) —
xlsx_value_present, required - West June (contains a duplicate and a refund) —
xlsx_value_present, required - one sentence says North has no March data (a gap, not zero sales) —
text_sentence_matches, required - memo mentions duplicates or refunds —
text_contains_any, required
Recorded results, desk release
| System | Repetitions | Frozen / 3 | Original / 3 | Median minutes | Mean model estimate (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 2.5 | 0.0396 | |
| Codex / GPT-5.6 sol | 2 | 2 | 5.0 | 0.5271 |
These are repeated executions on the released fixture, not independent new business cases. Cost is captured model usage priced under the recorded assumptions, excluding infrastructure and human review. Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "monthly-report".
Recorded results, desk comparison, 28 September 2026
| System | Repetitions | Frozen / 3 | Original / 3 | Median minutes |
|---|---|---|---|---|
| Proto / GPT-6 sol | 0 | 0 | 1.9 | |
| Codex / GPT-6 sol | 3 | 3 | 3.1 | |
| Proto / DeepSeek V4.1 Flash | 1 | 1 | 1.3 |
A separate campaign under the same frozen scorer, not pooled with the release. Conditions and data.