Tasks · desk · Bookkeeping
2026 deferred revenue schedule for the annual plans
deferred-revenue-schedule · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Graham needs the 2026 deferred revenue schedule for the year-end file, and his email in the folder says how he wants it done. Save it as deferred_revenue.xlsx.
Workspace
- billing_change_log_2026.csv
- billing_contracts_export.xlsx
- email_from_graham_cpa.txt
Expected deliverables
- deferred_revenue.xlsx
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| live formulas | xlsx_has_formulas | yes | 0 |
| no formula errors | xlsx_no_errors | yes | 1 |
| revenue recognized, one line per month | custom | yes | 1 |
| total recognized in 2026 | xlsx_value_present | yes | 1 |
| deferred balance at year end | xlsx_value_present | yes | 1 |
| late-start contract | xlsx_value_present | yes | 1 |
| carried-over contract | xlsx_value_present | yes | 0 |
| upgraded contract | xlsx_value_present | yes | 2 |
| cancelled contract | xlsx_value_present | yes | 2 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 2 | 9.2 | 0.1034 | |
| Codex / GPT-5.6 sol | 1 | 1 | 7.7 | 0.7401 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "deferred-revenue-schedule".