Tasks · desk · Spreadsheet
Active subscribers and cancellations by month
subscription-status-monthly · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
How many coffee subscribers did we have each month this year, January to June, and how many cancelled? Work it out from the billing export and the help desk tickets and save it as subscriptions.xlsx with the counts as formulas. My note explains how I count.
Workspace
- billing_invoices_2025-10-01_2026-07-05.csv
- helpdesk_cancellation_tickets.csv
- note_from_owner.txt
Expected deliverables
- subscriptions.xlsx
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| live formulas | xlsx_has_formulas | yes | 0 |
| no error cells | xlsx_no_errors | yes | 2 |
| active and cancelled by month | custom | yes | 3 |
| June active subscribers | xlsx_value_present | yes | 2 |
| April cancellations | xlsx_value_present | yes | 3 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 4.8 | 0.0510 | |
| Codex / GPT-5.6 sol | 0 | 0 | 5.1 | 0.5826 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "subscription-status-monthly".