Draft the standing order price increase notice
price-increase-notice · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Please draft the price increase notice for our standing order customers and save it as notice.md. Grace's emails and the new price schedule are in the folder.
Workspace
- email_price_increase.txt
- new_price_schedule_2026.xlsx
- price_notice_2025.md
- standing_order_customers.csv
Expected deliverables
- notice.md
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| effective date | text_sentence_matches | yes | 0 |
| counter arrangement: new price and rounded percent | text_sentence_matches | yes | 0 |
| lobby arrangement: new price and rounded percent | text_sentence_matches | yes | 3 |
| statement arrangement: new price and rounded percent | text_sentence_matches | yes | 0 |
| delivery fee: new price and rounded percent | text_sentence_matches | yes | 0 |
| current prices stated | text_numbers_present | yes | 0 |
| bud vase sets discontinued | text_sentence_matches | yes | 0 |
| no discount code | text_not_contains | yes | 1 |
| mailing note names the annual-contract customers | custom | yes | 1 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 2 | 1 | 1.1 | 0.0130 | |
| Codex / GPT-5.6 sol | 0 | 0 | 1.5 | 0.1405 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "price-increase-notice".