Tasks · desk · Extraction
Key the August pay stubs for the CPA
pay-stubs-extraction · timeout 1800 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Nadia needs our August payroll detail from the pay stubs in the folder. Can you put it in pay_stubs.csv the way her email asks?
Workspace
- august_pay_stubs/Ledgerline_LL-00436.pdf
- august_pay_stubs/Ledgerline_LL-00437.pdf
- august_pay_stubs/PayCrest_Garcia_2026-08-07.pdf
- august_pay_stubs/PayCrest_Okafor_2026-07-24.pdf
- august_pay_stubs/PayCrest_Okafor_2026-08-07.pdf
- august_pay_stubs/PayCrest_Rogers_2026-08-07.pdf
- august_pay_stubs/manual_check_5532.pdf
- august_pay_stubs/scan_paystub_rogers_0821.pdf
- email_from_cpa.txt
- employee_list.csv
Expected deliverables
- pay_stubs.csv
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| requested columns | csv_columns | yes | 0 |
| one row per stub | csv_set_equal | yes | 0 |
| row count | csv_row_count | yes | 0 |
| employee and pay date | csv_values_match | yes | 0 |
| gross and net | csv_values_match | yes | 0 |
| bonus | csv_values_match | yes | 0 |
| taxes | csv_values_match | yes | 0 |
| pre-tax deductions | csv_values_match | yes | 0 |
| after-tax deductions | csv_values_match | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 2.7 | 0.0315 | |
| Codex / GPT-5.6 sol | 3 | 3 | 1.5 | 0.2874 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "pay-stubs-extraction".