Tasks · desk · Extraction
Compare three laptop quotes (USD/EUR, hidden fees) and recommend one
quote-comparison · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Compare these three quotes for the 40 laptops and tell me which to pick. Put the numbers in comparison.xlsx and a short recommendation in recommendation.md.
Workspace
- notes.txt
- quote_bluepine.pdf
- quote_kestrel_gmbh.pdf
- quote_northstar.pdf
Expected deliverables
- comparison.xlsx
- recommendation.md
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| comparison.xlsx exists | file_exists | yes | 0 |
| Northstar delivered total (incl. freight, no early-pay discount) | xlsx_value_present | yes | 0 |
| no formula evaluates to an error | xlsx_no_errors | yes | 0 |
| Kestrel delivered total in USD (40+ tier x 1.08, shipping included) | xlsx_value_present | yes | 0 |
| Bluepine delivered total (incl. mandatory onboarding fee and freight) | xlsx_value_present | yes | 0 |
| memo prices the vendor it recommends | custom | yes | 0 |
| memo weighs warranty | text_contains_all | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 5.9 | 0.0744 | |
| Codex / GPT-5.6 sol | 3 | 3 | 3.2 | 0.3401 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "quote-comparison".