Renewal summary for the Wi-Fi services contract
contract-renewal-summary · timeout 1800 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Timothy wants a renewal summary of our Brightline Wi-Fi contract before we decide whether to go out to bid; the note in the folder says what is needed. Save it as renewal_summary.md.
Workspace
- contracts/Amendment_1_Driftwood_Cottages.pdf
- contracts/Amendment_2_signed.pdf
- contracts/Amendment_3_DRAFT_for_discussion.pdf
- contracts/Brightline_MSA_Saltmarsh_executed.pdf
- contracts/Go-Live_Certificate.pdf
- email_from_brightline.txt
- note_from_owner.txt
Expected deliverables
- renewal_summary.md
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| renewal_summary.md exists | file_exists | yes | 0 |
| current annual value | text_numbers_present | yes | 0 |
| term, notice and the vendor's figures | custom | yes | 1 |
| automatic renewal | text_sentence_matches | yes | 0 |
| renewal price cap | text_sentence_matches | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 2 | 1 | 0.8 | 0.0114 | |
| Codex / GPT-5.6 sol | 3 | 3 | 1.5 | 0.1406 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "contract-renewal-summary".