Tasks · desk · Reformatting
Fall catalog into Shopify's product import
shopify-product-import · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
We're launching the online store. Turn the fall catalog into a Shopify product import using their template; Jess's notes have the rules. Save it as shopify_products.csv.
Workspace
- catalog_fall_2026.xlsx
- import_notes_from_jess.txt
- product_template.csv
- shopify_products_export_2025-11.csv
Expected deliverables
- shopify_products.csv
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| Shopify template columns in order | csv_columns | yes | 0 |
| one row per SKU | csv_set_equal | yes | 0 |
| row count | csv_row_count | yes | 0 |
| handles | csv_values_match | yes | 0 |
| option values | csv_values_match | yes | 0 |
| variant price | csv_values_match | yes | 1 |
| grams | csv_values_match | yes | 0 |
| inventory quantity | csv_values_match | yes | 0 |
| barcodes | csv_values_match | yes | 0 |
| first-row product fields and prices | custom | yes | 1 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 2 | 2 | 1.9 | 0.0274 | |
| Codex / GPT-5.6 sol | 3 | 3 | 2.4 | 0.2849 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "shopify-product-import".