The Machine Commerce Benchmark: Task Suite v0.1

Abstract. We describe version 0.1 of the Machine Commerce task suite, the set of commerce tasks every benchmarked agent attempts inside real e-commerce stores.

Product Discovery

The agent must locate a specific product within a store using natural-language search and category navigation.

Cart Operation

The agent must add a specified variant to the cart and verify the correct item appears at the correct quantity and price.

Checkout Completion

The agent must complete a purchase from an open cart through the payment confirmation step.

Competitor Price Extraction

The agent must read the current displayed price for a given product across multiple stores and return a structured comparison.

Dynamic Pricing

The agent must detect a price change on a product page and record the updated value with a timestamp.

Cart Recovery and Personalization

The agent must resume an abandoned cart session and evaluate whether personalization signals are applied correctly at checkout.

Procedure

Each task is run three times per store per reporting cycle. Results are aggregated only after all three runs complete. Disagreements across runs trigger a fourth tiebreaker run before the score is recorded.

Output

Each task produces a pass/fail result, a structured trace of agent actions, the terminal state reached on failure, and a latency measurement. Outputs are stored in a versioned dataset released with each benchmark report.

Status

Task suite v0.1 is complete and fixed for the September 2026 benchmark cycle. Changes to task definitions will produce a new version number and will not be applied to historical data.