Retail AI agents struggle with complex, multi-step customer-service workflows, often losing context, acting on the wrong records, or failing to execute the required actions. RetailServe targets these failures with realistic scenarios, verifiable tool-call trajectories, and a controlled difficulty gradient designed for post-training.
RetailServe is a post-training corpus of multi-turn customer-service tasks, built for training and evaluating tool-using conversational agents. It spans a wide range of retail scenarios and difficulty levels. With around 2K tasks and two gold trajectories each, post-training on it lifted Qwen3-8B's Pass@1 on the official τ²-bench Retail benchmark from 31.6% to 60.2%, a gain of 28.6 points.
Multi-turn, tool-calling agents are moving into production fast, but training data has become a bottleneck. Real customer-service workflows are rarely simple: an agent may need to identify the right customer, find the correct order from incomplete information, reason about policy, execute a write action, and confirm the result — all while keeping track of the original request.
Models keep falling into the same traps. They generalize poorly to unfamiliar patterns, get stuck in loops on steps they cannot recover from, act on the wrong record, or claim a job is done without ever executing the required write. Training data therefore needs to do more than cover common customer-service conversations. It needs realistic scenarios, verifiable tool-call trajectories, and a difficulty curve that gives models progressively harder workflows to learn from.
RetailServe is built around these requirements. Every task is synthesized in-house by Abaka rather than derived from the 114 official τ²-bench Retail evaluation tasks, which remain fully held out. It follows the benchmark's tool-calling and conversation conventions so that training tasks run in the same environment and evaluation harness while remaining separate from the test set.
Each RetailServe task is a self-contained, runnable unit with a stateful store backend, a user simulator, a gold trajectory, and a programmatic verifier. Together, these components make both the task and its outcome fully checkable.
Consider a hard-tier task from the dataset. The customer wants to exchange one item from a delivered order for a different variant of the same product, but does not remember the order number. All they can provide is a rough description of what was in the order.
Customer: "I got my order and want to swap one item for a different version. It's the bookshelf from my delivered order. I want the black-glass 3 ft one, the $549.84 model, instead."
The simulator knows the customer's email and, if asked, will confirm their identity and the payment method on the order. It will not volunteer this information. The task contains nine independent checks, each targeting a different point where the agent can go wrong. Five of them illustrate the range:
identity_resolved: Find the right customer from the email, then the right order. The customer never gives an order number, so the agent has to work out which order they mean from what was in it: a bookshelf and an electric kettle. The check passes only if the resolved user and order match the gold record. Pick the wrong order and everything after it is wrong too.
only_target_item_changed: That same order also holds an electric kettle. The agent has to swap the bookshelf and leave the kettle alone. The verifier diffs the order line by line, so touching the kettle, or "fixing" the whole order, fails the check even when the bookshelf itself was handled correctly.
exact_variant_match: There are several bookshelf versions, and the customer pinned theirs down by price ($549.84). The check compares the new item ID against the exact variant in the brief. A close but different one scores zero.
original_payment_method: Any price difference goes on the payment method already attached to the order, not some other card. The verifier reads the payment ID in the write call and matches it against the order's stored method.
exchange_actually_applied: This is the one that catches weaker agents. Saying the swap is done is not enough. The check reads the store's records after the dialogue and confirms the exchange write actually landed. If the agent produces a polished "all set, your bookshelf has been exchanged" but never calls the write tool, every state-dependent check above it fails with it, and the task scores zero.
This is what makes RetailServe useful as training data. A plausible response is not enough: success is grounded in the agent's actions and the resulting database state. Each check verifies a specific part of the workflow, so an agent that completes the task correctly passes, while one that only sounds correct fails at the point where its execution diverges.
RetailServe spans a broad range of retail customer-service workflows, including order management, returns, refunds, exchanges, cancellations, account-related requests, delivery updates, customer verification, and escalation handling. The dataset also includes multi-step and multi-intent interactions, extending beyond routine requests to more complex cases that require agents to track context and coordinate multiple actions.
Before training on this corpus, we checked whether its difficulty distribution was well calibrated. An effective training set needs enough accessible tasks to establish a learning base, while preserving harder tasks that leave room for continued improvement.
The distribution reflects this balance. Foundational tasks make up 18% of the corpus, providing a solid base without dominating it. Easy (30%) and Moderate (27%) are the two largest bands, placing more than half of the tasks in the middle of the difficulty range. Hard (17%) and Very Hard (8%) preserve a meaningful challenging tail. Overall, difficulty is concentrated in the middle and tapers toward the hard end, rather than spreading evenly or clustering at the extremes.
To test whether RetailServe translates into measurable improvements in agent performance, we post-trained Qwen3-8B in two stages. We first applied supervised fine-tuning (SFT) on the gold trajectories from around 2K RetailServe tasks, with two trajectories per task. We then used curriculum reinforcement learning with GRPO and verifiable outcome rewards on a subset of harder tasks, ordered from easier to harder based on baseline pass rate.
SFT alone raises average Pass@1 on the official τ²-bench Retail benchmark from 31.6% to 47.4%. Adding curriculum-based RL pushes performance further to 60.2%, a total improvement of 28.6 percentage points over the base model.
The metric breakdown shows where the gains come from. SFT nearly saturates read-action accuracy, increasing it from 72% to 90.5%, while write-action accuracy improves from approximately 37% to 57.5%. RL primarily strengthens write actions, raising accuracy further to 74%, while read-action accuracy remains broadly stable at 89%. DB Match follows the same pattern, improving from approximately 37% to 52% after SFT and 64% after RL.
At 60.2% Pass@1, the post-trained model also exceeds other reported 8B systems on τ²-bench Retail, including TopoCurate-RL at 54.5% and Simia-Tau-RL at 52.9%. RetailServe reaches this result with a comparatively small training set and with test-time thinking turned off, while those baselines are evaluated with thinking enabled.
Building capable retail customer-service agents is ultimately a training data problem. The challenge is not just covering more scenarios, but creating tasks that reflect the failure modes agents encounter in real workflows — from resolving ambiguous requests to executing the right actions against the right records.
RetailServe is built around that goal: realistic scenario coverage, verifiable tool-use trajectories, and a controlled difficulty gradient for post-training. With around 1.5K tasks and two gold trajectories each, post-training on RetailServe improves Qwen3-8B's Pass@1 on the official τ²-bench Retail benchmark from 31.6% to 60.2%.
For teams building retail AI agents, RetailServe provides training data designed to move models beyond plausible responses toward reliable execution in real customer-service workflows.
Retail agent training data consists of realistic customer-service tasks and interaction trajectories designed to teach AI agents how to complete retail workflows, including order management, returns, refunds, exchanges, cancellations, and customer verification.
Retail AI agents need to do more than generate the right response. They must identify the correct records, follow policy, call the right tools, and make the intended changes to the underlying system. Tool-use training data helps agents learn these multi-step workflows.
RetailServe combines realistic scenarios, verifiable tool-use trajectories, and controlled task difficulty for post-training. In our experiments, post-training on RetailServe improved Qwen3-8B's Pass@1 on the official τ²-bench Retail benchmark from 31.6% to 60.2%.
Missing data
We collect it.
Messy data
We label it.
No time
We have itOff-The-Shelf.
Pick the closest fit, we'll take the call from there.
Missing data
We collect it.
Messy data
We label it.
No time
We have it Off-The-Shelf.
Pick the closest fit, we'll take the call from there.
