Why we should all build custom benchmarks for automated workflows

Most teams decide whether an AI workflow works by running it a few times and reading what comes back. That is a reasonable way to learn what a model is capable of, and a poor way to know whether the workflow will keep behaving after the next prompt edit, tool change, or model release. Because language models are not deterministic, a run that succeeds once is an example rather than an evaluation, and the failures that matter most in operations rarely show up in the output that reads well.

A custom benchmark is the instrument that turns those impressions into a measurement. It is a fixed set of your own cases, each with the information available when the task begins, a written definition of an acceptable outcome, and a scoring rule. You rerun the whole set every time anything in the workflow changes. Public leaderboards rank models on shared general tasks and can help you shortlist candidates, but they cannot tell you whether a return was approved under the right policy or a product attribute was saved to the right variant. Only your own cases encode what acceptable work means in your business.

My training is in psychology research at McGill, and the habit that carried over most directly into building workflows is operationalization. In research, you cannot test anything until you turn the idea you care about into something measurable, which means deciding which observable outcome stands in for the concept and exactly how you will score it, because a vague construct cannot produce data. Benchmarking a workflow asks for the same move. "Handles returns well" cannot be measured, but "chooses the correct decision, creates the authorization on the right order, and never promises a refund before the item arrives" can, and once the outcome is defined that precisely, every run produces real data you can build on, improve against, and iterate from instead of an impression.

Why this matters for workflows and not for chat

For a simple question-and-answer chat with an agent, most of this rigor is unnecessary. When I ask a model to explain a concept, I judge the answer on the spot and ask again if it misses, so the person in the conversation is already the evaluation. That changes once the model sits inside a complex workflow, a multi-step process, or an agent meant for production. Each step consumes the output of the one before it, so a small early error carries forward quietly, and the run often ends in an action in a real system rather than a message someone reads. In that setting, tracing and evals against a benchmark are what make reliability possible, because tracing shows what actually happened inside each run while the benchmark defines whether what happened was acceptable.

What one case actually contains

Take a returns workflow for an online store as the worked example. An agent reads the customer's message, looks up the order, and checks it against the return policy. It then decides whether to approve, decline, offer an exchange, or escalate, creates the return authorization when one is warranted, and drafts the reply. Each case in the benchmark captures a real situation from that workflow along with the outcome a person who handles returns would accept. The rows below use an illustrative 30-day policy rather than any real store's rules.

case_id,customer_message,order_facts,expected_decision,required_actions,must_not,case_type
R-014,"The jacket arrived torn, I would like a refund","delivered 9 days ago, full price",approve_return,"create return authorization on this order, send prepaid label",promise refund before item is received,ordinary
R-022,"Can I still return these boots from the sale?","delivered 41 days ago, final sale",decline,"reply citing the sale policy",create a return authorization or label,ordinary
R-031,"Tracking says delivered but nothing came","delivered 2 days ago, no scan at door",escalate_carrier_claim,"route to carrier claims queue",approve a return for an item never received,hard
R-047,"Est-ce que je peux échanger pour une taille plus grande?","delivered 5 days ago, size in stock",exchange,"create exchange order, reply in French",reply in English,hard

Three columns do most of the work. The expected decision states the business outcome, and the required actions describe what must actually land in the store's systems. The must-not column records the mistake that would make the result unusable even if everything else were correct. Defining these metrics relies on the people who hold the tacit understanding and business logic tied to that specific process.

The case type matters as well. A set drawn only from past failures will overstate how often the workflow breaks, while a set drawn only from ordinary requests will hide the cases that cost money. A useful benchmark mixes both and labels them, which lets you report the two rates separately.

Score the whole workflow, not the step that reads well

The most common scoring mistake is to grade the reply text, since it is the part a person can read. A reply that politely confirms a refund proves nothing about whether the return authorization was created on the right order. In case R-022, a perfect decline message paired with a prepaid label sent anyway is a failed case, not a mostly successful one. Scoring the complete workflow means checking the decision, the actions recorded in the system of record, the reply, and the route the request took through models, tools, and people. Any must-not violation fails the whole case, regardless of how the other checks went.

In practice this splits naturally into two kinds of checks. The decision, the actions, and the route can almost always be scored with ordinary code that compares the run's recorded result against the expected fields, which makes those checks cheap and repeatable. The reply's tone, accuracy against policy, and language are harder to score mechanically, so a model can act as a judge for them. Before relying on that judge, compare its verdicts with the person who normally reviews returns on a sample of cases, and only trust it where the two agree.

Alongside quality, it is worth recording retries, elapsed time, model cost, and the minutes a person spent correcting the output. A version that matches the old one on quality while doubling review time is not an improvement.

How tracing shows where a run went wrong

A trace is the record of a single run, organized as a tree. The root is the run itself, and underneath it sit spans, one for each step the workflow took. Each span records what went into that step and what came out, along with when it started and how long it took. For a model call, that means the exact prompt sent, the model and its settings, the response, and the tokens used, which is where the cost figure comes from. For a tool call, such as the order lookup or the step that creates a return authorization, it means the arguments passed and the result returned, and for retrieval it means which documents were pulled and in what order. Spans nest, so a step that calls a model and then two tools shows those three calls as children beneath it, and the tree reads as the sequence of decisions the workflow made.

That structure is what makes a failed benchmark case diagnosable. If case R-022 fails because a label went out on a final-sale order, the trace shows whether retrieval pulled the general return policy instead of the sale policy, whether the model read the right policy and still chose to approve, or whether the decision was correct and the label tool ran anyway because of a logic error in the code around it. Each of those has a different fix, and without the trace you would be guessing between them from the final output alone.

Tracing tools such as Langfuse also let you attach scores to traces, so a benchmark run becomes a set of traces, one per case, each carrying its pass or fail result and any judge scores. On our AI Fit readiness tool, the extraction step has its own set of twelve synthetic cases with scores in Langfuse, which means a change to the prompt or model behind that step can be compared against the same cases instead of against my memory of how the last version behaved.

Rerun everything on every change, and compare case by case

A benchmark earns its keep through regression catching. Every time you edit a prompt, swap a model, add a tool, or change a routing rule, you rerun the full set with the same inputs and scoring rules. Then you compare the results case by case rather than only by the average. A new version can raise the overall pass rate while breaking the one case that matters most, such as the final-sale decline, and an average alone will hide that trade.

Changing one thing at a time keeps the comparison honest, and it helps to hold some cases aside while you develop. Once you have tuned a prompt against a case, that case has helped shape the workflow and can no longer tell you how it handles requests it has not seen. The held-out cases become the real test of whether a fix generalizes.

The set should grow with the workflow. Every failure found in production, usually spotted in the traces of live runs, becomes a new case with its expected outcome written down. During our internal hackathon, an onboarding agent answered new-hire questions from the open web instead of our internal documents, and the answers sounded fluent while citing the wrong source. In benchmark terms, that becomes a case whose question can only be answered from internal material and whose pass condition requires citing it, so the same failure cannot quietly return after a later change.

A spreadsheet and the person who knows the work are enough to start. Once the set grows or several people change the workflow, a tracing tool like Langfuse can hold the datasets alongside the traces, and Promptfoo can run declared cases in continuous integration so that a change that fails the benchmark is blocked before release. Neither replaces the written definition of acceptable work, which only your team can supply.

What changes when you have one

Decisions about models and prompts stop depending on how the last few outputs felt. When a cheaper model is released, you can run it against the same cases and see whether it still passes the hard ones. When someone proposes a clever new step, you can find out whether it fixes the failure it was meant to fix without breaking something else.

To start, pick one recurring workflow and ask the person who does it today to write twenty or thirty cases with expected outcomes and must-not mistakes. Run your current process against them before changing anything, so that every later version has an honest baseline to beat. Models will keep changing every few weeks, and the set of cases that defines good work for your business is the part that carries over from one to the next.