What Should You Fix: the Model, the Harness, the Context, or the Specification?

How Enterprise AI Gets Better · Part 3 of 6: Experiments and interventions

Suppose a support agent refunds an order that has already received a refund. An engineer adds “never refund twice” to the prompt. The example passes. A legitimate partial refund now fails.

The example is green. The workflow is worse. The new instruction blocks legitimate work along with the duplicate.

Use the refund to choose a discriminating experiment, then repair the path it exposes. Here, harness means the runnable container around the model: instructions, context, tool contracts, orchestration, and execution controls. Diagnosis separates the information reaching a decision from the code that acts on it.

Use the diagnostic worksheet beside a failing run. Jump to the probe table, transaction evidence, retry handling, or repair report as needed.

Establish that the failure is valid

Inspect the request, rule, starting state, available records and tools, actions, and resulting state. Confirm that the case is solvable and the evaluator distinguishes a duplicate payment from a permitted adjustment or valid alternative.

Anthropic identifies task clarity and grading quality as evaluation dependencies.¹ Correct a faulty assessment and rerun baseline and candidate; report a measurement correction.

Verify the actual account change and amount. τ-bench’s original design compares final database state with the goal state and examines repeated-trial reliability.²

Choose the probe that distinguishes the explanations

Four surfaces are available:

• Specification: Required behavior, applicability, permissions, and unacceptable outcomes.
• Context: Instructions, records, history, and tool results available at the decision.
• Harness execution: Orchestration, tools, retries, state handling, and execution controls. Some repairs belong in an underlying service.
• Model: Learned capabilities and tendencies, inference configuration, and weights.

The evaluation infrastructure runs and assesses that system; it is distinct from the agent’s execution harness.

These are diagnostic views of an interacting system. The harness assembles context, a prompt expresses a rule, and a stronger model may handle a confusing interface. The location of a change alone does not establish its cause.

Use the failing refund case to select a probe:

Start where the trace points. Include the nearest contrast that could invalidate the repair and cases that already pass. Hold other conditions fixed, reset state, and repeat when variability matters. Record effort and run cost before expanding. Bring in a practitioner for an unsettled rule; investigate a clear retry contract directly.

When the evidence points to an interaction, test it directly. Compare the current and revised context with both model candidates. If the new model helps only with the revised context, the adoption decision concerns that combination. A sequence of isolated changes could miss it. Keep the four setups and their costs visible.

First branch: what counts as a duplicate?

“Already refunded” may cover several states: a completed item refund, a shipping adjustment, a partial refund, or a canceled attempt.

Bring unsettled combinations to the process owner and practitioners. Which states block another adjustment? Which remain eligible?

The organization might prohibit exceeding an item’s eligible amount while permitting a separate shipping adjustment. Engineering makes that rule explicit and assessable.

Test both sides of the boundary. Stopping duplicate refunds by refusing every refund violates the intended workflow.

Second branch: did the transaction history reach the decision?

Suppose the rule is clear, but the agent’s context omits previous adjustments.

Supply a minimal evidence packet containing the relevant order, current transaction history, and applicable rule. If behavior improves, inspect the production path that should supply those records.

Check identity, freshness, timestamps, and summaries that discarded adjustment amounts. More documents would leave a missing amount unresolved.

Anthropic’s context-engineering guidance follows information across an evolving interaction.³ Check what survives into the action and what a state change has invalidated.

The probe must supply evidence the production system could legitimately obtain. An expected answer disguised as a transaction record tests whether the model can repeat a solution.

A successful direct-record probe gives the team a reason to inspect retrieval. Shipping the manually supplied record would leave the next customer’s information problem unsolved. Restore the repaired lookup in the full workflow and verify what reaches the decision.

Third branch: did a retry create the duplicate?

Suppose the agent received the correct history, submitted a valid refund, encountered a timeout, and retried. The first request had succeeded despite the timeout.

A timeout leaves two possibilities: the request failed before commitment, or it committed and the response was lost. An instruction to retry cannot settle which occurred. The execution contract must distinguish a retry from a new operation, reconcile authoritative state, and enforce the remaining eligible amount.

Use an idempotency mechanism appropriate to the operation and enforce exact eligibility constraints where authoritative state is available. A retry of the same request and a new, legitimate partial adjustment must remain distinguishable.

The model interprets the request, chooses permitted actions, and explains uncertainty. The service enforces the transaction invariant.

The interface must explain an uncertain outcome. Anthropic’s tool-engineering guidance emphasizes meaningful responses and clear boundaries.⁴ A timeout must not imply that the mutation failed.

The retry probe loses a response after commitment, then repeats the same operation identity. Expect one adjustment and preserve a distinct eligible adjustment. Verify both through the actual service path, including canceled attempts and recovery.

Fourth branch: does the model still miss the distinction?

Investigate the model when clear requirements, usable evidence, and correct execution still leave recurring interpretation errors.

Compare models on unfamiliar partial-adjustment layouts with the same cases, records, tools, and documented inference settings. Report quality and cost, including budget differences.

If the model handles the transaction in isolation but fails after several turns, inspect what the full workflow loses. Our recorded question-design investigation shows a different isolation: separate the primary decision from another obligation before interpreting the model’s judgment.

Training can target a stable distinction: interpreting partial adjustments, choosing an eligible action, or recognizing missing evidence. Keep changing account state and permissions externally available; train the model to consult them.

Turn the successful probe into a complete repair

Compare successful alternatives on outcomes, latency, cost, complexity, and handoffs. Combine complementary repairs where useful, then evaluate the complete design through its real execution path.

The probe asks whether one intervention changes the behavior. The complete candidate must meet the requirements across the workflow. A model that succeeds with directly supplied history can still fail when the production tool summarizes away the relevant adjustment. Preserve the probe as diagnostic evidence and test the adopted combination end to end.

Check the original failure, independently constructed cases exercising the distinction, and nearby behavior the repair might damage. Paraphrases alone add little breadth.

Pair the results by task. Suppose a candidate rises from 80 to 84 passes on the same 100 cases. That could mean four fixes and no regressions—or ten fixes offset by six newly broken cases. Inspect both directions, especially the requirements behind the losses. For repeated trials, compare each task’s results across candidates before summarizing the task population. A single favorable rerun should not decide which explanation won.

In Teammately’s Trialground, compare the candidate with its baseline and inspect the outputs, recorded trajectories, and requirements behind the difference. Lemon Goal investigates failures and tests competing harness candidates. For this refund, the assignment must protect eligible partial adjustments as well as prevent duplicates. Engineers decide what to integrate.

Close the investigation with a claim someone can challenge

A repair report should say what changed, what stayed fixed, how the comparison was made, and what remains untested. For the refund workflow, that could look like:

Change: Revised transaction lookup and retry handling.

Required behavior: Prevent repeat adjustments while preserving eligible partial refunds.

Comparison: Baseline and candidate use the same approved rule and reset starting states.

Evidence to report: Outcomes by transaction condition, repeated-trial results, regressions, and handoffs.

Paired change: Cases repaired, cases newly broken, and unresolved comparisons on the same task basis.

Scope boundary: Identify transaction formats and recovery conditions outside the evaluation.

Complete the report with observed outcomes and preserve the versions, cases, grader, and execution settings. The diagnostic worksheet also records engineer effort, run cost, maintenance, and remaining scope. Those are part of judging the repair.

A useful experiment makes the next production change easier to justify. A useful repair survives the conditions that exposed the failure.

Continue with your question

If the same cases guided the repair, read how to preserve separate evidence for an improvement claim.

Next: Your Production Logs Tell You What Happened, Not What Must Work

Browse the series and resources.

References