---
id: playbooks.conflicting-expert-opinions
title: Handling Conflicting Expert Opinions
summary: Turn expert disagreement into sharper standards instead of unresolved review noise.
kind: recipe
product_area: playbooks
status: stable
updated: 2026-08-23
canonical: /docs/playbooks/handling-conflicting-expert-opinions
---

# Handling Conflicting Expert Opinions

Use this playbook when experts disagree and the disagreement needs to become an explicit standard, not a hidden source of benchmark noise.

## Classify the disagreement

Use this when two or more qualified experts reach different judgments about the same Case, response, source hierarchy, Policy boundary, applicability condition, or Rubric. Do not average the answers before identifying the object in dispute.

## Resolution path

1. Preserve each answer, rationale, and source context within its Contribution provenance.
2. Determine whether the disagreement concerns a factual source, Case completeness, Policy rule, applicability boundary, Rubric wording, or a legitimate product tradeoff.
3. Reconstruct the strongest version of each position and test whether a missing context field or time boundary resolves it.
4. If both positions are valid in different situations, split the applicability or coverage boundary instead of forcing consensus.
5. Have the accountable owner approve the revised governed object in Correctness Governance. Contribution completion or a Checkpoint alone does not approve it.
6. Add boundary Cases that distinguish the resolved situations, review them, and create a new Benchmark Version when the evaluation boundary changes.
7. Run new evaluations without rewriting the earlier expert responses or Runs.

## Escalation outcomes

- Missing evidence: leave the question unresolved and request the controlling source.
- Different valid contexts: split applicability and add boundary Cases.
- Incorrect Case context: revise the Case through its normal version boundary.
- Incorrect reusable standard: revise and approve the Policy or Rubric.
- Product tradeoff: record the accountable owner's decision without presenting it as expert consensus.

{% example-demo title="Escalation threshold dispute" %}
One specialist accepts ordinary troubleshooting for an enterprise-managed account; another requires escalation. Their rationales reveal that one assumed a read-only action and the other assumed a contractual configuration change. The team adds the missing action-type context, splits applicability, approves the revised Policy and Rubrics, and adds two boundary Cases. Both original judgments remain attributable to the context each expert saw.
{% /example-demo %}

## Evidence to collect

- Conflicting reviewer judgments, rationales, and source context.
- The owning artifact: Case, source boundary, Policy, applicability, or Rubric.
- Human owner resolution, including any unresolved governance tradeoff.
- Boundary cases added to preserve the resolved standard.
- Benchmark comparison results after the standard is approved.

## Related docs

{% related-card-grid title="Related docs" %}
- [Handling Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Compare Harness Versions](/docs/benchmark-evaluations/compare)
- [Read run results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}

## Source confidence

Doctrine-backed: the approved product model keeps attributable expert judgment, governed approval, applicability, coverage, and evaluation evidence distinct. Linked pages define the current Contribution, conflict-resolution, and versioning surfaces.