---
id: troubleshooting.low-expert-agreement
title: Low expert agreement
summary: Investigate disagreement as evidence about context, applicability, source authority, or unresolved product policy.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-08-23
canonical: /docs/troubleshooting/low-expert-agreement
---

# Low expert agreement

## Symptoms

- Experts choose different answers for the same Case or Checkpoint.
- Rationale shows different assumptions even when the selected answer matches.
- Agreement is high on routine Cases and collapses on one boundary segment.
- A Contribution completes, but its learning cannot be materialized into one defensible standard.

## Likely causes

- Experts saw different Case versions, sources, or task context.
- The Case or question leaves the required judgment ambiguous.
- Reviewers use different applicability or source-authority assumptions.
- The domain genuinely contains unresolved or context-dependent policy.

## Preserve disagreement before interpreting it

Do not average away the responses or select the majority answer before reading rationale. Confirm that each response belongs to the intended expert, task, Case version, question, and source context. A routing mistake and a legitimate domain disagreement require different fixes.

## Diagnostic checks

1. Open the review or session detail and compare the exact prompt, Case context, available sources, response, rationale, and timestamps.
2. Check whether all experts saw the same version and whether any required context was missing.
3. Separate four causes: ambiguous Case, unclear question, different applicability assumptions, and conflicting domain judgment.
4. Segment the disagreement by source authority, user type, risk, product state, or other relevant coverage facet.
5. Ask whether the current Policy already resolves the question. If it does, determine whether the Policy or its applicability was invisible or unclear to reviewers.

## Fix

- **Missing context:** correct or version the Case and repeat only the affected review.
- **Unclear question:** rewrite the task so the requested judgment and evidence threshold are explicit.
- **Different applicability:** refine the Policy boundary or split Cases into distinct contexts.
- **Conflicting expert judgment:** route the evidence through conflict resolution and an accountable owner.
- **Wrong expertise:** revise assignment criteria without deleting the attributable response.

Do not treat consensus as approval. Even unanimous expert input becomes a governed Policy, Rubric, Case-scoped reference output, or Benchmark Dataset decision only through the owning workflow.

## Prevention

Package the same visible evidence for every assigned expert, state the requested judgment precisely, and include applicability or source-authority context when it controls the answer. Preserve rationale so later disagreement can be diagnosed without repeating every Contribution.

{% example-demo title="Example: agreement drops only for enterprise accounts" %}
Three experts agree on standard refunds but split on enterprise exceptions. Session detail shows that two use a newer enterprise program while one follows the public policy. The operator records both sources, adds the account-program boundary, and routes a Policy revision for approval. The disagreement becomes useful evidence instead of a reviewer score.
{% /example-demo %}

## Source confidence

Code-backed: review logs, session detail, review-result normalization, and expert-review types preserve task, session, response, and rationale context. The causes of disagreement and the appropriate governed resolution require human interpretation.

## Related task pages

{% related-card-grid title="Related workflows" %}
- [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Approval History and Reviewer Activity](/docs/governance/approval-history-and-reviewer-activity)
{% /related-card-grid %}

## Related reference pages

{% related-card-grid title="Related reference" %}
- [Expert Contributions](/docs/expert-contributions)
- [Human Approval Boundaries](/docs/governance/human-approval-boundaries)
- [Reviewer and Project Access](/docs/governance/reviewer-and-project-access)
{% /related-card-grid %}