# Handling Conflicting Expert Opinions Generated: 2026-09-13T04:43:22.535Z Source build: local Canonical docs: https://teammately.ai/docs --- id: playbooks.conflicting-expert-opinions title: Handling Conflicting Expert Opinions summary: Turn expert disagreement into sharper standards instead of unresolved review noise. kind: recipe product_area: playbooks status: stable updated: 2026-08-23 canonical: /docs/playbooks/handling-conflicting-expert-opinions --- # Handling Conflicting Expert Opinions Use this playbook when experts disagree and the disagreement needs to become an explicit standard, not a hidden source of benchmark noise. ## Classify the disagreement Use this when two or more qualified experts reach different judgments about the same Case, response, source hierarchy, Policy boundary, applicability condition, or Rubric. Do not average the answers before identifying the object in dispute. ## Resolution path 1. Preserve each answer, rationale, and source context within its Contribution provenance. 2. Determine whether the disagreement concerns a factual source, Case completeness, Policy rule, applicability boundary, Rubric wording, or a legitimate product tradeoff. 3. Reconstruct the strongest version of each position and test whether a missing context field or time boundary resolves it. 4. If both positions are valid in different situations, split the applicability or coverage boundary instead of forcing consensus. 5. Have the accountable owner approve the revised governed object in Correctness Governance. Contribution completion or a Checkpoint alone does not approve it. 6. Add boundary Cases that distinguish the resolved situations, review them, and create a new Benchmark Version when the evaluation boundary changes. 7. Run new evaluations without rewriting the earlier expert responses or Runs. ## Escalation outcomes - Missing evidence: leave the question unresolved and request the controlling source. - Different valid contexts: split applicability and add boundary Cases. - Incorrect Case context: revise the Case through its normal version boundary. - Incorrect reusable standard: revise and approve the Policy or Rubric. - Product tradeoff: record the accountable owner's decision without presenting it as expert consensus. {% example-demo title="Escalation threshold dispute" %} One specialist accepts ordinary troubleshooting for an enterprise-managed account; another requires escalation. Their rationales reveal that one assumed a read-only action and the other assumed a contractual configuration change. The team adds the missing action-type context, splits applicability, approves the revised Policy and Rubrics, and adds two boundary Cases. Both original judgments remain attributable to the context each expert saw. {% /example-demo %} ## Evidence to collect - Conflicting reviewer judgments, rationales, and source context. - The owning artifact: Case, source boundary, Policy, applicability, or Rubric. - Human owner resolution, including any unresolved governance tradeoff. - Boundary cases added to preserve the resolved standard. - Benchmark comparison results after the standard is approved. ## Related docs {% related-card-grid title="Related docs" %} - [Handling Boundary Cases](/docs/coverage-engineering/boundary-cases) - [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Compare Harness Versions](/docs/benchmark-evaluations/compare) - [Read run results](/docs/benchmark-evaluations/inspect-results) {% /related-card-grid %} ## Source confidence Doctrine-backed: the approved product model keeps attributable expert judgment, governed approval, applicability, coverage, and evaluation evidence distinct. Linked pages define the current Contribution, conflict-resolution, and versioning surfaces. --- id: governance.conflict-resolution title: Resolve conflicting correctness evidence summary: Reconcile disagreement without hiding the Cases, expert judgments, sources, or versions that produced it. kind: task product_area: governance status: stable updated: 2026-08-23 canonical: /docs/governance/conflict-resolution --- # Resolve conflicting correctness evidence ## When a conflict needs resolution Resolve a conflict when experts reach different conclusions from the same Case, when governed Policies contradict one another, when a Rubric tests a broader or narrower rule than its Policy, or when new source evidence changes the standard that earlier Benchmark Versions used. Disagreement is not automatically a reviewer-quality problem. It often exposes missing context, mixed applicability, an unresolved source hierarchy, or two legitimate product boundaries that should be modeled separately. ## Prerequisites - Name the exact Case versions, outputs, Policies, Rubrics, expert responses, and sources in conflict. - Preserve attribution and timestamps. Do not collapse opposing judgments into an unattributed summary. - Separate factual disagreement from scope disagreement and from differences in desired product behavior. - Identify the accountable owner for any governed object that may change. ### Task steps: Resolve a correctness conflict 1. Open the affected governed object or Contribution and collect the linked Cases, expert rationale, source material, and activity history. 2. Reconstruct each position in its strongest form: what evidence it uses, which situations it covers, and which outcome it recommends. 3. Test whether the conflict disappears when applicability, target-system context, user segment, source authority, or time boundary is made explicit. 4. If one position lacks required evidence, record that finding without erasing the original contribution. 5. If both positions are valid in different contexts, split or refine the Policy, applicability, Rubric, Case, or coverage facet that conflated them. 6. Have the accountable owner approve the resulting governed change. Expert participation alone does not approve it. 7. Mark affected current evidence for follow-up, create new versions or a Snapshot where required, and preserve older Runs under their original boundary. ## Object and state changes - Correct the **Case** when required context or the judged output is wrong. - Correct the **Policy** when the behavioral rule or its scope is wrong. - Correct the **Rubric** when the test does not faithfully check the Policy. - Correct **coverage facets or membership** when the benchmark over- or under-represents a boundary. - Create a new **Benchmark Version** when the governed evaluation boundary changes. - Keep an unresolved observation explicit when the source evidence cannot yet support a decision. ## Success criteria - A reviewer can see the original positions and the evidence behind each. - The resolution names the artifact and version that changed. - Approval authority is explicit. - Downstream Case selection, standards, Snapshots, Runs, or customer-owned human review context are either still valid under a named boundary or routed for refresh. - The team did not manufacture agreement by deleting dissenting evidence. ## Common failure modes - Voting before reconstructing the evidence and applicability behind each position. - Editing a downstream Rubric when the conflict belongs to a Case or Policy boundary. - Treating expert participation as approval of a governed object. - Erasing dissent or historical versions after a resolution is approved. {% example-demo title="Two valid refund rules" %} One specialist rejects every refund exception; another approves exceptions for enterprise accounts. Their Cases reveal that both followed different authoritative programs. The team adds an account-program applicability boundary, revises the Policy and linked Rubrics, records the approval in activity history, and creates a new Benchmark Version. The earlier expert responses remain attributable evidence for why the split was needed. {% /example-demo %} ## Source confidence Code-backed: Policy and Rubric detail routes expose governed objects, linked Cases, approval, and activity history, while Contribution review results preserve attributable expert learning. The evidence-reconciliation method is doctrine-backed; no single product screen automatically adjudicates every cross-object conflict. ## Related reference pages {% related-card-grid title="Related workflows" %} - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) - [Versions, staleness, and resolution](/docs/object-model/versions-staleness-and-resolution) - [Approval History and Reviewer Activity](/docs/governance/approval-history-and-reviewer-activity) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Diagnose disagreement" %} - [Low expert agreement](/docs/troubleshooting/low-expert-agreement) - [Unclear Cases](/docs/troubleshooting/unclear-cases) - [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics) {% /related-card-grid %} --- id: expert-contributions.complete title: Complete an Expert Contribution summary: Work through form, chat, interview, case-review, and checkpoint tasks while keeping specialist judgment attributable. kind: task product_area: expert_contributions status: stable updated: 2026-08-22 canonical: /docs/expert-contributions/complete-contribution --- # Complete an Expert Contribution Complete a Contribution by following its prepared task sequence and making the requested specialist judgments from the evidence shown. The expert experience can adapt between structured forms, agent chat, interviews, case review, and checkpoints. ## Prerequisites - A valid Contribution link or authenticated expert entry point. - Access to the Contribution and its assigned tasks. - Enough source and case context to explain each answer. - A stable connection when the task uses realtime agent interaction. ## Steps 1. Open the Contribution and read its objective, selected cases, and expected components before answering. 2. Complete each task according to its type. Planned activities can be Case Review, Form, Chat, or Interview; Curation, Comparative, and Trajectory components shape the prepared work those activities present. 3. Use attachments and visible case materials as the evidence boundary. State uncertainty when the supplied material does not resolve the question. 4. At a checkpoint, inspect the proposed summary or artifact meaning. Checkpoints prepare and reconcile requirements, consolidator or Policy statements, interview requests or records, and Rubrics. Confirm only what matches your judgment; retry, revise, or leave unresolved anything that does not. 5. Continue through the task handoff until the Contribution reaches its final step. 6. Review the completion state. If the experience shows a waiting, retry, or synchronization state, do not assume the administrator has received final evidence until the product confirms it. ## Object and state changes Answers create durable task responses and can advance task sessions, checkpoints, handoffs, and Contribution status. Chat or interview activity can produce transcripts and structured learning. Case review can attach judgment to selected cases. Completion makes the contribution available for reconciliation and materialization but does not itself make every proposed artifact governed. Review tasks can be `PREPARING`, `BLOCKED`, `READY`, `IN_PROGRESS`, `COMPLETED`, `SKIPPED`, or `SUPERSEDED`. The expert runtime can be `PREPARING`, `READY`, `ACTIVE`, `FINAL_CHECKPOINT`, `COMPLETED`, or `EXHAUSTED`. Checkpoints can be preparing, ready, or reconciled, with individual requirements pending, retryable, materialized, empty, or failed. These layered states explain why a Contribution can be active while one task is blocked or a final checkpoint is still pending. ## Success criteria - Every answer addresses the Contribution objective and cites the visible evidence where needed. - Case-level judgments remain connected to the relevant case. - Checkpoints distinguish accepted, revised, and unresolved meaning. - The final state is visibly complete rather than inferred from navigation. - Uncertainty or source conflict remains explicit for the administrator. ## Common failure modes - Answering from private background without identifying that the supplied evidence is incomplete. - Treating an agent summary as accurate without checking the checkpoint. - Leaving a form or chat task in a local unsynchronized state. - Continuing after a stale task handoff instead of following the current Contribution route. - Assuming that completion directly changes policies, rubrics, cases, or coverage. {% example-demo title="Example: checkpoint correction" %} An interview summary says that every expired agreement should be ignored. The expert corrects the checkpoint: expired agreements may still be relevant when the current agreement explicitly incorporates them. The corrected statement remains attributable and prevents an overbroad policy from being materialized. {% /example-demo %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Expert Contributions](/docs/expert-contributions) - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Expert Contribution problems](/docs/troubleshooting/expert-contributions) - [Permissions](/docs/troubleshooting/permissions) - [Authentication](/docs/troubleshooting/authentication) {% /related-card-grid %} ## Source confidence Code-backed: the current expert experience supports form, chat, interview, case-review, checkpoint, completion, waiting, and task-handoff routes with durable command and reconciliation behavior. --- id: governance.human-approval-boundaries title: Human Approval Boundaries summary: Define which AI-assisted suggestions require accountable human review before becoming standards. kind: reference product_area: governance status: stable updated: 2026-09-07 canonical: /docs/governance/human-approval-boundaries --- # Human Approval Boundaries ## Definition Human approval boundaries separate preparation from governed correctness evidence. A suggestion, reviewer comment, interview answer, or draft standard can inform the loop, but it should not govern benchmark interpretation until the relevant human approval state is clear. Use this page when prepared or contributed material is about to become a governed Policy, Rubric, Case-scoped reference output, or selected Benchmark Dataset evidence. Comparison Directions are a narrower configuration object, not a governed standard. An AI-suggested Comparison Direction can be active without a separate approval step, but generated Cases, Benchmark membership, Policies, Rubrics, and reference outputs still follow their owning review or approval boundaries. A customer-owned human review packet is assembled from evidence; it is not a Teammately approval state. > Approval is a state transition > > AI-assisted suggestions do not become governed standards until an accountable human approves the relevant artifact. ## Fields, states, or lifecycle rules - Draft suggestions and reviewer notes are preparation material. - Approved Policies and Rubrics, reviewed Case changes, selected Dataset membership, and supported Case-scoped reference outputs can affect governed evidence. - AI-suggested Comparison Directions can affect future variant generation as active directions, but they do not approve the generated cases or standards they help explore. - Approval should name the artifact being approved, not only the discussion that produced it. - Stale or superseded approvals should be visible before older benchmark evidence is reused. - This page does not claim external compliance approval, legal signoff, or production deployment authorization. ## Related objects Read this with [What AI Features Can and Cannot Do](/docs/governance/what-ai-features-can-and-cannot-do), [Comparison Directions](/docs/assets/comparison-directions), [Approving Suggested Policies](/docs/correctness-governance/policies-and-rubrics), [Editing Suggested Rubrics](/docs/correctness-governance/policies-and-rubrics), and [Approval History and Reviewer Activity](/docs/governance/approval-history-and-reviewer-activity). {% example-demo title="Human Approval Boundaries boundary" %} Reviewer context: Several experts reject unsupported refund exceptions. Suggested policy: The system proposes a policy that exceptions require approved support. Approval boundary: The suggestion becomes governed only when a human owner approves the policy and its applicability. Benchmark interpretation: Runs should cite the approved policy, not the unapproved suggestion that preceded it. {% /example-demo %} ## Source confidence Code-backed: Policy approval and activity components expose accountable approval state and history; contributed-artifact types keep expert learning distinct from materialized governed objects. Comparison Directions deliberately use active, archived, and advisory-stale behavior instead of the Policy approval lifecycle. ## Related task pages {% related-card-grid title="Related task pages" %} - [What AI Features Can and Cannot Do](/docs/governance/what-ai-features-can-and-cannot-do) - [Comparison Directions](/docs/assets/comparison-directions) - [Review Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %}