# Overly Broad Policies Generated: 2026-09-13T04:42:52.835Z Source build: local Canonical docs: https://teammately.ai/docs --- id: troubleshooting.overly-broad-policies title: Overly Broad Policies summary: Narrow policies that are too vague to guide review, rubrics, or human reviews. kind: error product_area: troubleshooting status: stable updated: 2026-08-23 canonical: /docs/troubleshooting/overly-broad-policies --- # Overly Broad Policies Use this when a policy is too general to guide review, rubric writing, or human reviews. ## Symptom Reviewers agree with the policy in principle but cannot decide whether a specific output satisfies it. Rubrics derived from the policy become vague, noisy, or impossible to apply consistently. ## Likely causes - The policy describes a value such as “be helpful” without a behavioral boundary. - Several domain rules were combined into one policy. - The policy lacks linked cases that show pass and fail examples. - The policy was approved from one expert comment without broader review. ## Diagnostic checks - Ask whether the policy can produce a binary rubric without adding new assumptions. - Find linked cases that should pass and fail the policy. - Compare the policy to existing standards for duplicate or conflicting scope. - Review benchmark failures to see whether the policy explains the evidence. ## Fix - Narrow the policy to one behavior boundary. - Split unrelated requirements into separate policies. - Add linked cases and applicability before approval. - Use the Policy's own governed approval and version controls after revision. A Contribution Checkpoint confirms Contribution progress; it is not automatically Policy approval. ## Prevention - Draft policies from repeated judgments, not one-off comments. - Require applicability and examples before approval. - Review policy overlap during standards maintenance. - Version revisions so old benchmark evidence remains interpretable. ## Related task pages {% related-card-grid title="Related task pages" %} - [Create a policy](/docs/correctness-governance/policies-and-rubrics) - [Using expert judgment](/docs/concepts/correctness-elicitation) - [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Policies](/docs/object-model/policies) - [Policy versions](/docs/governance/policy-versioning) - [Correctness Elicitation](/docs/concepts/correctness-elicitation) {% /related-card-grid %} ## Source confidence Code-backed: Policy list and detail routes expose governed Policy content, approval, versions, linked Cases, and Rubrics. This page keeps Contribution Checkpoints separate from Policy approval. --- id: object-model.policies title: Policies summary: Define policies as reusable statements of what correct AI behavior requires. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/policies --- # Policies ## Definition Policies are reusable statements of what correct behavior requires. They preserve expert judgment as a governed standard that can later be tested by applicability logic and rubrics. Use this reference when a team needs to know whether a rule is a draft suggestion, an approved correctness standard, or a standard whose version may affect benchmark evidence. ## Fields, states, or lifecycle rules - A policy names the rule; a rubric tests the rule. - A policy needs applicability logic before the team can know which cases it should judge. - Policy approval state matters before benchmark use. - Policy revisions can make older benchmark evidence stale or require comparison notes. - This page does not define legal, compliance, retention, or external policy-management guarantees. ## Related objects Policies should be read with [Applicability logic](/docs/object-model/applicability-logic), [Rubrics](/docs/object-model/rubrics), [Policy versions](/docs/governance/policy-versioning), and [Human Approval Boundaries](/docs/governance/human-approval-boundaries). Use [Create a policy](/docs/correctness-governance/policies-and-rubrics) for the operating workflow. {% example-demo title="Policies boundary" %} Expert judgment: Answers should not claim compatibility unless source data explicitly supports the claim. Policy: Compatibility claims require explicit source support. Interpretation: The policy explains the standard; linked applicability and rubrics decide when and how a specific output is judged. {% /example-demo %} ## Source confidence Code-backed: Policy types and list/detail routes expose identity, description, applicability, linked Cases and Rubrics, version facts, and approval state. The product object is a governed correctness standard, not an external legal-policy system. ## Related task pages {% related-card-grid title="Related task pages" %} - [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Applicability logic](/docs/object-model/applicability-logic) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.applicability-logic title: Applicability logic summary: Explain when a policy or rubric should be used for a case, output, or coverage segment. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/applicability-logic --- # Applicability logic ## Definition Applicability logic is the boundary that decides whether a policy or rubric should be considered for a specific case, output, or coverage segment. It prevents a good standard from being applied to the wrong behavior. Use this reference when a benchmark result is hard to explain because a standard seems relevant in some cases but not others. The question is not only whether the policy is correct; it is whether the policy was eligible to judge that output. ## Fields, states, or lifecycle rules - Applicability sits between the case/output and the policy/rubric that may judge it. - Weak applicability makes benchmark failures noisy: a candidate can fail a good rubric on a case where the rule should not have applied. - Strong applicability names the behavior condition, source context, or case segment that brings the standard into scope. - Applicability changes can make old benchmark evidence stale because the same output may be judged by a different standard boundary. - This page explains the public object relationship, not a public rule language, API schema, or export contract. ## Related objects Applicability logic should be read with [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Cases](/docs/object-model/cases), and [Coverage Dimensions](/docs/object-model/coverage-dimensions). Use the Correctness Governance workflow to write the boundary and troubleshooting when the observed Case set is wrong. {% example-demo title="Applicability logic boundary" %} Raw case: A user asks whether a product works with equipment they already own. Policy: Compatibility claims require explicit source support. Applicability logic: The policy applies only when the answer recommends, validates, or compares a product for a concrete use context. Benchmark interpretation: If the case only asks for a product description, the compatibility rubric should not judge it. If the answer claims the product will work with the user's equipment, the rule is in scope. {% /example-demo %} ## Source confidence Code-backed: Policy and Rubric types and their detail routes expose applicability fields and the links among governed standards, Cases, and evaluation checks. This page describes those product relationships; it does not define a portable rule language. ## Related task pages {% related-card-grid title="Related task pages" %} - [Work with Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Fix weak applicability logic](/docs/troubleshooting/weak-applicability-logic) - [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: governance.conflict-resolution title: Resolve conflicting correctness evidence summary: Reconcile disagreement without hiding the Cases, expert judgments, sources, or versions that produced it. kind: task product_area: governance status: stable updated: 2026-08-23 canonical: /docs/governance/conflict-resolution --- # Resolve conflicting correctness evidence ## When a conflict needs resolution Resolve a conflict when experts reach different conclusions from the same Case, when governed Policies contradict one another, when a Rubric tests a broader or narrower rule than its Policy, or when new source evidence changes the standard that earlier Benchmark Versions used. Disagreement is not automatically a reviewer-quality problem. It often exposes missing context, mixed applicability, an unresolved source hierarchy, or two legitimate product boundaries that should be modeled separately. ## Prerequisites - Name the exact Case versions, outputs, Policies, Rubrics, expert responses, and sources in conflict. - Preserve attribution and timestamps. Do not collapse opposing judgments into an unattributed summary. - Separate factual disagreement from scope disagreement and from differences in desired product behavior. - Identify the accountable owner for any governed object that may change. ### Task steps: Resolve a correctness conflict 1. Open the affected governed object or Contribution and collect the linked Cases, expert rationale, source material, and activity history. 2. Reconstruct each position in its strongest form: what evidence it uses, which situations it covers, and which outcome it recommends. 3. Test whether the conflict disappears when applicability, target-system context, user segment, source authority, or time boundary is made explicit. 4. If one position lacks required evidence, record that finding without erasing the original contribution. 5. If both positions are valid in different contexts, split or refine the Policy, applicability, Rubric, Case, or coverage facet that conflated them. 6. Have the accountable owner approve the resulting governed change. Expert participation alone does not approve it. 7. Mark affected current evidence for follow-up, create new versions or a Snapshot where required, and preserve older Runs under their original boundary. ## Object and state changes - Correct the **Case** when required context or the judged output is wrong. - Correct the **Policy** when the behavioral rule or its scope is wrong. - Correct the **Rubric** when the test does not faithfully check the Policy. - Correct **coverage facets or membership** when the benchmark over- or under-represents a boundary. - Create a new **Benchmark Version** when the governed evaluation boundary changes. - Keep an unresolved observation explicit when the source evidence cannot yet support a decision. ## Success criteria - A reviewer can see the original positions and the evidence behind each. - The resolution names the artifact and version that changed. - Approval authority is explicit. - Downstream Case selection, standards, Snapshots, Runs, or customer-owned human review context are either still valid under a named boundary or routed for refresh. - The team did not manufacture agreement by deleting dissenting evidence. ## Common failure modes - Voting before reconstructing the evidence and applicability behind each position. - Editing a downstream Rubric when the conflict belongs to a Case or Policy boundary. - Treating expert participation as approval of a governed object. - Erasing dissent or historical versions after a resolution is approved. {% example-demo title="Two valid refund rules" %} One specialist rejects every refund exception; another approves exceptions for enterprise accounts. Their Cases reveal that both followed different authoritative programs. The team adds an account-program applicability boundary, revises the Policy and linked Rubrics, records the approval in activity history, and creates a new Benchmark Version. The earlier expert responses remain attributable evidence for why the split was needed. {% /example-demo %} ## Source confidence Code-backed: Policy and Rubric detail routes expose governed objects, linked Cases, approval, and activity history, while Contribution review results preserve attributable expert learning. The evidence-reconciliation method is doctrine-backed; no single product screen automatically adjudicates every cross-object conflict. ## Related reference pages {% related-card-grid title="Related workflows" %} - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) - [Versions, staleness, and resolution](/docs/object-model/versions-staleness-and-resolution) - [Approval History and Reviewer Activity](/docs/governance/approval-history-and-reviewer-activity) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Diagnose disagreement" %} - [Low expert agreement](/docs/troubleshooting/low-expert-agreement) - [Unclear Cases](/docs/troubleshooting/unclear-cases) - [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics) {% /related-card-grid %}