Teammately Docs
Docs menu

error

Overly Broad Policies

Narrow policies that are too vague to guide review, rubrics, or human reviews.

Overly Broad Policies

Use this when a policy is too general to guide review, rubric writing, or human reviews.

Symptom

Reviewers agree with the policy in principle but cannot decide whether a specific output satisfies it. Rubrics derived from the policy become vague, noisy, or impossible to apply consistently.

Likely causes

  • The policy describes a value such as “be helpful” without a behavioral boundary.
  • Several domain rules were combined into one policy.
  • The policy lacks linked cases that show pass and fail examples.
  • The policy was approved from one expert comment without broader review.

Diagnostic checks

  • Ask whether the policy can produce a binary rubric without adding new assumptions.
  • Find linked cases that should pass and fail the policy.
  • Compare the policy to existing standards for duplicate or conflicting scope.
  • Review benchmark failures to see whether the policy explains the evidence.

Fix

  • Narrow the policy to one behavior boundary.
  • Split unrelated requirements into separate policies.
  • Add linked cases and applicability before approval.
  • Use the Policy's own governed approval and version controls after revision. A Contribution Checkpoint confirms Contribution progress; it is not automatically Policy approval.

Prevention

  • Draft policies from repeated judgments, not one-off comments.
  • Require applicability and examples before approval.
  • Review policy overlap during standards maintenance.
  • Version revisions so old benchmark evidence remains interpretable.

Related task pages

Reference / Correctness GovernancePolicies and RubricsUnderstand the governed relationship between behavior policies, applicability, binary rubrics, linked cases, and expert provenance.Concept / Correctness ElicitationCorrectness ElicitationTurn tacit specialist judgment into attributable contributions, governed policies, applicability conditions, and binary rubrics.Task / Governance and ReproducibilityResolve conflicting correctness evidenceReconcile disagreement without hiding the Cases, expert judgments, sources, or versions that produced it.

Related reference pages

Reference / Object ModelPoliciesDefine policies as reusable statements of what correct AI behavior requires.Reference / Governance and ReproducibilityPolicy VersioningPreserve policy changes so decisions can be interpreted against the standard used at the time.Concept / Correctness ElicitationCorrectness ElicitationTurn tacit specialist judgment into attributable contributions, governed policies, applicability conditions, and binary rubrics.

Source confidence

Code-backed: Policy list and detail routes expose governed Policy content, approval, versions, linked Cases, and Rubrics. This page keeps Contribution Checkpoints separate from Policy approval.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Object Model Policies Define policies as reusable statements of what correct AI behavior requires. Object Model Applicability logic Explain when a policy or rubric should be used for a case, output, or coverage segment. Governance and Reproducibility Resolve conflicting correctness evidence Reconcile disagreement without hiding the Cases, expert judgments, sources, or versions that produced it.
AI context