# Rubrics Generated: 2026-09-13T04:34:26.640Z Source build: local Canonical docs: https://teammately.ai/docs --- id: object-model.rubrics title: Rubrics summary: Turn policies into binary, reviewable checks that explain pass and fail evidence. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/rubrics --- # Rubrics ## Definition Rubrics turn policies into reviewable checks for a specific output. A good rubric makes pass, fail, or incomplete evidence explainable without replacing the policy it tests. Use this reference when a benchmark failure needs to show exactly which condition a candidate output satisfied or violated. ## Fields, states, or lifecycle rules - Rubrics should be tied to a policy and an applicability boundary. - A binary rubric should be narrow enough to judge one output clearly. - Linked Cases make the intended boundary concrete; they do not replace the Rubric's applicability rule. - Overlapping rubrics can make failures hard to interpret. - Rubric versions matter when pass/fail logic changes between benchmark runs. - This page describes rubric semantics, not a public scoring API. ## Related objects Rubrics should be read with [Policies](/docs/object-model/policies), [Applicability logic](/docs/object-model/applicability-logic), [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics), and [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics). {% example-demo title="Rubrics boundary" %} Policy: Compatibility claims require explicit source support. Rubric: Pass only if the answer cites explicit compatibility support or states that compatibility is uncertain. Interpretation: A candidate output that simply says "yes, it works" fails because the rubric names the missing evidence. {% /example-demo %} ## Source confidence Code-backed: Rubric types and list/detail routes expose criteria, Policy and Case links, applicability, and versioned evaluation context. The reference explains product semantics and does not promise a public scoring schema. ## Related task pages {% related-card-grid title="Related task pages" %} - [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics) - [Create a rubric](/docs/correctness-governance/policies-and-rubrics) - [Policies](/docs/object-model/policies) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: correctness.binary-rubrics title: Write Binary Rubrics summary: Write atomic pass-or-fail criteria grounded in governed policies, applicable cases, and observable candidate behavior. kind: task product_area: correctness_governance status: stable updated: 2026-08-22 canonical: /docs/correctness-governance/binary-rubrics --- # Write Binary Rubrics Write a rubric when a governed policy needs an observable pass-or-fail check for benchmark evaluation. A strong rubric identifies one behavior, the cases where it applies, and the evidence that distinguishes pass from fail. ## Prerequisites - A policy or expert-grounded correctness statement. - Representative passing, failing, and boundary cases. - Clear applicability for the behavior being checked. - Access to Correctness Governance → Rubrics. ## Steps 1. State one behavior that can be inspected in the candidate response and visible case evidence. 2. Name the policy or specialist judgment that authorizes the criterion. 3. Define applicability before writing exceptions into the pass condition. 4. Write explicit pass evidence and fail evidence. Avoid “good,” “appropriate,” or “high quality” without observable conditions. 5. Link representative cases and test whether two informed reviewers would reach the same binary result. 6. Split independent requirements into separate rubrics when each failure should be diagnosed separately. 7. Inspect contribution provenance and approval state before relying on the rubric in benchmark interpretation. ## Object and state changes This task creates or updates a project-level rubric and can change its wording, policy relationship, linked cases, evaluation use, activity, and approval context. Linking a case does not add it to a benchmark dataset. Editing a rubric does not alter historical Run evidence that used an earlier benchmark boundary. ## Success criteria - The rubric tests one behavior and can be answered from visible evidence. - Applicability excludes irrelevant cases without hidden reviewer judgment. - Pass and fail conditions are explicit. - Linked cases include at least one meaningful boundary. - Policy authority and expert provenance are inspectable. ## Common failure modes - Combining several behaviors into one criterion. - Restating the policy without defining observable evidence. - Encoding applicability only as exceptions inside the rubric. - Using a suggested or contributed draft as if it were already governed. - Changing rubric wording and comparing Runs without checking the benchmark version boundary. {% example-demo title="Example: grounding rubric" %} Policy: material claims must use the controlling source or state uncertainty. Rubric: pass only when every material claim is supported by the current controlling source, or the response explicitly says the available sources do not resolve the claim. Unsupported blending of current and superseded sources fails. {% /example-demo %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Applicability Logic](/docs/object-model/applicability-logic) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics) - [Weak applicability logic](/docs/troubleshooting/weak-applicability-logic) - [Low expert agreement](/docs/troubleshooting/low-expert-agreement) {% /related-card-grid %} ## Source confidence Code-backed: the active Correctness Governance rubric list and detail surfaces support rubric inspection, relationships, and lifecycle context. The drafting guidance is constrained to those verified artifact boundaries. --- id: object-model.policies title: Policies summary: Define policies as reusable statements of what correct AI behavior requires. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/policies --- # Policies ## Definition Policies are reusable statements of what correct behavior requires. They preserve expert judgment as a governed standard that can later be tested by applicability logic and rubrics. Use this reference when a team needs to know whether a rule is a draft suggestion, an approved correctness standard, or a standard whose version may affect benchmark evidence. ## Fields, states, or lifecycle rules - A policy names the rule; a rubric tests the rule. - A policy needs applicability logic before the team can know which cases it should judge. - Policy approval state matters before benchmark use. - Policy revisions can make older benchmark evidence stale or require comparison notes. - This page does not define legal, compliance, retention, or external policy-management guarantees. ## Related objects Policies should be read with [Applicability logic](/docs/object-model/applicability-logic), [Rubrics](/docs/object-model/rubrics), [Policy versions](/docs/governance/policy-versioning), and [Human Approval Boundaries](/docs/governance/human-approval-boundaries). Use [Create a policy](/docs/correctness-governance/policies-and-rubrics) for the operating workflow. {% example-demo title="Policies boundary" %} Expert judgment: Answers should not claim compatibility unless source data explicitly supports the claim. Policy: Compatibility claims require explicit source support. Interpretation: The policy explains the standard; linked applicability and rubrics decide when and how a specific output is judged. {% /example-demo %} ## Source confidence Code-backed: Policy types and list/detail routes expose identity, description, applicability, linked Cases and Rubrics, version facts, and approval state. The product object is a governed correctness standard, not an external legal-policy system. ## Related task pages {% related-card-grid title="Related task pages" %} - [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Applicability logic](/docs/object-model/applicability-logic) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: governance.rubric-versioning title: Rubric Versioning summary: Track changes to pass/fail criteria and the evidence they produce. kind: reference product_area: governance status: stable updated: 2026-08-23 canonical: /docs/governance/rubric-versioning --- # Rubric Versioning ## Definition A Rubric version preserves one observable binary evaluation criterion and its relationship to a Policy, applicability boundary, and normative force. The durable Rubric identity connects revisions; the version identity explains which exact criterion produced a historical result. ## Fields, states, or lifecycle rules - Change the version when criterion meaning, pass/fail threshold, applicability, Policy relationship, or normative force changes. - Editorial cleanup that genuinely preserves meaning should not be described as a new correctness rule without evidence. - A Rubric must remain testable as one binary criterion; combining several independent requirements makes failures ambiguous. - Suggested or contributed wording is not admitted evaluation authority until the owning workflow approves it. - Existing Runs retain the exact Rubric version they evaluated. - A new Rubric version can require a new Benchmark Version before it affects future evidence. - Prefer-level improvement does not cancel a must-level regression. ## Related objects Use Correctness Governance to inspect criterion text, Policy linkage, applicability, Cases, and approval state. Use Binary Rubrics when authoring or splitting a criterion. Use Benchmark Versioning when the revised Rubric should become part of future evaluation evidence. {% example-demo title="Splitting an ambiguous criterion" %} A Rubric currently passes only when an answer cites the controlling source and states uncertainty when no source is available. Reviewers find that the combined criterion produces unclear failures. They retain the original historical version and approve two narrower Rubrics for citation support and missing-source uncertainty. Future Benchmark Versions admit the new criteria; old Runs keep their original result meaning. {% /example-demo %} ## Source confidence Code-backed: Rubric types and detail routes expose criterion, version, Policy relationship, applicability, normative force, linked Cases, and evaluation context. Linked authoring and Benchmark pages define approval and admission boundaries. ## Related task pages {% related-card-grid title="Related task pages" %} - [Rubrics](/docs/object-model/rubrics) - [Binary Rubrics](/docs/correctness-governance/binary-rubrics) - [Versioning and Staleness](/docs/governance/versioning-and-staleness) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %}