# Rubric Versioning Generated: 2026-09-13T04:39:34.043Z Source build: local Canonical docs: https://teammately.ai/docs --- id: governance.rubric-versioning title: Rubric Versioning summary: Track changes to pass/fail criteria and the evidence they produce. kind: reference product_area: governance status: stable updated: 2026-08-23 canonical: /docs/governance/rubric-versioning --- # Rubric Versioning ## Definition A Rubric version preserves one observable binary evaluation criterion and its relationship to a Policy, applicability boundary, and normative force. The durable Rubric identity connects revisions; the version identity explains which exact criterion produced a historical result. ## Fields, states, or lifecycle rules - Change the version when criterion meaning, pass/fail threshold, applicability, Policy relationship, or normative force changes. - Editorial cleanup that genuinely preserves meaning should not be described as a new correctness rule without evidence. - A Rubric must remain testable as one binary criterion; combining several independent requirements makes failures ambiguous. - Suggested or contributed wording is not admitted evaluation authority until the owning workflow approves it. - Existing Runs retain the exact Rubric version they evaluated. - A new Rubric version can require a new Benchmark Version before it affects future evidence. - Prefer-level improvement does not cancel a must-level regression. ## Related objects Use Correctness Governance to inspect criterion text, Policy linkage, applicability, Cases, and approval state. Use Binary Rubrics when authoring or splitting a criterion. Use Benchmark Versioning when the revised Rubric should become part of future evaluation evidence. {% example-demo title="Splitting an ambiguous criterion" %} A Rubric currently passes only when an answer cites the controlling source and states uncertainty when no source is available. Reviewers find that the combined criterion produces unclear failures. They retain the original historical version and approve two narrower Rubrics for citation support and missing-source uncertainty. Future Benchmark Versions admit the new criteria; old Runs keep their original result meaning. {% /example-demo %} ## Source confidence Code-backed: Rubric types and detail routes expose criterion, version, Policy relationship, applicability, normative force, linked Cases, and evaluation context. Linked authoring and Benchmark pages define approval and admission boundaries. ## Related task pages {% related-card-grid title="Related task pages" %} - [Rubrics](/docs/object-model/rubrics) - [Binary Rubrics](/docs/correctness-governance/binary-rubrics) - [Versioning and Staleness](/docs/governance/versioning-and-staleness) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.rubrics title: Rubrics summary: Turn policies into binary, reviewable checks that explain pass and fail evidence. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/rubrics --- # Rubrics ## Definition Rubrics turn policies into reviewable checks for a specific output. A good rubric makes pass, fail, or incomplete evidence explainable without replacing the policy it tests. Use this reference when a benchmark failure needs to show exactly which condition a candidate output satisfied or violated. ## Fields, states, or lifecycle rules - Rubrics should be tied to a policy and an applicability boundary. - A binary rubric should be narrow enough to judge one output clearly. - Linked Cases make the intended boundary concrete; they do not replace the Rubric's applicability rule. - Overlapping rubrics can make failures hard to interpret. - Rubric versions matter when pass/fail logic changes between benchmark runs. - This page describes rubric semantics, not a public scoring API. ## Related objects Rubrics should be read with [Policies](/docs/object-model/policies), [Applicability logic](/docs/object-model/applicability-logic), [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics), and [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics). {% example-demo title="Rubrics boundary" %} Policy: Compatibility claims require explicit source support. Rubric: Pass only if the answer cites explicit compatibility support or states that compatibility is uncertain. Interpretation: A candidate output that simply says "yes, it works" fails because the rubric names the missing evidence. {% /example-demo %} ## Source confidence Code-backed: Rubric types and list/detail routes expose criteria, Policy and Case links, applicability, and versioned evaluation context. The reference explains product semantics and does not promise a public scoring schema. ## Related task pages {% related-card-grid title="Related task pages" %} - [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics) - [Create a rubric](/docs/correctness-governance/policies-and-rubrics) - [Policies](/docs/object-model/policies) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: governance.versioning-staleness title: Versioning and Staleness summary: Know when correctness objects changed and when old evidence may need review. kind: concept product_area: governance status: stable updated: 2026-09-07 canonical: /docs/governance/versioning-and-staleness --- # Versioning and Staleness ## Definition Versioning preserves the exact artifact state used by an earlier decision or evaluation. Staleness is the signal that current evidence, configuration, or interpretation may no longer support the same claim after a related object changes. A stale signal routes review; it does not automatically delete an artifact, invalidate every historical result, or approve a replacement. ## Why it matters Cases, Policies, Rubrics, Benchmark Versions, Harness Versions, and Improvement Session goals can change independently. Named versions keep old evidence interpretable. Staleness helps teams decide which current Datasets, evaluator links, Runs, or customer-owned human review context need attention before being treated as current. ## Where it appears in the product Use the owning object page to inspect its current version and activity. Use Dataset Snapshots and Benchmark Versioning for immutable evaluation boundaries. Use Staleness Detection to identify downstream artifacts affected by a change. Use Conflict Resolution when expert or source evidence disagrees about what the new governed state should be. ## Artifacts it affects Common triggers include changed Case input or materials, revised Policy scope, revised Rubric criteria, changed dataset membership, changed Harness configuration, and superseded source material. The responsible next action depends on the owner: correct a Case, approve a new standard version, refresh coverage, create a new Snapshot, run a new evaluation, or preserve an old result as historical context. Comparison Directions use a narrower stale signal. **Potentially stale** is an advisory label for untouched AI-suggested directions, not a versioned approval state and not automatic removal. User-created and user-edited directions remain user-owned even when Teammately considers them while avoiding duplicate suggestions. ## Operational check Name the changed artifact and version, identify which downstream claim depended on it, and decide whether the old evidence remains historically valid, requires qualification, or needs replacement through a new canonical workflow. Never “resolve” staleness by editing a label while leaving the evidence boundary ambiguous. When the object is a Comparison Direction, also check whether a **Potentially stale** label is only advisory. Dismiss the label if the team decides the direction still represents a useful boundary. {% example-demo title="Revised applicability after evaluation" %} Experts revise a Policy so it applies only when the customer explicitly requests a recommendation. Runs against the old Benchmark Version remain valid evidence under the former applicability rule. The current Dataset and linked Rubrics are reviewed, a new Snapshot and Benchmark Version establish the revised boundary, and new Runs use it. Any customer-owned human review context names both boundaries instead of marking every old result simply “wrong.” {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Versions, staleness, and resolution](/docs/object-model/versions-staleness-and-resolution) - [Staleness Detection](/docs/governance/staleness-detection) - [Compare Harness Versions](/docs/benchmark-evaluations/compare) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} ## Source confidence Code-backed: current Policy and Rubric types preserve versioned governance facts, Snapshot routes preserve immutable benchmark evidence, and Comparison Directions expose a deliberately narrower advisory stale label. Cross-object staleness remains a review and routing decision, not an automatic global state transition. --- id: correctness.binary-rubrics title: Write Binary Rubrics summary: Write atomic pass-or-fail criteria grounded in governed policies, applicable cases, and observable candidate behavior. kind: task product_area: correctness_governance status: stable updated: 2026-08-22 canonical: /docs/correctness-governance/binary-rubrics --- # Write Binary Rubrics Write a rubric when a governed policy needs an observable pass-or-fail check for benchmark evaluation. A strong rubric identifies one behavior, the cases where it applies, and the evidence that distinguishes pass from fail. ## Prerequisites - A policy or expert-grounded correctness statement. - Representative passing, failing, and boundary cases. - Clear applicability for the behavior being checked. - Access to Correctness Governance → Rubrics. ## Steps 1. State one behavior that can be inspected in the candidate response and visible case evidence. 2. Name the policy or specialist judgment that authorizes the criterion. 3. Define applicability before writing exceptions into the pass condition. 4. Write explicit pass evidence and fail evidence. Avoid “good,” “appropriate,” or “high quality” without observable conditions. 5. Link representative cases and test whether two informed reviewers would reach the same binary result. 6. Split independent requirements into separate rubrics when each failure should be diagnosed separately. 7. Inspect contribution provenance and approval state before relying on the rubric in benchmark interpretation. ## Object and state changes This task creates or updates a project-level rubric and can change its wording, policy relationship, linked cases, evaluation use, activity, and approval context. Linking a case does not add it to a benchmark dataset. Editing a rubric does not alter historical Run evidence that used an earlier benchmark boundary. ## Success criteria - The rubric tests one behavior and can be answered from visible evidence. - Applicability excludes irrelevant cases without hidden reviewer judgment. - Pass and fail conditions are explicit. - Linked cases include at least one meaningful boundary. - Policy authority and expert provenance are inspectable. ## Common failure modes - Combining several behaviors into one criterion. - Restating the policy without defining observable evidence. - Encoding applicability only as exceptions inside the rubric. - Using a suggested or contributed draft as if it were already governed. - Changing rubric wording and comparing Runs without checking the benchmark version boundary. {% example-demo title="Example: grounding rubric" %} Policy: material claims must use the controlling source or state uncertainty. Rubric: pass only when every material claim is supported by the current controlling source, or the response explicitly says the available sources do not resolve the claim. Unsupported blending of current and superseded sources fails. {% /example-demo %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Applicability Logic](/docs/object-model/applicability-logic) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics) - [Weak applicability logic](/docs/troubleshooting/weak-applicability-logic) - [Low expert agreement](/docs/troubleshooting/low-expert-agreement) {% /related-card-grid %} ## Source confidence Code-backed: the active Correctness Governance rubric list and detail surfaces support rubric inspection, relationships, and lifecycle context. The drafting guidance is constrained to those verified artifact boundaries.