# Build policies and rubrics Generated: 2026-09-13T04:43:22.247Z Source build: local Canonical docs: https://teammately.ai/docs --- id: operating.build-policies-rubrics title: Build policies and rubrics summary: Materialize expert-grounded behavior rules, applicability, and binary criteria in Correctness Governance. kind: task product_area: operating_manual status: stable updated: 2026-08-22 canonical: /docs/operating-manual/build-policies-and-rubrics --- # Build policies and rubrics Build a governed policy and its rubrics when specialist judgment is clear enough to become reusable across cases, Contributions, and benchmark evaluations. ## Prerequisites - Attributable expert judgment or another accountable source of authority. - Representative and boundary Cases. - A clear behavior rule and enough context to define applicability. - Access to Correctness Governance. ## Steps 1. Inspect the Expert Contribution, source material, cases, and checkpoints that support the proposed rule. 2. Write the policy as expected behavior, not as a score or implementation technique. 3. Define applicability: the situations, inputs, or conditions where the policy controls. 4. Link representative cases that show ordinary, passing, failing, and boundary behavior. 5. Create binary rubrics that test observable parts of the policy. 6. Split independent criteria so each failure remains diagnosable. 7. Inspect approval, activity, provenance, and proposed application state in Correctness Governance. 8. Confirm the benchmark version boundary before using the standard in evaluation interpretation. ## Object and state changes This task creates or updates project Policies, applicability, Rubrics, case links, activity, approval context, and contribution provenance. It can affect future benchmark versions and evaluations. Historical Runs retain the correctness boundary recorded when they ran. ## Success criteria - The policy expresses one reusable behavior rule and its authority. - Applicability distinguishes relevant from irrelevant Cases. - Rubrics define observable pass and fail evidence. - Linked cases demonstrate meaningful boundaries. - Suggested, contributed, and governed states are not conflated. - Later evaluation results can trace a failure back to the rule and evidence. ## Common failure modes - Turning a source document directly into a policy without expert interpretation. - Writing a policy so broad that applicability cannot be inspected. - Combining unrelated criteria into one rubric. - Treating contribution completion as automatic governance. - Comparing Runs across a changed policy or rubric boundary without acknowledging it. {% example-demo title="Example: exception escalation standard" %} An expert confirms that unresolved eligibility exceptions must be escalated. The policy states the rule and its applicability. One rubric checks that the response avoids promising an exception; another checks the correct escalation path. Linked Cases include both ordinary and conflicting-source situations. {% /example-demo %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Correctness Governance](/docs/correctness-governance) - [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Overly broad policies](/docs/troubleshooting/overly-broad-policies) - [Weak applicability logic](/docs/troubleshooting/weak-applicability-logic) - [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics) {% /related-card-grid %} ## Source confidence Code-backed: current policy and rubric list/detail surfaces support governed artifacts, links, approval context, and contribution provenance used in this task. --- id: correctness.overview title: Correctness Governance summary: Govern project policies and rubrics, their applicability, linked cases, approval state, and contribution provenance. kind: concept product_area: correctness_governance status: stable updated: 2026-09-07 canonical: /docs/correctness-governance --- # Correctness Governance Correctness Governance is the project-level surface for policies and rubrics. It makes the specialist standards used by expert work and benchmark evaluation inspectable, attributable, and reusable across benchmark workspaces. ## Definition The surface has **Policies** and **Rubrics**. Policy pages describe a behavior rule, its scope, linked examples, approval state, activity, and connected rubrics. Rubric pages define testable evaluation criteria and show how they relate to cases, policies, and evaluation use. Correctness Elicitation is the broader capability that discovers and resolves specialist judgment. Correctness Governance is the product surface that owns the resulting governed project artifacts. Expert Contributions may propose or contribute policies and rubrics, but those artifacts retain their own lifecycle and provenance. ## Decision checkpoint | Need | Inspect or change | Boundary | | --- | --- | --- | | State a reusable behavior rule | Policy | Keep source and expert rationale visible | | Decide where a rule applies | Policy scope and applicability | Do not encode broad intent only in rubric wording | | Make the rule testable | Binary rubric | Define observable pass and fail evidence | | Connect standards to examples | Linked cases | A linked case does not automatically belong to every benchmark | | Review expert-originated material | Contribution provenance and approval | Contribution output is not silently governed | | Explain evaluation movement | Policy, rubric, case, and version context | Do not rely on an aggregate score alone | ## Policies and rubrics A policy explains what behavior should occur and why. Applicability determines the situations in which the rule controls. A rubric turns that rule into an evaluation question whose result can be traced to observable behavior. Several rubrics may operationalize different parts of one policy, and cases can help demonstrate where each rubric applies. Rubrics should remain atomic enough to interpret. If one rubric simultaneously checks grounding, tone, escalation, and completeness, a failure does not identify the responsible behavior. Split criteria where independent failure evidence matters. ## Human and agent roles Agents can draft possible wording, surface linked cases, identify proposed applications, and prepare follow-up questions. Experts contribute domain authority through benchmark-scoped work. Project operators inspect and maintain the governed artifacts. Approval history and activity should make the transition between proposal, contribution, and governed state visible. Changing a project-level policy or rubric may affect several benchmarks. Before interpreting a later Run, confirm which benchmark version and standard boundary it used. {% example-demo title="Unsupported compatibility" %} A specialist confirms that compatibility may be claimed only when an authoritative source explicitly supports the exact equipment combination. Correctness Governance records the policy, scopes it to recommendation and validation responses, links representative cases, and defines a binary rubric that passes only when the response cites support or clearly states uncertainty. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) - [Write binary rubrics](/docs/correctness-governance/binary-rubrics) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Policies](/docs/object-model/policies) - [Applicability Logic](/docs/object-model/applicability-logic) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Source confidence Code-backed: the active Correctness Governance layout, policy list, rubric list, and detail surfaces establish the current ownership and relationships described here. --- id: concepts.correctness-elicitation title: Correctness Elicitation summary: Turn tacit specialist judgment into attributable contributions, governed policies, applicability conditions, and binary rubrics. kind: concept product_area: correctness_elicitation status: stable updated: 2026-09-07 canonical: /docs/concepts/correctness-elicitation --- # Correctness Elicitation Correctness Elicitation is the capability for turning specialist judgment into explicit, reusable correctness specifications. It handles the parts of AI behavior that cannot be settled by a generic score: domain preferences, exceptions, conflicts between sources, applicability boundaries, unacceptable failure modes, and the evidence an expert needs before making a decision. ## Definition Elicitation begins before the expert opens a task. Teammately agents can organize relevant Reference Materials, cases, candidate responses, possible policies, rubric questions, and unresolved conflicts into a focused Contribution. The expert then works through forms, chat, interviews, case review, or checkpoints according to the requested components. The contribution remains attributable. Accepted learning can be materialized into policies, rubrics, cases, or coverage observations, while drafts and unresolved statements keep their own state. Correctness Governance owns the resulting policies and rubrics; Expert Contributions owns the benchmark-scoped human work that produced them. ## Decision checkpoint | Need | Elicitation method | Durable destination | | --- | --- | --- | | Confirm a known rule across several cases | Focused form or case review | Policy, applicability, or rubric contribution | | Discover reasoning that is hard to pre-structure | Chat or interview | Attributable transcript, checkpoints, and contributed artifacts | | Resolve disagreement or ambiguity | Targeted comparison and checkpoint | Explicit unresolved or accepted statement | | Identify missing benchmark behavior | Cases plus coverage questions | Contributed case or coverage observation | | Configure reviewer presentation | Assets → Review Screens | Reusable Review Screen | | Set a contribution's agent behavior | Expert Contribution | Contribution behavior and selected components | ## Why preparation matters An expert should not have to reconstruct the project, search for the controlling source, or infer why a case was selected. Project Context gives the agent the stable brief. Reference Materials provide indexed project knowledge. The Contribution selects the benchmark evidence, states the objective, and configures agent behavior. Review Screen controls presentation. This separation preserves authority. Reference material can inform an answer without becoming a policy. Agent-authored draft wording can focus the expert without becoming approved. A completed expert task can contribute evidence without automatically changing every project artifact. ## Relationship to the product Correctness Elicitation is broader than any single screen. Use **Expert Contributions** to request and track benchmark-scoped expert work. Use **Correctness Governance** to inspect and maintain policies and rubrics after they are materialized. Use **Agent Setup** to configure what agents understand, **Assets → Review Screens** for reusable expert-facing presentation, and the Contribution itself for scoped behavior and components. The capability also returns learning to Coverage Engineering and Weave. An expert may identify a missing behavior combination, contribute a new case, or explain that existing material is insufficient. Those outputs should update the owning coverage or case artifacts instead of being flattened into a general review note. {% example-demo title="Exception handling" %} An agent prepares three cases where a procurement rule might allow an exception, attaches the controlling policy material, and asks a specialist to distinguish approved exceptions from escalation-only situations. The specialist completes a case review and confirms a checkpoint. The accepted contribution materializes a scoped policy and binary rubric, while one unresolved source conflict becomes a coverage observation for follow-up. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Manage policies and rubrics](/docs/correctness-governance) - [Configure Agent Setup](/docs/agent-setup) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) - [Policies](/docs/object-model/policies) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Source confidence Doctrine-backed: this page defines the public capability. The linked product pages are code-backed and define the active contribution, governance, and agent-configuration surfaces. --- id: expert-contributions.artifacts title: Contributed Artifacts summary: Inspect policies, rubrics, cases, and coverage observations produced through attributable expert contribution work. kind: reference product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions/contributed-artifacts --- # Contributed Artifacts ## Definition Contributed Artifacts is the benchmark workspace for inspecting durable material produced through Expert Contributions. It organizes contributed **Policies**, **Rubrics**, **Cases**, and **new coverage observations** while preserving their relationship to the Contribution and expert work that produced them. The view is a provenance and reconciliation surface. The final owner of a materialized artifact remains Correctness Governance, Assets, or Coverage Management according to artifact type. ## Fields, states, or lifecycle rules - Policy contributions represent expert-grounded behavior rules or revisions. - Rubric contributions represent proposed or accepted evaluation criteria tied to specialist judgment. - Case contributions represent situations supplied or corrected through expert work. - Coverage observations identify missing, thin, conflicting, or newly important benchmark behavior. - A coverage observation preserves its source, proposed facet applications, and application status so an operator can distinguish a recorded observation from one incorporated into coverage structure. - Each artifact should remain traceable to the Contribution, expert, selected evidence, task responses, and checkpoints that support it. - Contribution completion and artifact governance are separate transitions. Inspect the artifact's owning surface before treating it as active policy, active rubric, benchmark dataset membership, or resolved coverage. - Reconciliation can accept, revise, route, or leave material unresolved according to the active workflow. ## Interpreting contributed material Use the artifact type to choose the next surface. A contributed policy or rubric belongs in Correctness Governance. A contributed case belongs in the project Assets pool before benchmark selection. A coverage observation belongs in Coverage Management and may motivate a Coverage Story, case construction, or another focused Contribution. Preserve disagreements. Two experts can contribute conflicting Policy interpretations, and the artifact view should help an operator trace each interpretation rather than merge them into an invented consensus. Materialization should keep the Contribution, activity, checkpoint, expert, and scoped evidence links needed to explain why the artifact exists. {% example-demo title="Example: contribution provenance" %} An expert contributes a policy limiting compatibility claims, a rubric for explicit uncertainty, and a new case involving an unsupported adapter. The policy and rubric move to Correctness Governance for their lifecycle. The case enters Assets and is later selected into a benchmark dataset snapshot. All three retain the Contribution as their provenance. {% /example-demo %} ## Source confidence Code-backed: the active Contributed Artifacts workspace exposes policy, rubric, case, and new-coverage groupings. This page preserves the separation between contribution provenance and the lifecycle of each owning artifact. ## Related task pages {% related-card-grid title="Related task pages" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) {% /related-card-grid %}