# Correctness Elicitation Generated: 2026-09-13T04:39:12.442Z Source build: local Canonical docs: https://teammately.ai/docs --- id: concepts.correctness-elicitation title: Correctness Elicitation summary: Turn tacit specialist judgment into attributable contributions, governed policies, applicability conditions, and binary rubrics. kind: concept product_area: correctness_elicitation status: stable updated: 2026-09-07 canonical: /docs/concepts/correctness-elicitation --- # Correctness Elicitation Correctness Elicitation is the capability for turning specialist judgment into explicit, reusable correctness specifications. It handles the parts of AI behavior that cannot be settled by a generic score: domain preferences, exceptions, conflicts between sources, applicability boundaries, unacceptable failure modes, and the evidence an expert needs before making a decision. ## Definition Elicitation begins before the expert opens a task. Teammately agents can organize relevant Reference Materials, cases, candidate responses, possible policies, rubric questions, and unresolved conflicts into a focused Contribution. The expert then works through forms, chat, interviews, case review, or checkpoints according to the requested components. The contribution remains attributable. Accepted learning can be materialized into policies, rubrics, cases, or coverage observations, while drafts and unresolved statements keep their own state. Correctness Governance owns the resulting policies and rubrics; Expert Contributions owns the benchmark-scoped human work that produced them. ## Decision checkpoint | Need | Elicitation method | Durable destination | | --- | --- | --- | | Confirm a known rule across several cases | Focused form or case review | Policy, applicability, or rubric contribution | | Discover reasoning that is hard to pre-structure | Chat or interview | Attributable transcript, checkpoints, and contributed artifacts | | Resolve disagreement or ambiguity | Targeted comparison and checkpoint | Explicit unresolved or accepted statement | | Identify missing benchmark behavior | Cases plus coverage questions | Contributed case or coverage observation | | Configure reviewer presentation | Assets → Review Screens | Reusable Review Screen | | Set a contribution's agent behavior | Expert Contribution | Contribution behavior and selected components | ## Why preparation matters An expert should not have to reconstruct the project, search for the controlling source, or infer why a case was selected. Project Context gives the agent the stable brief. Reference Materials provide indexed project knowledge. The Contribution selects the benchmark evidence, states the objective, and configures agent behavior. Review Screen controls presentation. This separation preserves authority. Reference material can inform an answer without becoming a policy. Agent-authored draft wording can focus the expert without becoming approved. A completed expert task can contribute evidence without automatically changing every project artifact. ## Relationship to the product Correctness Elicitation is broader than any single screen. Use **Expert Contributions** to request and track benchmark-scoped expert work. Use **Correctness Governance** to inspect and maintain policies and rubrics after they are materialized. Use **Agent Setup** to configure what agents understand, **Assets → Review Screens** for reusable expert-facing presentation, and the Contribution itself for scoped behavior and components. The capability also returns learning to Coverage Engineering and Weave. An expert may identify a missing behavior combination, contribute a new case, or explain that existing material is insufficient. Those outputs should update the owning coverage or case artifacts instead of being flattened into a general review note. {% example-demo title="Exception handling" %} An agent prepares three cases where a procurement rule might allow an exception, attaches the controlling policy material, and asks a specialist to distinguish approved exceptions from escalation-only situations. The specialist completes a case review and confirms a checkpoint. The accepted contribution materializes a scoped policy and binary rubric, while one unresolved source conflict becomes a coverage observation for follow-up. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Manage policies and rubrics](/docs/correctness-governance) - [Configure Agent Setup](/docs/agent-setup) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) - [Policies](/docs/object-model/policies) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Source confidence Doctrine-backed: this page defines the public capability. The linked product pages are code-backed and define the active contribution, governance, and agent-configuration surfaces. --- id: expert-contributions.overview title: Expert Contributions summary: Coordinate benchmark-scoped expert work, attributable judgment, governed artifacts, and the decisions that move correctness forward. kind: concept product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions --- # Expert Contributions Expert Contributions is the benchmark-scoped workspace for requesting, conducting, and materializing specialist work. It coordinates the expert, objective, selected evidence, task sequence, checkpoints, attributable responses, and contributed artifacts needed to move a benchmark forward. ## Definition The administrator workspace contains **Overview**, **Contributions**, **Contributed Artifacts**, and **Logs & Status**. **Request Contribution** opens the composer for a new contribution. The expert follows a contribution-specific experience that can contain form, chat, interview, and case-review tasks, along with checkpoints and completion states. A Contribution is the unit of requested expert effort. It replaces broad workflow configuration with a bounded statement of what this benchmark needs from this expert now. The work can result in contributed policies, rubrics, cases, or coverage observations without flattening all expert activity into one generic approval record. ## Decision checkpoint | Need | Contribution element | Result to inspect | | --- | --- | --- | | Resolve a specific benchmark question | Contribution statement and scoped objectives | The expert can explain the requested decision | | Ground work in concrete behavior | Selected or designated cases | Case-level responses remain attributable | | Supply supporting knowledge | Attachments and scoped statements | The expert sees the relevant source boundary | | Choose the right interaction | Form, chat, interview, or case review task | Task output matches the kind of judgment needed | | Confirm consequential learning | Checkpoint | Accepted, revised, or unresolved state is explicit | | Reuse the result | Contributed Artifacts | Policies, rubrics, cases, and coverage observations retain provenance | ## Lifecycle and status The durable Contribution statuses are `PREPARING_DIRECTION`, `AWAITING_DIRECTION_ALIGNMENT`, `MATERIALIZING_TASKS`, `READY`, `IN_PROGRESS`, `COMPLETED`, and `CANCELLED`. The interface presents these as planning direction, waiting for alignment, preparing tasks, ready, active, completed, or cancelled. The exact task sequence can vary by Contribution. Realtime updates and durable transitions help the administrator and expert see current progress without inventing completion. A waiting state, checkpoint, or finalization step should be shown as such. Completing the expert experience does not imply that every proposed artifact has been accepted into its project-level owner. ## Contribution evidence Logs & Status exposes operational and engagement records. Contributed Artifacts organizes materialized or contributed cases, policies, rubrics, and new coverage observations. Correctness Governance, Assets, or Coverage Management owns the resulting project or benchmark artifact after materialization. This model improves return on expert effort. Agents prepare focused work from project context, indexed material, benchmark cases, and unresolved questions. The expert supplies the authority; the result can be reused across standards, coverage, evaluation, and improvement. {% example-demo title="Resolve source authority" %} A benchmark contains cases where an operational runbook conflicts with a newer policy page. The operator requests a Contribution from the policy owner, selects the conflicting cases, attaches both sources, and uses case review plus a checkpoint. The expert establishes which source controls, contributes a scoped policy and rubric, and records one coverage observation for an unrepresented exception. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Contribution lifecycle and status](/docs/expert-contributions/lifecycle-and-status) - [Logs & Status](/docs/expert-contributions/logs-and-status) - [Agent Setup](/docs/agent-setup) - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active benchmark workspace, Contribution dashboard, composer, administrator detail, and expert routes support the scope, task, status, and artifact model described here. --- id: correctness.overview title: Correctness Governance summary: Govern project policies and rubrics, their applicability, linked cases, approval state, and contribution provenance. kind: concept product_area: correctness_governance status: stable updated: 2026-09-07 canonical: /docs/correctness-governance --- # Correctness Governance Correctness Governance is the project-level surface for policies and rubrics. It makes the specialist standards used by expert work and benchmark evaluation inspectable, attributable, and reusable across benchmark workspaces. ## Definition The surface has **Policies** and **Rubrics**. Policy pages describe a behavior rule, its scope, linked examples, approval state, activity, and connected rubrics. Rubric pages define testable evaluation criteria and show how they relate to cases, policies, and evaluation use. Correctness Elicitation is the broader capability that discovers and resolves specialist judgment. Correctness Governance is the product surface that owns the resulting governed project artifacts. Expert Contributions may propose or contribute policies and rubrics, but those artifacts retain their own lifecycle and provenance. ## Decision checkpoint | Need | Inspect or change | Boundary | | --- | --- | --- | | State a reusable behavior rule | Policy | Keep source and expert rationale visible | | Decide where a rule applies | Policy scope and applicability | Do not encode broad intent only in rubric wording | | Make the rule testable | Binary rubric | Define observable pass and fail evidence | | Connect standards to examples | Linked cases | A linked case does not automatically belong to every benchmark | | Review expert-originated material | Contribution provenance and approval | Contribution output is not silently governed | | Explain evaluation movement | Policy, rubric, case, and version context | Do not rely on an aggregate score alone | ## Policies and rubrics A policy explains what behavior should occur and why. Applicability determines the situations in which the rule controls. A rubric turns that rule into an evaluation question whose result can be traced to observable behavior. Several rubrics may operationalize different parts of one policy, and cases can help demonstrate where each rubric applies. Rubrics should remain atomic enough to interpret. If one rubric simultaneously checks grounding, tone, escalation, and completeness, a failure does not identify the responsible behavior. Split criteria where independent failure evidence matters. ## Human and agent roles Agents can draft possible wording, surface linked cases, identify proposed applications, and prepare follow-up questions. Experts contribute domain authority through benchmark-scoped work. Project operators inspect and maintain the governed artifacts. Approval history and activity should make the transition between proposal, contribution, and governed state visible. Changing a project-level policy or rubric may affect several benchmarks. Before interpreting a later Run, confirm which benchmark version and standard boundary it used. {% example-demo title="Unsupported compatibility" %} A specialist confirms that compatibility may be claimed only when an authoritative source explicitly supports the exact equipment combination. Correctness Governance records the policy, scopes it to recommendation and validation responses, links representative cases, and defines a binary rubric that passes only when the response cites support or clearly states uncertainty. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) - [Write binary rubrics](/docs/correctness-governance/binary-rubrics) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Policies](/docs/object-model/policies) - [Applicability Logic](/docs/object-model/applicability-logic) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Source confidence Code-backed: the active Correctness Governance layout, policy list, rubric list, and detail surfaces establish the current ownership and relationships described here. --- id: agent-setup.overview title: Agent Setup summary: Configure Project Context and Reference Materials so Teammately agents have reusable project understanding before contribution work. kind: concept product_area: agent_setup status: stable updated: 2026-09-07 canonical: /docs/agent-setup --- # Agent Setup Agent Setup is the project-level workspace for configuring reusable project understanding before Teammately agents prepare or conduct expert contribution work. Expert-facing presentation and contribution-specific behavior are configured through Assets and the Contribution workflow. ## Definition Agent Setup contains one project-understanding group: - **Project Understanding:** Project Context and Reference Materials. These settings are reusable project foundations rather than settings for one benchmark or one expert. Review Screens and Comparison Directions are project Assets, not Agent Setup tabs. Project Context contains the Project Agent Brief. Reference Materials uses the Materials and Indexed Reference tabs to organize project knowledge for agents. A Contribution selects its benchmark-specific objective, components, and agent behavior; Review Screens control reusable expert-facing presentation from Assets. ## Decision checkpoint | Need | Open | Keep distinct from... | | --- | --- | --- | | Explain the project, target behavior, and stable operating context | Project Context | Project name or memo in General settings | | Supply manuals, sites, repositories, or files to agents | Reference Materials | Governed policies, rubrics, and case materials | | Set benchmark-specific agent behavior | Expert Contribution | Project-wide context and screen configuration | | Guide meaningful response variation | Assets → Comparison Directions | Coverage facets, generated cases, or approved standards | | Configure what an expert sees while reviewing | Assets → Review Screens | Contribution objectives and selected cases | ## Project and benchmark scope Agent Setup belongs to the project because the same project context may support many benchmarks. A benchmark-specific Contribution still selects its own objective, expert, cases, attachments, contribution components, and agent behavior. Review Screens and Comparison Directions are authored under Assets and selected when the Contribution needs them. Agent Setup provides the reusable understanding foundation; it does not create or schedule contribution work by itself. Changes can affect future agent preparation. Before making broad edits, inspect active benchmark work and confirm whether the new context should apply across the project. A narrow contribution-specific request belongs in the Contribution rather than in permanent Agent Setup. ## Authority boundaries Reference Materials can inform agents but does not automatically create policies or rubrics. Review Screens change presentation and requested inputs, not the meaning of the underlying case or standard. Contribution configuration guides agent behavior but cannot supply human approval. These boundaries make contribution evidence interpretable. Another operator can distinguish what the project told the agent, what evidence the contribution supplied, what the agent proposed, and what the expert decided. {% example-demo title="Policy-review preparation" %} Project Context explains that the assistant must prioritize the current procurement agreement. Reference Materials indexes the agreement repository. A Review Screen shows the controlling document and relevant case-material fields. The benchmark Contribution then asks an expert to decide which behavior should become policy. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Product quickstart](/docs/quickstart) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Configure Project Input Schema](/docs/project-settings/input-schema) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Project Context](/docs/agent-setup/project-context) - [Reference Materials](/docs/agent-setup/reference-materials) - [Comparison Directions](/docs/assets/comparison-directions) - [Review Screens](/docs/assets/review-screens) {% /related-card-grid %} ## Source confidence Code-backed: the active Agent Setup layout and project navigation define these groups, labels, and routes.