Correctness Elicitation
Correctness Elicitation is the capability for turning specialist judgment into explicit, reusable correctness specifications. It handles the parts of AI behavior that cannot be settled by a generic score: domain preferences, exceptions, conflicts between sources, applicability boundaries, unacceptable failure modes, and the evidence an expert needs before making a decision.
Definition
Elicitation begins before the expert opens a task. Teammately agents can organize relevant Reference Materials, cases, candidate responses, possible policies, rubric questions, and unresolved conflicts into a focused Contribution. The expert then works through forms, chat, interviews, case review, or checkpoints according to the requested components.
The contribution remains attributable. Accepted learning can be materialized into policies, rubrics, cases, or coverage observations, while drafts and unresolved statements keep their own state. Correctness Governance owns the resulting policies and rubrics; Expert Contributions owns the benchmark-scoped human work that produced them.
Decision checkpoint
| Need | Elicitation method | Durable destination |
|---|---|---|
| Confirm a known rule across several cases | Focused form or case review | Policy, applicability, or rubric contribution |
| Discover reasoning that is hard to pre-structure | Chat or interview | Attributable transcript, checkpoints, and contributed artifacts |
| Resolve disagreement or ambiguity | Targeted comparison and checkpoint | Explicit unresolved or accepted statement |
| Identify missing benchmark behavior | Cases plus coverage questions | Contributed case or coverage observation |
| Configure reviewer presentation | Assets → Review Screens | Reusable Review Screen |
| Set a contribution's agent behavior | Expert Contribution | Contribution behavior and selected components |
Why preparation matters
An expert should not have to reconstruct the project, search for the controlling source, or infer why a case was selected. Project Context gives the agent the stable brief. Reference Materials provide indexed project knowledge. The Contribution selects the benchmark evidence, states the objective, and configures agent behavior. Review Screen controls presentation.
This separation preserves authority. Reference material can inform an answer without becoming a policy. Agent-authored draft wording can focus the expert without becoming approved. A completed expert task can contribute evidence without automatically changing every project artifact.
Relationship to the product
Correctness Elicitation is broader than any single screen. Use Expert Contributions to request and track benchmark-scoped expert work. Use Correctness Governance to inspect and maintain policies and rubrics after they are materialized. Use Agent Setup to configure what agents understand, Assets → Review Screens for reusable expert-facing presentation, and the Contribution itself for scoped behavior and components.
The capability also returns learning to Coverage Engineering and Weave. An expert may identify a missing behavior combination, contribute a new case, or explain that existing material is insufficient. Those outputs should update the owning coverage or case artifacts instead of being flattened into a general review note.
Worked example
Exception handling
An agent prepares three cases where a procurement rule might allow an exception, attaches the controlling policy material, and asks a specialist to distinguish approved exceptions from escalation-only situations. The specialist completes a case review and confirms a checkpoint. The accepted contribution materializes a scoped policy and binary rubric, while one unresolved source conflict becomes a coverage observation for follow-up.
Related workflows
Related reference pages
Source confidence
Doctrine-backed: this page defines the public capability. The linked product pages are code-backed and define the active contribution, governance, and agent-configuration surfaces.