# Expert Contributions Generated: 2026-09-13T04:36:39.544Z Source build: local Canonical docs: https://teammately.ai/docs --- id: expert-contributions.overview title: Expert Contributions summary: Coordinate benchmark-scoped expert work, attributable judgment, governed artifacts, and the decisions that move correctness forward. kind: concept product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions --- # Expert Contributions Expert Contributions is the benchmark-scoped workspace for requesting, conducting, and materializing specialist work. It coordinates the expert, objective, selected evidence, task sequence, checkpoints, attributable responses, and contributed artifacts needed to move a benchmark forward. ## Definition The administrator workspace contains **Overview**, **Contributions**, **Contributed Artifacts**, and **Logs & Status**. **Request Contribution** opens the composer for a new contribution. The expert follows a contribution-specific experience that can contain form, chat, interview, and case-review tasks, along with checkpoints and completion states. A Contribution is the unit of requested expert effort. It replaces broad workflow configuration with a bounded statement of what this benchmark needs from this expert now. The work can result in contributed policies, rubrics, cases, or coverage observations without flattening all expert activity into one generic approval record. ## Decision checkpoint | Need | Contribution element | Result to inspect | | --- | --- | --- | | Resolve a specific benchmark question | Contribution statement and scoped objectives | The expert can explain the requested decision | | Ground work in concrete behavior | Selected or designated cases | Case-level responses remain attributable | | Supply supporting knowledge | Attachments and scoped statements | The expert sees the relevant source boundary | | Choose the right interaction | Form, chat, interview, or case review task | Task output matches the kind of judgment needed | | Confirm consequential learning | Checkpoint | Accepted, revised, or unresolved state is explicit | | Reuse the result | Contributed Artifacts | Policies, rubrics, cases, and coverage observations retain provenance | ## Lifecycle and status The durable Contribution statuses are `PREPARING_DIRECTION`, `AWAITING_DIRECTION_ALIGNMENT`, `MATERIALIZING_TASKS`, `READY`, `IN_PROGRESS`, `COMPLETED`, and `CANCELLED`. The interface presents these as planning direction, waiting for alignment, preparing tasks, ready, active, completed, or cancelled. The exact task sequence can vary by Contribution. Realtime updates and durable transitions help the administrator and expert see current progress without inventing completion. A waiting state, checkpoint, or finalization step should be shown as such. Completing the expert experience does not imply that every proposed artifact has been accepted into its project-level owner. ## Contribution evidence Logs & Status exposes operational and engagement records. Contributed Artifacts organizes materialized or contributed cases, policies, rubrics, and new coverage observations. Correctness Governance, Assets, or Coverage Management owns the resulting project or benchmark artifact after materialization. This model improves return on expert effort. Agents prepare focused work from project context, indexed material, benchmark cases, and unresolved questions. The expert supplies the authority; the result can be reused across standards, coverage, evaluation, and improvement. {% example-demo title="Resolve source authority" %} A benchmark contains cases where an operational runbook conflicts with a newer policy page. The operator requests a Contribution from the policy owner, selects the conflicting cases, attaches both sources, and uses case review plus a checkpoint. The expert establishes which source controls, contributes a scoped policy and rubric, and records one coverage observation for an unrepresented exception. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Contribution lifecycle and status](/docs/expert-contributions/lifecycle-and-status) - [Logs & Status](/docs/expert-contributions/logs-and-status) - [Agent Setup](/docs/agent-setup) - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active benchmark workspace, Contribution dashboard, composer, administrator detail, and expert routes support the scope, task, status, and artifact model described here. --- id: expert-contributions.complete title: Complete an Expert Contribution summary: Work through form, chat, interview, case-review, and checkpoint tasks while keeping specialist judgment attributable. kind: task product_area: expert_contributions status: stable updated: 2026-08-22 canonical: /docs/expert-contributions/complete-contribution --- # Complete an Expert Contribution Complete a Contribution by following its prepared task sequence and making the requested specialist judgments from the evidence shown. The expert experience can adapt between structured forms, agent chat, interviews, case review, and checkpoints. ## Prerequisites - A valid Contribution link or authenticated expert entry point. - Access to the Contribution and its assigned tasks. - Enough source and case context to explain each answer. - A stable connection when the task uses realtime agent interaction. ## Steps 1. Open the Contribution and read its objective, selected cases, and expected components before answering. 2. Complete each task according to its type. Planned activities can be Case Review, Form, Chat, or Interview; Curation, Comparative, and Trajectory components shape the prepared work those activities present. 3. Use attachments and visible case materials as the evidence boundary. State uncertainty when the supplied material does not resolve the question. 4. At a checkpoint, inspect the proposed summary or artifact meaning. Checkpoints prepare and reconcile requirements, consolidator or Policy statements, interview requests or records, and Rubrics. Confirm only what matches your judgment; retry, revise, or leave unresolved anything that does not. 5. Continue through the task handoff until the Contribution reaches its final step. 6. Review the completion state. If the experience shows a waiting, retry, or synchronization state, do not assume the administrator has received final evidence until the product confirms it. ## Object and state changes Answers create durable task responses and can advance task sessions, checkpoints, handoffs, and Contribution status. Chat or interview activity can produce transcripts and structured learning. Case review can attach judgment to selected cases. Completion makes the contribution available for reconciliation and materialization but does not itself make every proposed artifact governed. Review tasks can be `PREPARING`, `BLOCKED`, `READY`, `IN_PROGRESS`, `COMPLETED`, `SKIPPED`, or `SUPERSEDED`. The expert runtime can be `PREPARING`, `READY`, `ACTIVE`, `FINAL_CHECKPOINT`, `COMPLETED`, or `EXHAUSTED`. Checkpoints can be preparing, ready, or reconciled, with individual requirements pending, retryable, materialized, empty, or failed. These layered states explain why a Contribution can be active while one task is blocked or a final checkpoint is still pending. ## Success criteria - Every answer addresses the Contribution objective and cites the visible evidence where needed. - Case-level judgments remain connected to the relevant case. - Checkpoints distinguish accepted, revised, and unresolved meaning. - The final state is visibly complete rather than inferred from navigation. - Uncertainty or source conflict remains explicit for the administrator. ## Common failure modes - Answering from private background without identifying that the supplied evidence is incomplete. - Treating an agent summary as accurate without checking the checkpoint. - Leaving a form or chat task in a local unsynchronized state. - Continuing after a stale task handoff instead of following the current Contribution route. - Assuming that completion directly changes policies, rubrics, cases, or coverage. {% example-demo title="Example: checkpoint correction" %} An interview summary says that every expired agreement should be ignored. The expert corrects the checkpoint: expired agreements may still be relevant when the current agreement explicitly incorporates them. The corrected statement remains attributable and prevents an overbroad policy from being materialized. {% /example-demo %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Expert Contributions](/docs/expert-contributions) - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Expert Contribution problems](/docs/troubleshooting/expert-contributions) - [Permissions](/docs/troubleshooting/permissions) - [Authentication](/docs/troubleshooting/authentication) {% /related-card-grid %} ## Source confidence Code-backed: the current expert experience supports form, chat, interview, case-review, checkpoint, completion, waiting, and task-handoff routes with durable command and reconciliation behavior. --- id: expert-contributions.artifacts title: Contributed Artifacts summary: Inspect policies, rubrics, cases, and coverage observations produced through attributable expert contribution work. kind: reference product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions/contributed-artifacts --- # Contributed Artifacts ## Definition Contributed Artifacts is the benchmark workspace for inspecting durable material produced through Expert Contributions. It organizes contributed **Policies**, **Rubrics**, **Cases**, and **new coverage observations** while preserving their relationship to the Contribution and expert work that produced them. The view is a provenance and reconciliation surface. The final owner of a materialized artifact remains Correctness Governance, Assets, or Coverage Management according to artifact type. ## Fields, states, or lifecycle rules - Policy contributions represent expert-grounded behavior rules or revisions. - Rubric contributions represent proposed or accepted evaluation criteria tied to specialist judgment. - Case contributions represent situations supplied or corrected through expert work. - Coverage observations identify missing, thin, conflicting, or newly important benchmark behavior. - A coverage observation preserves its source, proposed facet applications, and application status so an operator can distinguish a recorded observation from one incorporated into coverage structure. - Each artifact should remain traceable to the Contribution, expert, selected evidence, task responses, and checkpoints that support it. - Contribution completion and artifact governance are separate transitions. Inspect the artifact's owning surface before treating it as active policy, active rubric, benchmark dataset membership, or resolved coverage. - Reconciliation can accept, revise, route, or leave material unresolved according to the active workflow. ## Interpreting contributed material Use the artifact type to choose the next surface. A contributed policy or rubric belongs in Correctness Governance. A contributed case belongs in the project Assets pool before benchmark selection. A coverage observation belongs in Coverage Management and may motivate a Coverage Story, case construction, or another focused Contribution. Preserve disagreements. Two experts can contribute conflicting Policy interpretations, and the artifact view should help an operator trace each interpretation rather than merge them into an invented consensus. Materialization should keep the Contribution, activity, checkpoint, expert, and scoped evidence links needed to explain why the artifact exists. {% example-demo title="Example: contribution provenance" %} An expert contributes a policy limiting compatibility claims, a rubric for explicit uncertainty, and a new case involving an unsupported adapter. The policy and rubric move to Correctness Governance for their lifecycle. The case enters Assets and is later selected into a benchmark dataset snapshot. All three retain the Contribution as their provenance. {% /example-demo %} ## Source confidence Code-backed: the active Contributed Artifacts workspace exposes policy, rubric, case, and new-coverage groupings. This page preserves the separation between contribution provenance and the lifecycle of each owning artifact. ## Related task pages {% related-card-grid title="Related task pages" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) {% /related-card-grid %} --- id: expert-contributions.lifecycle title: Contribution Lifecycle and Status summary: Interpret Contribution, activity, task, expert-runtime, and checkpoint states without inferring completion. kind: reference product_area: expert_contributions status: stable updated: 2026-08-22 canonical: /docs/expert-contributions/lifecycle-and-status --- # Contribution Lifecycle and Status ## Definition Contribution status, task status, expert-runtime state, and checkpoint state describe different layers. Read them together before diagnosing delay or declaring completion. ## Fields, states, or lifecycle rules ### Contribution status | Status | Meaning | | --- | --- | | `PREPARING_DIRECTION` | Teammately is planning the direction and requested work | | `AWAITING_DIRECTION_ALIGNMENT` | An administrator decision or alignment is required | | `MATERIALIZING_TASKS` | The confirmed plan is becoming executable expert activities | | `READY` | Work is prepared and available to the expert | | `IN_PROGRESS` | At least part of the expert work is active | | `COMPLETED` | Required expert work and finalization reached the durable terminal state | | `CANCELLED` | Further work was stopped while preserving the record | Planned activities can be Case Review, Form, Chat, or Interview. An individual review task can be preparing, blocked, ready, in progress, completed, skipped, or superseded. A superseded task remains provenance for what changed; it should not be counted as current outstanding work. ### Runtime and checkpoints The expert runtime moves through preparing, ready, active, final checkpoint, completed, or exhausted. `EXHAUSTED` means the current executable path has no further work; it is not a synonym for a successfully completed Contribution. A checkpoint moves from preparing to ready to reconciled. Its requirements may be pending, retryable, materialized, terminal-empty, or failed. Checkpoints can consolidate requirements, capture Policy meaning, prepare or record an Interview, or materialize Rubric meaning. A ready checkpoint still needs the expert's reconciliation. ### Diagnosing state If the Contribution is `IN_PROGRESS` but the expert sees no action, inspect the current task, runtime, and checkpoint. A blocked task may be waiting for evidence; a final checkpoint may need confirmation; an exhausted runtime may require administrator follow-up. Use Logs & Status for durable timing and engagement evidence rather than relying on a stale open browser view. > Completion boundary > > `COMPLETED` makes contributed evidence available for downstream governance. It does not automatically approve a Policy or Rubric, select a Case into a benchmark, or apply a coverage observation. ## Related task pages {% related-card-grid title="Related task pages" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) {% /related-card-grid %} ## Source confidence Code-backed: administrator and expert runtime sources define the layered Contribution, task, runtime, checkpoint, and requirement states described here. --- id: expert-contributions.logs-status title: Logs & Status summary: Inspect contribution reviews, sessions, interviews, engagement, and operational records within the benchmark scope. kind: reference product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions/logs-and-status --- # Logs & Status ## Definition Logs & Status is the benchmark-scoped operational record for Expert Contributions. It exposes reviews, review sessions and plans, interviews, expert or contribution engagement, and related activity tables. Use it to answer when work happened, which record is active, and where a stalled or disputed result came from. ## Fields, states, or lifecycle rules - Every table remains scoped to the selected benchmark. - Review, session, interview, engagement, and artifact records keep separate identities and statuses. - Filters, sorting, and detail views change presentation without rewriting the underlying Contribution. - Available transcript and response detail follows the user's project permissions. ## Choose the right record - **Reviews** shows review work and its Case or artifact relationship. - **Review Sessions** and plans show the bounded sequence that presented work to an expert. - **Interviews** exposes interview identity, targets, state, and available transcript detail. - **Expert Engagement** and **Contribution Engagement** show participation and operational progress. - Specialized tables can expose new-Case, output, draft-Policy, Case-specific Rubric, or review-question activity. ![Expert Contribution progress summary showing unassigned, in-review, aligned, pending, and reviewed counts by assignee.](/docs-assets/assets/screenshots/expert-contribution-progress-table.png) Use progress counts to locate work operationally; they do not establish whether contributed judgment was accepted. ![Expert Contribution review-session table showing assignee planning state, completed sessions, result counts, task counts, and reviewers.](/docs-assets/assets/screenshots/expert-contribution-sessions-table.png) Open a session when you need its task and result provenance rather than inferring it from aggregate progress. Filter and sort within the benchmark scope before comparing counts. A session, interview, review, and contributed artifact are separate objects and may have different completion times. Do not add their row counts together as a measure of unique Contributions or experts. ## Provenance and diagnosis Start from the Contribution when explaining intent, then follow its planned activity to the session, review, interview, checkpoint, and materialized artifact. Transcript or response evidence may explain a decision, but the owning Policy, Rubric, Case, or coverage surface determines the artifact's current governance status. When work appears stuck, compare durable status with the latest task or checkpoint rather than inferring from engagement alone. An expert can open a Contribution without completing a task, and a completed interview can still be waiting for checkpoint reconciliation or materialization. > Operational evidence, not correctness proof > > Logs demonstrate activity and provenance. They do not prove that contributed content was accepted, that experts agreed, or that a candidate system passed an evaluation. ## Related task pages {% related-card-grid title="Related task pages" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) {% /related-card-grid %} ## Source confidence Code-backed: the active Logs & Status routes define benchmark-scoped review, session, interview, and engagement tables and their detail surfaces. --- id: expert-contributions.request title: Request an Expert Contribution summary: Create a focused benchmark contribution with an accountable expert, clear objectives, selected cases, attachments, and appropriate task components. kind: task product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions/request-contribution --- # Request an Expert Contribution Request a Contribution when a benchmark needs a bounded piece of specialist judgment. The request should make the expert's decision clear, prepare the relevant evidence, and choose only the task components needed to obtain an attributable answer. ## Prerequisites - A selected project and benchmark. - An expert eligible for the contribution domain. - A concrete contribution statement or unresolved correctness question. - Selected cases, attachments, or scoped statements when the question depends on them. - Project Context and Reference Materials prepared in Agent Setup; use **Assets → Review Screens** when the Contribution needs reusable expert-facing presentation. ## Steps 1. Open the benchmark and select **Expert Contributions → Contributions**. 2. Choose **Request Contribution**. 3. Complete **Objectives & Missions**. State the decision or knowledge the benchmark needs and select the application domain: Coverage Model, Benchmark Setup, or Evaluation Validation. 4. Complete **Choose Experts** and confirm that each selected expert has the right authority for the mission. 5. Complete **Contribution Components**. Available components are Curation, Comparative, Trajectory, Form, Chat, and Interview. Choose conservative, balanced, or exploratory agent behavior; Comparative accepts two to five candidates and can allow improvement. 6. Designate the relevant Cases. Select exact Case IDs and decide whether the contribution may add Cases beyond that set. 7. Add attachments and scoped statements only when they help resolve the mission. Supported attachment scopes include completed Contributions, Policies, Rubrics, Dimensions, ontology values, Project Topics or Groups, Construction Patterns, and Case candidates. 8. Review the captured attachment snapshot version and hash, generated activities, and checkpoints. Confirm that controlling evidence is frozen and consequential meaning will be reconciled. 9. Send the request and follow its state through Overview, Contributions, or Logs & Status. ## Object and state changes This task creates a benchmark-scoped Contribution, associates experts, and records missions, application domain, Case designation, attachments, scoped statements, component behavior, and improvement permission. Planning materializes activities such as Case Review, Form, Chat, and Interview. Sending or starting work moves the Contribution toward `READY` or `IN_PROGRESS`; cancellation preserves the record. ## Success criteria - The Contribution asks one coherent specialist question. - The selected expert and application domain are appropriate. - Every case or attachment is relevant to the objective. - The chosen task types match the judgment required. - Checkpoints protect decisions that should not be silently inferred. - Attachment identities, scope statements, snapshot version, and hash are visible. - The administrator can tell what artifacts may result and where they will be governed. ## Common failure modes - Asking for general review without a materializable objective. - Selecting many cases that do not illuminate the same decision. - Leaving one-time behavior directions outside the Contribution objective, components, or scoped statements. - Omitting the source or case material needed to explain a judgment. - Assuming that task completion automatically approves contributed policies or rubrics. {% example-demo title="Example: focused coverage contribution" %} The objective asks an expert to decide whether source-freshness and customer-impact should form a distinct coverage slice. The operator selects six cases spanning those facets, attaches the controlling policy, and chooses case review plus a final checkpoint. The request can yield a coverage observation and a scoped rubric without asking the expert to redesign the entire benchmark. {% /example-demo %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Expert Contributions](/docs/expert-contributions) - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Review Screen](/docs/assets/review-screens) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Expert Contribution problems](/docs/troubleshooting/expert-contributions) - [Permissions](/docs/troubleshooting/permissions) - [Low expert agreement](/docs/troubleshooting/low-expert-agreement) {% /related-card-grid %} ## Source confidence Code-backed: the active Contribution composer defines expert selection, objectives, cases, attachments, statements, settings, and contribution components. Exact available components can depend on project and benchmark context. --- id: concepts.correctness-elicitation title: Correctness Elicitation summary: Turn tacit specialist judgment into attributable contributions, governed policies, applicability conditions, and binary rubrics. kind: concept product_area: correctness_elicitation status: stable updated: 2026-09-07 canonical: /docs/concepts/correctness-elicitation --- # Correctness Elicitation Correctness Elicitation is the capability for turning specialist judgment into explicit, reusable correctness specifications. It handles the parts of AI behavior that cannot be settled by a generic score: domain preferences, exceptions, conflicts between sources, applicability boundaries, unacceptable failure modes, and the evidence an expert needs before making a decision. ## Definition Elicitation begins before the expert opens a task. Teammately agents can organize relevant Reference Materials, cases, candidate responses, possible policies, rubric questions, and unresolved conflicts into a focused Contribution. The expert then works through forms, chat, interviews, case review, or checkpoints according to the requested components. The contribution remains attributable. Accepted learning can be materialized into policies, rubrics, cases, or coverage observations, while drafts and unresolved statements keep their own state. Correctness Governance owns the resulting policies and rubrics; Expert Contributions owns the benchmark-scoped human work that produced them. ## Decision checkpoint | Need | Elicitation method | Durable destination | | --- | --- | --- | | Confirm a known rule across several cases | Focused form or case review | Policy, applicability, or rubric contribution | | Discover reasoning that is hard to pre-structure | Chat or interview | Attributable transcript, checkpoints, and contributed artifacts | | Resolve disagreement or ambiguity | Targeted comparison and checkpoint | Explicit unresolved or accepted statement | | Identify missing benchmark behavior | Cases plus coverage questions | Contributed case or coverage observation | | Configure reviewer presentation | Assets → Review Screens | Reusable Review Screen | | Set a contribution's agent behavior | Expert Contribution | Contribution behavior and selected components | ## Why preparation matters An expert should not have to reconstruct the project, search for the controlling source, or infer why a case was selected. Project Context gives the agent the stable brief. Reference Materials provide indexed project knowledge. The Contribution selects the benchmark evidence, states the objective, and configures agent behavior. Review Screen controls presentation. This separation preserves authority. Reference material can inform an answer without becoming a policy. Agent-authored draft wording can focus the expert without becoming approved. A completed expert task can contribute evidence without automatically changing every project artifact. ## Relationship to the product Correctness Elicitation is broader than any single screen. Use **Expert Contributions** to request and track benchmark-scoped expert work. Use **Correctness Governance** to inspect and maintain policies and rubrics after they are materialized. Use **Agent Setup** to configure what agents understand, **Assets → Review Screens** for reusable expert-facing presentation, and the Contribution itself for scoped behavior and components. The capability also returns learning to Coverage Engineering and Weave. An expert may identify a missing behavior combination, contribute a new case, or explain that existing material is insufficient. Those outputs should update the owning coverage or case artifacts instead of being flattened into a general review note. {% example-demo title="Exception handling" %} An agent prepares three cases where a procurement rule might allow an exception, attaches the controlling policy material, and asks a specialist to distinguish approved exceptions from escalation-only situations. The specialist completes a case review and confirms a checkpoint. The accepted contribution materializes a scoped policy and binary rubric, while one unresolved source conflict becomes a coverage observation for follow-up. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Manage policies and rubrics](/docs/correctness-governance) - [Configure Agent Setup](/docs/agent-setup) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) - [Policies](/docs/object-model/policies) - [Rubrics](/docs/object-model/rubrics) {% /related-card-grid %} ## Source confidence Doctrine-backed: this page defines the public capability. The linked product pages are code-backed and define the active contribution, governance, and agent-configuration surfaces.