# Coverage Management Generated: 2026-09-13T04:33:49.056Z Source build: local Canonical docs: https://teammately.ai/docs --- id: coverage-management.overview title: Coverage Management summary: Manage benchmark coverage from setup through representation, Coverage Stories, case review, Case Foundry, and contribution requests. kind: concept product_area: coverage_management status: stable updated: 2026-09-07 canonical: /docs/coverage-management --- # Coverage Management Coverage Management is the benchmark-scoped workspace for deciding whether the current dataset represents the behavior space the benchmark is meant to test. It connects project Coverage Facets to dataset representation, coverage guidance, Coverage Stories, case preparation, and expert contribution requests. ## Definition Project Coverage Facets define reusable Dimensions, Project Topics, and Case Construction Patterns. Coverage Management applies those foundations to one benchmark. **Get Started** establishes benchmark coverage guidance. The overview shows representation and operational status. Coverage Stories organize meaningful slices or gaps. Case Review inspects prepared cases and materials. Case Foundry coordinates case construction work. The goal is to make missing or thin behavior explicit before evaluation evidence is trusted. Coverage Management does not replace Benchmark Datasets; it explains and improves the representation of the selected data. ## Decision checkpoint | Observation | Use | Next durable result | | --- | --- | --- | | Benchmark purpose or guidance is missing | Get Started | Saved coverage setup and readiness | | A facet tuple is thin or absent | Representation and Coverage Stories | Named coverage need and intended evidence | | More cases are needed | Case Foundry | Bounded construction work tied to the gap | | Generated cases may be unclear | Case Review | Reviewed case and material quality | | Specialist judgment is required | Contribution request from coverage context | Benchmark-scoped Expert Contribution | | Coverage changed materially | Benchmark Datasets | Updated selection and snapshot boundary | ## Coverage Stories and case work A Coverage Story gives a gap or behavior slice an operational narrative: why it matters, which facet combinations define it, what evidence exists, and what sourcing work remains. It should be concrete enough to guide case construction and expert attention. Case Foundry can prepare case work from that structure. Case Review checks the resulting inputs and generated materials before they enter trusted dataset evidence. AI assistance can accelerate preparation, but selection and benchmark interpretation remain explicit human and product-state decisions. ## Coverage and correctness Coverage gaps sometimes reveal missing correctness rather than missing cases. If experts cannot say how a represented situation should be judged, request an Expert Contribution and update policies or rubrics. If the standard is clear but no case exercises it, use Weave and Case Foundry. If cases exist but are not selected or snapshotted, use Benchmark Datasets. This routing prevents Comparison Directions, Contribution-scoped agent behavior, and coverage structure from being mixed together. Comparison Directions guide response variation; Coverage Facets and Coverage Stories describe the benchmark behavior space. {% example-demo title="Conflicting-source story" %} Representation shows that the benchmark covers current-source questions but almost never combines them with a plausible superseded document. A Coverage Story names the conflict pattern, relevant source-freshness and impact facets, and the desired case count. Case Foundry prepares candidates, Case Review rejects unrealistic material, and the accepted cases enter a new dataset snapshot. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Set up benchmark coverage](/docs/coverage-management/get-started) - [Work with Coverage Stories](/docs/coverage-management/coverage-stories) - [Run Case Foundry](/docs/coverage-management/case-foundry) - [Review prepared Cases](/docs/coverage-management/case-review) - [Plan benchmark coverage](/docs/coverage-engineering/plan-benchmark-coverage) - [Work with Benchmark Datasets](/docs/benchmark-datasets) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coverage Engineering](/docs/coverage-engineering) - [Cases](/docs/assets/cases) - [Comparison Directions](/docs/assets/comparison-directions) {% /related-card-grid %} ## Source confidence Code-backed: the active benchmark coverage routes expose setup, overview, Coverage Stories, Case Review, Case Foundry integration, realtime state, and contribution-request entry points. --- id: coverage-management.case-foundry title: Case Foundry summary: Generate or update bounded case candidates from the saved coverage setup and Story map. kind: task product_area: coverage_management status: stable updated: 2026-08-22 canonical: /docs/coverage-management/case-foundry --- # Case Foundry ## Prerequisites - Ready coverage setup, active Stories, and actionable tuple targets. - A current input snapshot whose freshness can be verified. Case Foundry coordinates bounded case preparation from the saved coverage setup and Coverage Stories. Use **Generate** for the first run and **Update** after the governed inputs change. A blocked action means required setup, readiness, or upstream evidence is not yet available. ## Run states and freshness A Foundry run reports `QUEUED`, `RUNNING`, `COMPLETED`, `PARTIAL`, `FAILED`, or `CANCELLED`. Keep the run identity and input snapshot together when diagnosing it. Completion can report Cases added, synthesized, or retrieved; these counts explain construction activity, not acceptance into trusted benchmark evidence. The workspace distinguishes a fresh result from one whose input snapshot changed. If coverage setup, Story structure, or relevant dataset context changed after the run began, update the Foundry work. Do not use an old completion badge as proof that the current coverage contract has been handled. ## Before and after Foundry Before running, make the setup ready, activate the intended Stories, and check tuple targets. If a specialist must settle an ambiguous standard, request an Expert Contribution instead of generating around the ambiguity. After running, open Case Review. Prepared candidates start as reviewable material rather than automatically becoming durable benchmark Cases. Inspect their evidence fit, sources, facet traces, inputs, and generated artifacts. Remove weak or redundant candidates and resolve material failures before inclusion. > Case Foundry is not a Dataset Snapshot > > Foundry prepares candidates. Case Review admits suitable Cases into the current benchmark set, and Benchmark Datasets creates the immutable Snapshot used for evaluation. ## Failure handling For a partial or failed run, preserve successful bounded work, read the failure detail, and retry only the missing scope when the product offers that action. Cancellation stops the current operation; it does not roll back Cases already materialized by a completed portion. Recheck freshness after any retry. ## Object and state changes A Foundry action creates a run and can prepare, synthesize, retrieve, or add candidate Cases. Update creates new bounded work from changed input. The run does not create a Dataset Snapshot or bypass Case Review. ## Success criteria - Terminal status and input freshness are known. - Summary counts are interpreted as construction activity. - Prepared candidates move to review rather than automatic trust. ## Common failure modes - Treating `COMPLETED` as Case acceptance. - Retrying stale work without updating its input. - Generating around an unresolved correctness question. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coverage Management](/docs/coverage-management) - [Coverage Stories](/docs/coverage-management/coverage-stories) - [Benchmark Dataset Cases](/docs/benchmark-datasets/cases) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Unrealistic synthetic Cases](/docs/troubleshooting/unrealistic-synthetic-cases) - [Unclear Cases](/docs/troubleshooting/unclear-cases) {% /related-card-grid %} ## Source confidence Code-backed: the active Foundry API, run types, and workspace hook define actions, states, freshness, and summary counts. Case acceptance is verified in the separate Case Review surface. --- id: coverage-management.case-review title: Case Review summary: Prepare, inspect, refine, and admit Case candidates and generated materials into the benchmark dataset. kind: task product_area: coverage_management status: stable updated: 2026-08-22 canonical: /docs/coverage-management/case-review --- # Case Review ## Prerequisites - Prepared candidates tied to a frozen setup, Story map, input contract, and output formats. - Permission to admit or remove candidates from current benchmark membership. Case Review is the admission boundary between prepared coverage candidates and the current benchmark dataset. Preparation freezes the relevant setup, Stories, input contract, and output formats so a candidate can be judged against the instructions that produced it. Preparation can move through planning, searching, evaluating, generating, synthesizing, materializing, ready, partial, failed, and cancelled states. Read this state together with the frozen input rather than assuming every visible card is complete. ## Review candidates Candidates begin included for review. Remove a candidate when it is weak, redundant, misplaced, unsupported, or does not prove its tuple. Inspect its fit explanation, source and facet trace, input content, materials, and relationship to the tuple target. Inclusion should mean the candidate is suitable to enter the current benchmark set, not merely that generation succeeded. For one tuple, **Generate more** appends candidates using count and operator instructions. **Regenerate** can reuse or synthesize source evidence, or run in synthesize-only mode. These actions have different provenance implications; preserve the displayed source relationship when deciding which candidate to keep. ## Resolve generated materials Artifact expectations from coverage setup can be mandatory or optional. Material resolution reports generating, verifying, ready, failed, or skipped. A mandatory artifact failure blocks trustworthy inclusion. An optional artifact may be skipped when the Case remains coherent without it. Retry uses the original material specification. If the specification itself is wrong, correct the governed setup or candidate design rather than repeatedly retrying the same request. Verify that a ready file actually supports the Case and conforms to Project Input Schema. ## Complete the review Accepted included candidates materialize into the benchmark's current Case set. Then inspect Dataset Representation and create a new Snapshot only after evaluator readiness and Snapshot blockers are clear. Existing Snapshots remain unchanged. {% example-demo title="Example: rejecting decorative evidence" %} A tuple requires the candidate system to reconcile two contradictory tables. One prepared Case has a table that never affects the answer, while another requires comparing two columns and citing the newer record. The reviewer removes the decorative Case, verifies the second table, and admits only the candidate that proves the intended transformation. {% /example-demo %} ## Object and state changes Inclusion and removal change the review selection; accepted included candidates materialize into the current Case set. Generate-more and regenerate create new candidates. Material retries update resolution state without changing the original specification. ## Success criteria - Included Cases prove their tuples and have usable provenance. - Mandatory materials are ready and verified. - Dataset membership and the next Snapshot reflect only accepted work. ## Common failure modes - Keeping decorative or redundant candidates to meet a count. - Treating material-generation success as Case quality. - Assuming acceptance changed an existing Snapshot. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Project Input Schema](/docs/project-settings/input-schema) - [Benchmark Dataset Cases](/docs/benchmark-datasets/cases) - [Dataset Snapshots](/docs/benchmark-datasets/snapshots) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Unclear Cases](/docs/troubleshooting/unclear-cases) - [Unrealistic synthetic Cases](/docs/troubleshooting/unrealistic-synthetic-cases) {% /related-card-grid %} ## Source confidence Code-backed: the active Case Review page, preparation modal, tuple actions, and Generated Materials component define frozen inputs, run states, inclusion, generation modes, and artifact resolution. --- id: coverage-management.coverage-stories title: Coverage Stories summary: Organize benchmark coverage intent into governed Stories and testable facet tuples. kind: task product_area: coverage_management status: stable updated: 2026-08-22 canonical: /docs/coverage-management/coverage-stories --- # Coverage Stories ## Prerequisites - Ready coverage setup or a clearly justified manual Story. - Coverage Facets that can name the intended behavior slice. A Coverage Story turns benchmark intent into a coherent behavior slice. It explains what matters, why it matters, and which facet combinations must be exercised without pretending that a chart category alone describes a real product situation. Stories can be created manually or proposed by generation. Their lifecycle is `draft`, `active`, or `archived`, and their origin remains visible as AI-generated or manual. A Story contains a title, description, rationale, intent, budget, referenced facets, suggested Topic Groups, and one or more tuples. ## Make tuples testable Each tuple names a smaller test obligation through its title, `must_prove` statement, facet references, target, accepted count, status, and flags. Write `must_prove` so Case Review can decide whether a candidate actually supplies the required evidence. Avoid vague goals such as “good edge cases.” Name the actor, conflict, evidence, constraint, or transformation that distinguishes the tuple. Targets express desired evidence volume; accepted counts report materialized evidence. Neither number proves quality. A tuple can meet its count while still containing redundant or unrealistic Cases, so review remains mandatory. ## Edit and govern the Story map Use the editor to create or revise a Story and add, edit, or remove tuples. Archive a Story whose behavior is no longer in benchmark scope. Do not delete or rewrite the rationale merely because the current dataset already covers it; that rationale explains why the evidence exists. When generation is running, partial, failed, or based on changed setup, read the displayed generation state before acting. A generated Story remains a proposal until the saved story and tuple structure reflects the benchmark intent. {% example-demo title="Example: superseded-source conflict" %} A Story covers answers that cite a plausible but superseded policy. One tuple must prove that the candidate detects the date conflict; another must prove that it asks for clarification when the current source is incomplete. Their separate targets prevent several near-duplicate date cases from masquerading as coverage of both behaviors. {% /example-demo %} ## Object and state changes Creating or editing changes the saved Story and tuple map. Archiving removes a Story from active planning while preserving it. Generation can propose Stories but does not accept Cases into the dataset. ## Success criteria - Every active Story has a clear rationale and testable tuples. - Targets and accepted counts remain distinguishable. - Story origin and generation freshness are visible. ## Common failure modes - Writing tuples that cannot be judged in Case Review. - Treating target count as evidence quality. - Merging distinct Topic or facet obligations into vague coverage prose. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coverage Management](/docs/coverage-management) - [Project Topics](/docs/coverage-engineering/project-topics) - [Case Construction Patterns](/docs/coverage-engineering/case-construction-patterns) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Unbalanced coverage](/docs/troubleshooting/unbalanced-coverage) - [Stale Dimensions](/docs/troubleshooting/stale-dimensions) {% /related-card-grid %} ## Source confidence Code-backed: the current Coverage Stories page, editor, and types define Story origin, lifecycle, fields, tuple structure, counts, and generation presentation. --- id: coverage-management.get-started title: Set Up Benchmark Coverage summary: Define benchmark intent, facet handling, artifact preferences, and evidence requirements before generating coverage work. kind: task product_area: coverage_management status: stable updated: 2026-08-22 canonical: /docs/coverage-management/get-started --- # Set Up Benchmark Coverage ## Prerequisites - Project Coverage Facets and Input Schema are available. - The benchmark intent and evidence risk can be stated concretely. **Get Started** records the coverage contract that drives Coverage Stories and Case Foundry. Complete it before treating generated coverage work as aligned to the benchmark. ## Define the intent and facet treatment Describe the benchmark intent and concrete requirements. For each Dimension ontology value, choose the benchmark role required by the setup. Configure Topic Group handling and Case Construction Pattern behavior rather than assuming every active project facet must be represented equally. Dimension roles, Topic Group handling, and Pattern modes are different controls. A required Dimension value constrains represented behavior. A Topic Group can require every Topic, require group-level coverage, provide guidance, or be excluded. A Pattern can be left to the system, preferred, or avoided. Preserve those distinctions when explaining the resulting coverage plan. ## Define artifact and evidence expectations For image, document, tabular, presentation, source-text, and audio artifacts, choose **mandatory**, **optional**, or **never**. Set portfolio limits and accepted formats so construction does not create unsupported or gratuitous material. The evidence profile can specify actor, workflow, grounding, evidence carriers, difficulty, transformation, and data-handling expectations. These fields make a behavior testable. They are not decorative prose: Case Review uses them to judge whether a prepared candidate proves the intended situation. ## Save and check readiness Setup moves through `draft`, `ready`, `generated`, `changed_since_generation`, and `archived` states. Resolve the readiness guidance before generation. If the setup changes after stories or cases were generated, treat the previous work as based on an older input rather than silently presenting it as current. A coverage guideline can apply to `foundry_only` or `overall_coverage`. Overall coverage can require provenance backfill for existing Cases. The coverage compiler can preview reconciled revisions, but an operator confirms the durable update. > Generation boundary > > Saving setup does not create trusted Cases or a Dataset Snapshot. It defines the instructions and evidence profile for downstream story and case work. ## Object and state changes Saving creates or revises benchmark-scoped coverage setup and its readiness status. Generation records which setup revision it used. Archiving stops the setup from acting as the current contract without erasing history. ## Success criteria - Intent, requirements, facet treatment, artifacts, and evidence profile agree. - Readiness is explicit and downstream generation can identify the exact setup. - Overall-coverage provenance needs are handled deliberately. ## Common failure modes - Requiring every active facet without regard to benchmark intent. - Marking unsupported artifacts mandatory. - Editing setup after generation and ignoring the stale result. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coverage Management](/docs/coverage-management) - [Project Input Schema](/docs/project-settings/input-schema) - [Coverage Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Unbalanced coverage](/docs/troubleshooting/unbalanced-coverage) - [Unrealistic synthetic Cases](/docs/troubleshooting/unrealistic-synthetic-cases) {% /related-card-grid %} ## Source confidence Code-backed: the active setup page, types, readiness rules, and save states define the benchmark-scoped coverage contract and its lifecycle. --- id: coverage.overview title: Coverage Engineering summary: Design the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work. kind: concept product_area: coverage_engineering status: stable updated: 2026-09-07 canonical: /docs/coverage-engineering --- # Coverage Engineering Coverage Engineering is the capability for designing what a specialist AI benchmark must represent. It turns requirements, project knowledge, existing cases, and observed failures into a structured coverage map that guides dataset selection, case construction, expert contributions, and evaluation interpretation. ## Definition Coverage work has two scopes. At project scope, **Coverage Facets** manages reusable Dimensions, Project Topics, and Case Construction Patterns. At benchmark scope, **Coverage Management** applies those facets to setup, representation, Coverage Stories, Case Review, Case Foundry, and contribution requests. **Benchmark Datasets** owns the selected Cases and snapshots that embody the resulting coverage. Coverage Engineering is therefore broader than adding cases. It explains why a behavior slice matters, how it is represented, which combinations are thin, and what work should close the gap. ## Decision checkpoint | Question | Product surface | Durable result | | --- | --- | --- | | Which axes distinguish important behavior? | Coverage Facets → Dimensions | Reusable coverage axes | | Which domain subjects must be represented? | Coverage Facets → Project Topics | Project topic structure | | How should cases be constructed repeatedly? | Coverage Facets → Case Construction Patterns | Reusable construction guidance | | What should this benchmark cover? | Coverage Management → Get Started | Benchmark-specific coverage guidance | | Where is the selected dataset thin? | Representation and Coverage Stories | Named gaps and sourcing work | | Which exact cases define evidence? | Benchmark Datasets | Selection and snapshot boundary | ## Coverage map and benchmark evidence A coverage map should identify meaningful combinations rather than isolated tags. A source-freshness Dimension may be well populated overall while the combination of superseded source, high customer impact, and exception request remains absent. Coverage Stories make those combinations operational; Case Foundry and expert contribution requests can then target them. Representation is evidence about the dataset, not proof that the benchmark is complete. A large or balanced count can still omit a critical boundary. Product teams should use specialist judgment to decide which gaps materially affect trust. ## Relationship to correctness and construction Correctness Elicitation answers what should count as correct in a represented situation. Weave constructs or imports the cases and materials needed to exercise the situation. If the team cannot judge a coverage slice, request an Expert Contribution. If the judgment is clear but no case exists, use Case Foundry or other case-construction work. If cases exist but are not selected, update Benchmark Datasets. Comparison Directions are not Coverage Facets. They guide comparative response variation and belong to **Assets → Comparison Directions**. Keep benchmark representation in Dimensions, Topics, Patterns, Stories, and dataset snapshots. {% example-demo title="Authority-conflict coverage" %} A project creates source authority and customer impact Dimensions, a Project Topic for eligibility exceptions, and a pattern for pairing current and superseded documents. Coverage Management shows that the high-impact conflict tuple has no selected cases. A Coverage Story justifies the gap, Case Foundry prepares cases, and the accepted cases enter a new dataset snapshot. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Plan benchmark coverage](/docs/coverage-engineering/plan-benchmark-coverage) - [Manage Coverage](/docs/coverage-management) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coverage Dimensions](/docs/object-model/coverage-dimensions) - [Ontology](/docs/object-model/ontology) - [Cases](/docs/assets/cases) {% /related-card-grid %} ## Source confidence Code-backed: current navigation and benchmark coverage routes establish project Coverage Facets, benchmark Coverage Management, setup, representation, Stories, Case Review, and Case Foundry responsibilities. --- id: benchmark-datasets.overview title: Benchmark Datasets summary: Select benchmark Cases, inspect representation, and freeze immutable Snapshots for reproducible evidence. kind: concept product_area: benchmark_datasets status: stable updated: 2026-08-22 canonical: /docs/benchmark-datasets --- # Benchmark Datasets Benchmark Datasets defines the evidence set for one benchmark through **Cases**, **Representation**, and **Snapshots**. The current dataset is editable. It selects reusable project Cases and reflects current facet, policy, rubric, and contributor facts. A Snapshot freezes the exact dataset state needed by a Benchmark Version and its evaluations. These are deliberately different surfaces: editing the current set must not rewrite historical evidence. ## Decision checkpoint | Surface | Use it to | Evidence rule | | --- | --- | --- | | Cases | Inspect and change current benchmark membership | Selection is live until snapshotted | | Representation | Find concentration and absence across governed facets | Read distribution together with distinct Case counts | | Snapshots | Freeze Cases, evaluator links, and representation facts | Snapshot content is read-only | Coverage Management acts on gaps found in the dataset. Assets remains the project-level reusable pool. Benchmark Evaluations runs exact Harness Versions against an immutable Benchmark Version rather than an unspecified “current dataset.” ## Evidence flow Cases usually begin in project Assets or materialize through Case Review or Expert Contributions. Selecting them makes them part of the current benchmark dataset. Representation then summarizes the current assignments and evaluator relationships. Snapshot readiness checks whether that state can be frozen. A Snapshot supplies the immutable dataset facts used by a Benchmark Version. This flow is one-way for historical evidence. Later edits to an Asset, facet assignment, Policy, Rubric, or current membership may improve the next Snapshot, but they do not update a previous Snapshot. Compare candidates within one Benchmark Version unless the analysis explicitly accounts for a moved evidence boundary. ## Before creating evidence Check Case clarity and schema conformance, then inspect Representation for intended behavior and provenance. Confirm approved eligible evaluator links. Resolve Snapshot blockers and preserve the resulting label, version, content hash, creation time, and Case count. A Snapshot can be reproducible while still being incomplete as product coverage. Reproducibility answers which evidence was evaluated; Representation and Coverage Management answer whether that evidence supports the intended product claim. {% example-demo title="Example: editable set versus frozen evidence" %} The current dataset gains four Cases and a corrected Rubric link after an expert Contribution is reconciled. An earlier Run still points to its old Benchmark Version. The operator creates a new Snapshot and Version for the changed set rather than comparing the new candidate against the old Run as though only Harness behavior moved. {% /example-demo %} {% related-card-grid title="Dataset workflows" %} - [Manage benchmark Cases](/docs/benchmark-datasets/cases) - [Inspect Representation](/docs/benchmark-datasets/representation) - [Create and inspect Snapshots](/docs/benchmark-datasets/snapshots) - [Manage coverage](/docs/coverage-management) {% /related-card-grid %} ## Source confidence Code-backed: the active dataset routes establish the editable current set, representation workspace, and immutable Snapshot boundary. --- id: expert-contributions.overview title: Expert Contributions summary: Coordinate benchmark-scoped expert work, attributable judgment, governed artifacts, and the decisions that move correctness forward. kind: concept product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions --- # Expert Contributions Expert Contributions is the benchmark-scoped workspace for requesting, conducting, and materializing specialist work. It coordinates the expert, objective, selected evidence, task sequence, checkpoints, attributable responses, and contributed artifacts needed to move a benchmark forward. ## Definition The administrator workspace contains **Overview**, **Contributions**, **Contributed Artifacts**, and **Logs & Status**. **Request Contribution** opens the composer for a new contribution. The expert follows a contribution-specific experience that can contain form, chat, interview, and case-review tasks, along with checkpoints and completion states. A Contribution is the unit of requested expert effort. It replaces broad workflow configuration with a bounded statement of what this benchmark needs from this expert now. The work can result in contributed policies, rubrics, cases, or coverage observations without flattening all expert activity into one generic approval record. ## Decision checkpoint | Need | Contribution element | Result to inspect | | --- | --- | --- | | Resolve a specific benchmark question | Contribution statement and scoped objectives | The expert can explain the requested decision | | Ground work in concrete behavior | Selected or designated cases | Case-level responses remain attributable | | Supply supporting knowledge | Attachments and scoped statements | The expert sees the relevant source boundary | | Choose the right interaction | Form, chat, interview, or case review task | Task output matches the kind of judgment needed | | Confirm consequential learning | Checkpoint | Accepted, revised, or unresolved state is explicit | | Reuse the result | Contributed Artifacts | Policies, rubrics, cases, and coverage observations retain provenance | ## Lifecycle and status The durable Contribution statuses are `PREPARING_DIRECTION`, `AWAITING_DIRECTION_ALIGNMENT`, `MATERIALIZING_TASKS`, `READY`, `IN_PROGRESS`, `COMPLETED`, and `CANCELLED`. The interface presents these as planning direction, waiting for alignment, preparing tasks, ready, active, completed, or cancelled. The exact task sequence can vary by Contribution. Realtime updates and durable transitions help the administrator and expert see current progress without inventing completion. A waiting state, checkpoint, or finalization step should be shown as such. Completing the expert experience does not imply that every proposed artifact has been accepted into its project-level owner. ## Contribution evidence Logs & Status exposes operational and engagement records. Contributed Artifacts organizes materialized or contributed cases, policies, rubrics, and new coverage observations. Correctness Governance, Assets, or Coverage Management owns the resulting project or benchmark artifact after materialization. This model improves return on expert effort. Agents prepare focused work from project context, indexed material, benchmark cases, and unresolved questions. The expert supplies the authority; the result can be reused across standards, coverage, evaluation, and improvement. {% example-demo title="Resolve source authority" %} A benchmark contains cases where an operational runbook conflicts with a newer policy page. The operator requests a Contribution from the policy owner, selects the conflicting cases, attaches both sources, and uses case review plus a checkpoint. The expert establishes which source controls, contributes a scoped policy and rubric, and records one coverage observation for an unrepresented exception. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Contribution lifecycle and status](/docs/expert-contributions/lifecycle-and-status) - [Logs & Status](/docs/expert-contributions/logs-and-status) - [Agent Setup](/docs/agent-setup) - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active benchmark workspace, Contribution dashboard, composer, administrator detail, and expert routes support the scope, task, status, and artifact model described here.