Teammately Docs
Docs menu

concept

Coverage Management

Manage benchmark coverage from setup through representation, Coverage Stories, case review, Case Foundry, and contribution requests.

Coverage Management

Coverage Management is the benchmark-scoped workspace for deciding whether the current dataset represents the behavior space the benchmark is meant to test. It connects project Coverage Facets to dataset representation, coverage guidance, Coverage Stories, case preparation, and expert contribution requests.

Definition

Project Coverage Facets define reusable Dimensions, Project Topics, and Case Construction Patterns. Coverage Management applies those foundations to one benchmark. Get Started establishes benchmark coverage guidance. The overview shows representation and operational status. Coverage Stories organize meaningful slices or gaps. Case Review inspects prepared cases and materials. Case Foundry coordinates case construction work.

The goal is to make missing or thin behavior explicit before evaluation evidence is trusted. Coverage Management does not replace Benchmark Datasets; it explains and improves the representation of the selected data.

Decision checkpoint

ObservationUseNext durable result
Benchmark purpose or guidance is missingGet StartedSaved coverage setup and readiness
A facet tuple is thin or absentRepresentation and Coverage StoriesNamed coverage need and intended evidence
More cases are neededCase FoundryBounded construction work tied to the gap
Generated cases may be unclearCase ReviewReviewed case and material quality
Specialist judgment is requiredContribution request from coverage contextBenchmark-scoped Expert Contribution
Coverage changed materiallyBenchmark DatasetsUpdated selection and snapshot boundary

Coverage Stories and case work

A Coverage Story gives a gap or behavior slice an operational narrative: why it matters, which facet combinations define it, what evidence exists, and what sourcing work remains. It should be concrete enough to guide case construction and expert attention.

Case Foundry can prepare case work from that structure. Case Review checks the resulting inputs and generated materials before they enter trusted dataset evidence. AI assistance can accelerate preparation, but selection and benchmark interpretation remain explicit human and product-state decisions.

Coverage and correctness

Coverage gaps sometimes reveal missing correctness rather than missing cases. If experts cannot say how a represented situation should be judged, request an Expert Contribution and update policies or rubrics. If the standard is clear but no case exercises it, use Weave and Case Foundry. If cases exist but are not selected or snapshotted, use Benchmark Datasets.

This routing prevents Comparison Directions, Contribution-scoped agent behavior, and coverage structure from being mixed together. Comparison Directions guide response variation; Coverage Facets and Coverage Stories describe the benchmark behavior space.

Worked example

Conflicting-source story

Representation shows that the benchmark covers current-source questions but almost never combines them with a plausible superseded document. A Coverage Story names the conflict pattern, relevant source-freshness and impact facets, and the desired case count. Case Foundry prepares candidates, Case Review rejects unrealistic material, and the accepted cases enter a new dataset snapshot.

Related workflows

Task / Coverage ManagementSet Up Benchmark CoverageDefine benchmark intent, facet handling, artifact preferences, and evidence requirements before generating coverage work.Task / Coverage ManagementCoverage StoriesOrganize benchmark coverage intent into governed Stories and testable facet tuples.Task / Coverage ManagementCase FoundryGenerate or update bounded case candidates from the saved coverage setup and Story map.Task / Coverage ManagementCase ReviewPrepare, inspect, refine, and admit Case candidates and generated materials into the benchmark dataset.Task / Coverage EngineeringPlan Benchmark CoverageApply project Coverage Facets to one benchmark, inspect representation, and turn important gaps into concrete case or contribution work.Concept / Benchmark DatasetsBenchmark DatasetsSelect benchmark Cases, inspect representation, and freeze immutable Snapshots for reproducible evidence.Task / Expert ContributionsRequest an Expert ContributionCreate a focused benchmark contribution with an accountable expert, clear objectives, selected cases, attachments, and appropriate task components.

Related reference pages

Concept / Coverage EngineeringCoverage EngineeringDesign the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work.Reference / AssetsCasesUnderstand canonical project cases, their input and materials, and how they become members of benchmark datasets.Reference / AssetsComparison DirectionsCreate reusable guidance for meaningful candidate-output differences in comparative expert work.

Source confidence

Code-backed: the active benchmark coverage routes expose setup, overview, Coverage Stories, Case Review, Case Foundry integration, realtime state, and contribution-request entry points.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Coverage Management Set Up Benchmark Coverage Define benchmark intent, facet handling, artifact preferences, and evidence requirements before generating coverage work. Coverage Management Coverage Stories Organize benchmark coverage intent into governed Stories and testable facet tuples. Coverage Management Case Foundry Generate or update bounded case candidates from the saved coverage setup and Story map. Coverage Management Case Review Prepare, inspect, refine, and admit Case candidates and generated materials into the benchmark dataset. Coverage Engineering Coverage Engineering Design the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work. Benchmark Datasets Benchmark Datasets Select benchmark Cases, inspect representation, and freeze immutable Snapshots for reproducible evidence.
AI context