Coverage Design
Coverage Design applies reusable project facets to one Benchmark. Use Canvas to plan and inspect input coverage, and Inbox to decide whether new findings should change the selected Cases.
Definition
A coverage requirement names an input situation and how it should be represented. It can require a minimum Case count or record an observation. Dimension ontology values, Topics and Scenarios identify the intended slice; distribution settings describe its share of the active benchmark population.
Canvas connects those requirements to actual selected Cases. A large total does not establish that an important combination is present, and unknown classification does not establish absence.
Decision checkpoint
| Need | Use | Result |
|---|---|---|
| Define the intended behavior space | Canvas requirements and settings | Explicit coverage obligations |
| Inspect a slice or combination | Canvas selection and matrix | Matching Cases, unknowns and remaining need |
| Prepare more Cases | Case preparation | Reviewable candidates and materials |
| Act on an expert or Fieldloop finding | Inbox | An explicit adoption decision |
| Freeze the selected population | Benchmark Datasets → Snapshots | An immutable Benchmark Version |
Work from Canvas
Describe the Benchmark intent, select the relevant facets and inspect count or distribution requirements. Examine Cases behind a count before deciding that a gap is resolved. Distinguish requirements from observations and current membership from available project Cases.
Stories and their tuples describe meaningful situations used in planning and Case preparation. Case Foundry coordinates preparation; Case Review is where prepared inputs and materials are inspected. These mechanisms support Canvas work rather than replacing its current requirement and membership view.
Materials and input shape
Project Input Schema owns allowed artifact families, accepted formats and required material keys. Coverage Design does not add another Materials configuration. A tuple can require an allowed artifact but cannot remove a required input or enable an unsupported format.
Act from Inbox
Inbox links attributable input-coverage findings to possible dataset changes. Review each observation or proposed Case separately. Generating a Case stages another item; it does not automatically add membership. A resolved report can contain declined or withdrawn items.
Existing Snapshots and Runs retain their recorded population. Create a new Snapshot when changed membership or standards should define a later evaluation.