# Scenarios
Generated: 2026-11-03T08:08:20.436Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: coverage.scenarios
title: Scenarios
summary: Define reusable mechanisms for constructing cases and steer how benchmarks use or avoid them.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-10-30
canonical: /docs/coverage-engineering/scenarios
---
# Scenarios
## Definition
A Scenario describes a reusable mechanism for building cases. It answers “how should this situation be constructed?” while Dimensions describe differentiating values and Topics describe subject matter.
Examples include conflicting authorities, missing prerequisite supporting information, ambiguous user intent, multi-step state change, or a plausible but superseded source. A good Scenario is portable across Topics rather than tied to one case's wording.
## Fields, states, or lifecycle rules
Each Scenario has a name, description, origin, usage counts, examples, and benchmark statistics. Origins currently distinguish manual, AI-generated, expert-input, and imported Scenarios. Usage can show Case Pool cases, benchmark cases, requirements, and benchmark steering.
Scenario suggestions can be grounded in Topics, source material, existing cases and Scenarios, the Project Agent Brief, Dimensions and ontology, or expert input. Generated candidates include the proposed definition, why they were suggested, and source references. Accept or reject each candidate explicitly.
## Benchmark steering
Canvas settings can use one of three modes:
- **System choose:** allow construction to select appropriate Scenarios from the project library.
- **Prefer:** encourage named Scenarios when building Coverage Stories and cases.
- **Avoid:** prevent named Scenarios from shaping this benchmark's intended case supply.
Coverage Story tuples can also reference specific Scenarios. The tuple reference is the concrete construction instruction for that part of the story; the project Scenario remains reusable and independently editable.
## Design rules
- Name the mechanism, not the domain example. “Conflicting authorities” travels better than “Conflicting HR policies.”
- Explain what makes a case instantiate the Scenario.
- Keep Topics out of the Scenario definition unless they are only examples.
- Do not encode an expected answer or rubric verdict as a scenario.
- Inspect examples and usage before deleting or materially changing a Scenario.
- Treat AI and expert suggestions as reviewable proposals. Origin is provenance, not approval.
> Scenario versus requirement
>
> A Scenario describes a reusable structure. Benchmark requirements describe constraints for one benchmark, and a Coverage Tuple combines the relevant Dimensions, Topic Groups, Scenarios, proof statement, and target case count.
{% example-demo title="Example: portable scenario" %}
The project defines **Plausible superseded authority**: construct a case where an older source appears credible but a newer source controls. Coverage Stories reuse the Scenario for procurement, support, and compliance Topics while applying different source-condition and impact ontology values.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Topics](/docs/coverage-engineering/project-topics)
- [Coverage Design Settings](/docs/coverage-design/settings)
- [Coverage Stories](/docs/coverage-design/coverage-stories)
- [Review Prepared Cases](/docs/coverage-design/case-review)
{% /related-card-grid %}
---
id: coverage.overview
title: Coverage Engineering
summary: Design the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work.
kind: concept
product_area: coverage_engineering
status: stable
updated: 2026-10-30
canonical: /docs/coverage-engineering
---
# Coverage Engineering
Coverage Engineering is the capability for designing what a specialist AI benchmark must represent. It turns requirements, project knowledge, existing cases, and observed failures into a structured coverage map that guides dataset selection, case construction, expert contributions, and evaluation interpretation.
## Definition
Coverage work has two scopes. At project scope, **Coverage Facets** manages reusable Dimensions, Topics, and Scenarios. At benchmark scope, **Coverage Design** applies those facets to setup, representation, Coverage Stories, Case Review, Case Foundry, and contribution requests. **Benchmark Datasets** owns the selected Cases and snapshots that embody the resulting coverage.
Coverage Engineering is therefore broader than adding cases. It explains why a behavior slice matters, how it is represented, which combinations are thin, and what work should close the gap.
## Decision checkpoint
| Question | Product surface | Durable result |
| --- | --- | --- |
| Which axes distinguish important behavior? | Coverage Facets → Dimensions | Reusable coverage axes |
| Which domain subjects must be represented? | Coverage Facets → Topics | Topic structure |
| How should cases be constructed repeatedly? | Coverage Facets → Scenarios | Reusable construction guidance |
| What should this benchmark cover? | Coverage Design → Canvas | Benchmark-specific coverage guidance |
| Where is the selected dataset thin? | Representation and Coverage Stories | Named gaps and sourcing work |
| Which exact cases define supporting information? | Benchmark Datasets | Selection and snapshot boundary |
## Coverage map and benchmark results
A coverage map should identify meaningful combinations rather than isolated tags. A source-freshness Dimension may be well populated overall while the combination of superseded source, high customer impact, and exception request remains absent. Coverage Stories make those combinations operational; Case Foundry and expert contribution requests can then target them.
Representation is supporting information about the dataset, not proof that the benchmark is complete. A large or balanced count can still omit a critical boundary. Product teams should use specialist judgment to decide which gaps materially affect trust.
## Relationship to correctness and construction
Correctness Elicitation answers what should count as correct in a represented situation. Weave constructs or imports the cases and materials needed to exercise the situation. If the team cannot judge a coverage slice, request an Expert Contribution. If the judgment is clear but no case exists, use Case Foundry or other case-construction work. If cases exist but are not selected, update Benchmark Datasets.
Comparison Directions are not Coverage Facets. They guide review-agent response variation and belong to Agent Setup. Keep benchmark representation in Dimensions, Topics, Scenarios, Stories, and dataset snapshots.
{% example-demo title="Authority-conflict coverage" %}
A project creates source authority and customer impact Dimensions, a Topic for eligibility exceptions, and a pattern for pairing current and superseded documents. Coverage Design shows that the high-impact conflict tuple has no selected cases. A Coverage Story justifies the gap, Case Foundry prepares cases, and the accepted cases enter a new dataset snapshot.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Plan Benchmark Coverage](/docs/coverage-engineering/plan-benchmark-coverage)
- [Coverage Design](/docs/coverage-design)
- [Benchmark Datasets](/docs/benchmark-datasets)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage dimensions](/docs/object-model/coverage-dimensions)
- [Ontology](/docs/object-model/ontology)
- [Cases](/docs/assets/cases)
{% /related-card-grid %}
---
id: coverage.dimensions-ontology
title: Dimensions and Ontology
summary: Define reusable behavior axes and their allowed values, then inspect how cases and benchmarks cover them.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-10-30
canonical: /docs/coverage-engineering/dimensions-ontology
---
# Dimensions and Ontology
## Definition
Dimensions are reusable project-level axes for describing how cases differ. Each Dimension contains ontology values: the named members used to classify cases and measure representation. A Dimension might be **Source condition**, with values such as **Current**, **Superseded**, **Conflicting**, and **Missing**.
Use **Project Foundations → Coverage Facets → Dimension** to create, generate, inspect, and maintain them.
## Fields, states, or lifecycle rules
### What a Dimension contains
| Element | Purpose |
| --- | --- |
| Name and description | Explain the behavior axis and its boundary |
| Ontology values | Define the values used for classification |
| Examples | Show classified cases and the reason for a value assignment |
| Statistics | Show case-pool and benchmark distribution by ontology value |
| Benchmark focus | Show whether values are required, sampled, diagnostic, or ignored in benchmark setup |
Dimensions and ontology values are project foundations. A benchmark does not copy them. **Canvas settings** assigns benchmark-specific roles to the project values, and **Representation** reports how the selected cases cover them.
Use the table to compare each Dimension's definition with its ontology and current classification footprint before opening the detail view.
### Create or generate a schema
Create a Dimension manually when the axis and vocabulary are already understood. Use the dimension-schema generator when project context or source material should produce a reviewable proposal. Generated proposals can include a definition, why the Dimension matters, proposed ontology members, and warnings.
A proposal is not the active schema. Review each proposed Dimension and value before accepting it. Avoid accepting near-duplicates simply because they use different wording.
> Classification boundary
>
> Creating or editing a Dimension does not instantly classify every existing case. Missing or stale classifications can be queued and monitored separately. Treat unclassified cases as missing supporting information, not as an implicit ontology value.
### Design rules
- Make the Dimension answer one stable question. Split axes that mix several independent concerns.
- Give every ontology value a definition that distinguishes it from neighboring values.
- Prefer values that can be applied consistently to real cases.
- Do not use a Dimension to encode case quality, policy approval, or a desired model score.
- Review distributions after changing values. A clean schema can still leave important cases unclassified.
- Delete only after checking Case Pool and benchmark usage; removal changes the project coverage vocabulary.
### Benchmark roles
In benchmark Canvas settings, each Dimension and ontology value can receive a focus role:
- **Required:** the benchmark is expected to cover this value.
- **Sampled:** include it as part of the desired mix.
- **Diagnostic:** track it for analysis without making it part of the main denominator.
- **Ignored:** exclude it from the benchmark coverage intention.
- **Unset:** no explicit benchmark instruction has been recorded.
Those roles shape Coverage Story generation and interpretation. They do not alter the project-level definition of the value.
{% example-demo title="Example: source condition" %}
The project defines a Source condition Dimension with Current, Superseded, Conflicting, and Missing values. One benchmark marks all four as required; another marks Current as required and the remaining values as diagnostic. The same project vocabulary supports different benchmark intentions without duplicating the Dimension.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema)
- [Topics](/docs/coverage-engineering/project-topics)
- [Plan Benchmark Coverage](/docs/coverage-engineering/plan-benchmark-coverage)
- [Dataset Representation](/docs/benchmark-datasets/representation)
{% /related-card-grid %}
---
id: coverage.project-topics
title: Topics
summary: Maintain source-grounded subject areas, editable groups, and Atlas relationships used to organize project and benchmark coverage.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-10-30
canonical: /docs/coverage-engineering/project-topics
---
# Topics {% #project-topics %}
## Definition
Topics are reusable project-level subject areas derived from project knowledge, entered by an administrator, or contributed by an expert. They organize cases by what they are about. Unlike a Dimension, a Topic is not one value on a fixed behavior axis: a case can belong to several Topics.
The Topics workspace contains **Topics**, **Groups**, and **Atlas**.
## Fields, states, or lifecycle rules
### Topics
Each Topic has a name, description, active state, author type, case usage, benchmark usage, and source mentions where available. Source mentions retain provenance such as the originating Reference Material, Project Agent Brief, manual entry, or expert suggestion.
The Topic detail shows its definition and source grounding, linked and example cases, coverage confidence, nearby-topic signals, coverage by Dimensions and ontology values, and editable membership in Topic Groups.
An AI-created or source-extracted Topic remains an editable project artifact. Review its definition and grounding before using it to steer a benchmark.
### Groups
A Topic Group is an editable bundle of Topics. It does not merge or replace its members. Groups let benchmark setup express intent at a useful scale while retaining Topic-level traceability.
In **Canvas settings**, a Group can be handled as:
- **Cover every topic:** the benchmark should represent each member Topic.
- **Cover the group:** the group should be represented without requiring every member.
- **Use as guidance:** it can guide story construction without becoming a coverage obligation.
- **Do not use:** exclude the Group from this benchmark's setup.
- **Unset:** no explicit instruction.
Groups can start from manual work, AI suggestions, source material, or Atlas exploration. Manual edits remain significant; regenerating a suggestion should not be treated as authority to overwrite the reviewed group.
### Atlas
Atlas visualizes Topics, relationships, and Groups. Use it to inspect neighborhoods, redundancy, missing nearby Topics, and possible groupings. Relationships provide analytical context, not a taxonomy merge. A close position or strong relation score does not mean two Topics are interchangeable.
The Topic coverage view can distinguish direct case grounding from breadth, Dimension spread, and binding confidence. When a Topic looks thin, inspect the linked cases before generating more. The problem may be missing cases, weak classification, an overly broad definition, or a duplicate Topic.
### Refresh and review
Topics can be refreshed from current project context and indexed sources. Refreshing can create or update Topics, source mentions, supporting information, and relationship analysis. Review the resulting changes and recommendation runs before incorporating them into Groups or benchmark setup.
{% example-demo title="Example: group without flattening" %}
The project has Topics for Contract renewal, Price adjustment, and Termination notice. An administrator groups them as Agreement lifecycle. One benchmark chooses Cover every topic because each action has distinct risk. Another chooses Use as guidance because it only needs broad agreement-related examples. The individual Topics remain traceable in both benchmarks.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Dimensions and Ontology](/docs/coverage-engineering/dimensions-ontology)
- [Scenarios](/docs/coverage-engineering/scenarios)
- [Coverage Design Settings](/docs/coverage-design/settings)
- [Dataset Representation](/docs/benchmark-datasets/representation)
{% /related-card-grid %}
---
id: coverage-management.coverage-stories
title: Coverage Stories
summary: Describe meaningful input situations and testable facet combinations used in Coverage Design and Case preparation.
kind: task
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design/coverage-stories
---
# Coverage Stories
Stories explain the situation behind a coverage requirement. They connect Benchmark intent to testable facet combinations and the Cases prepared to exercise them.
## Prerequisites
- An identified Benchmark intent and current coverage requirements.
- Dimensions, Topics or Scenarios that describe the relevant distinctions.
- A Case input contract that can represent the situation.
## Make a Story concrete
Describe the actor, request, source condition, conflict or constraint that changes expected behavior. A Story should help an operator explain why a slice matters without treating a chart category as a complete situation.
Tuples name smaller obligations. Their statement of what must be demonstrated, facet references and intended count guide preparation and review. A tuple's requested count does not prove that its Cases are realistic or distinct.
## Read Stories with Canvas
Use Canvas to inspect current requirements and selected Cases. Stories supply planning context; requirement assessments supply current matched, unknown and remaining counts. Do not substitute a Story budget for actual dataset membership.
Where a Story is created or edited through the available controls, preserve its rationale and relationship to the relevant facets. An archived Story is history rather than a new instruction to remove every related Case.
## Materials
Project Input Schema supplies allowed artifacts and required material keys. Story planning can request an appropriate allowed artifact. It cannot disable required input, add an unaccepted family or create another materials policy.
## Object and state changes
Story edits change current planning context. Prepared work records the Story and input contract it accepted. Existing Snapshots and completed Runs retain their original Cases and criteria.
## Success criteria
- A reviewer can tell what situation each tuple must demonstrate.
- Facet references identify the intended project concepts.
- Counts are interpreted alongside actual Case quality and membership.
- Current requirements and captured preparation inputs are distinguished.
## Common failure modes
- Using a generic title as the entire requirement.
- Counting near-duplicate Cases as distinct coverage.
- Reconstructing an accepted preparation from today's edited Story.
- Treating Story generation as automatic Case adoption.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design](/docs/coverage-design)
- [Coverage Design Settings](/docs/coverage-design/settings)
- [Case Foundry](/docs/coverage-design/case-foundry)
- [Review Prepared Cases](/docs/coverage-design/case-review)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage)
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
{% /related-card-grid %}