# Coverage Gaps
Generated: 2026-09-13T04:33:47.736Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: coverage.coverage-gaps
title: Coverage Gaps
summary: Find missing or underrepresented behavior areas before benchmark evidence becomes misleading.
kind: task
product_area: coverage_engineering
status: stable
updated: 2026-08-23
canonical: /docs/coverage-engineering/coverage-gaps
---
# Coverage Gaps
## When to use it
Use this task when the team suspects that a benchmark result is incomplete because the case set does not represent an important behavior area. A coverage gap is not just a low score. It is a missing or thin slice of the behavior space: a dimension value, ontology category, source condition, user intent, boundary scenario, policy exception, failure cluster, or product flow that should be represented before results are trusted.
Coverage gaps matter because Teammately helps the team reason about whether the benchmark actually represents the correctness space, instead of only running checks over available examples.
## Prerequisites
- A Benchmark Dataset or a named intended behavior slice already exists.
- Dimensions or ontology values are available, or the team knows which behavior axis is missing.
- Relevant Evaluation Runs, failure clusters, Expert Contribution findings, or product signals are available for inspection.
- Policies and rubrics are clear enough that the team can tell whether the problem is missing coverage rather than weak standards.
## Required role or permission
AI engineers, evaluation owners, and product owners usually identify coverage gaps together. Experts may be needed when the missing behavior depends on domain judgment. If the UI blocks investigation or case changes, inspect project membership and artifact access before changing the benchmark.
### Task steps: Coverage Gaps
1. Name the behavior area that may be missing: dimension, ontology value, product flow, policy exception, source condition, or boundary scenario.
2. Inspect **Benchmark Datasets → Representation** and the current Benchmark Version. Check whether the area is absent, represented by too few selected Cases, or represented only by easy examples.
3. Compare the suspected gap against evaluation failures, failure clusters, Expert Contribution notes, and recent product signals.
4. Rule out look-alike problems: missing outputs, stale cases, weak applicability logic, overly broad policies, ambiguous rubrics, or output mapping errors.
5. Route the gap: update Coverage Facets, create a Coverage Story, source or synthesize Cases, request an Expert Contribution, or select already reviewed Cases in Benchmark Datasets.
6. Review candidates in Case Review, create a new Snapshot when membership changes, and preserve the gap rationale in the owning coverage surfaces.
When a gap points to specific candidates, the operator can select cases and prepare them for benchmark membership.
## Object and state changes
Confirming a gap can create a Coverage Story, candidate Cases, Coverage Facet changes, Case Review work, selected Dataset changes, or an Expert Contribution. A gap does not silently change historical Benchmark meaning. When selected membership changes, create a new Dataset Snapshot and Benchmark Version before treating the revised set as reproducible evidence.
## Success criteria
- The missing or underrepresented behavior area is named precisely.
- The team can explain why the issue is a true coverage gap rather than missing outputs, weak applicability, stale artifacts, or mapping errors.
- The resulting case, dimension, ontology, or benchmark change is traceable to source evidence or expert judgment.
- Future benchmark results can distinguish behavior improvement from coverage refresh.
## Common failure modes
- Treating a model failure as a coverage gap when the benchmark already contains representative cases.
- Adding many similar cases without naming the missing dimension or ontology value.
- Refreshing benchmark coverage without preserving the version boundary.
- Mistaking missing outputs or output mapping failures for missing coverage.
- Creating synthetic cases that are unrealistic because they lack source context or expert judgment.
- Ignoring a small high-risk slice because aggregate coverage looks balanced.
{% example-demo title="Boundary case for enterprise search" %}
Raw case: An employee asks for a policy that changed last week, and the retrieved documents contain both old and new guidance.
Expert judgment: Coverage must include cases where stale and current sources conflict.
Policy: Answers must prefer the approved current source and disclose conflicts when confidence is low.
Applicability: Applies when retrieval includes multiple policy versions or stale documents.
Binary rubric: The answer identifies the current source or asks for confirmation instead of blending policies.
Benchmark result: A candidate output fails because it combines old and new terms into one invented policy.
Interpretation: Coverage notes show whether stale-source boundary behavior is represented before the next run is trusted.
{% /example-demo %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Candidate and In-Use Cases](/docs/coverage-engineering/candidate-and-in-use-cases)
- [Case pool](/docs/coverage-engineering/case-pool)
- [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology)
- [Benchmark snapshots](/docs/coverage-engineering/benchmark-snapshots)
- [Case versions](/docs/governance/case-versioning)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Access troubleshooting](/docs/troubleshooting/authentication)
- [Unbalanced coverage](/docs/troubleshooting/unbalanced-coverage)
- [Weak applicability logic](/docs/troubleshooting/weak-applicability-logic)
- [Missing outputs](/docs/troubleshooting/missing-outputs)
- [Benchmark results changed unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
{% /related-card-grid %}
## Source confidence
Code-backed: Benchmark Dataset Representation exposes selected distribution; Coverage Management and Coverage Stories expose benchmark needs; Case Review exposes the admission boundary for prepared Cases. Human judgment determines whether an observed thin slice is consequential.
---
id: coverage.dimensions-ontology
title: Dimensions and Ontology
summary: Define reusable behavior axes and their allowed values, then inspect how cases and benchmarks cover them.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/dimensions-ontology
---
# Dimensions and Ontology
## Definition
Dimensions are reusable project-level axes for describing how cases differ. Each Dimension contains ontology values: the named members used to classify cases and measure representation. A Dimension might be **Source condition**, with values such as **Current**, **Superseded**, **Conflicting**, and **Missing**.
Use **Project Foundations → Coverage Facets → Dimension** to create, generate, inspect, and maintain them.
## Fields, states, or lifecycle rules
### What a Dimension contains
| Element | Purpose |
| --- | --- |
| Name and description | Explain the behavior axis and its boundary |
| Ontology values | Define the values used for classification |
| Examples | Show classified cases and the reason for a value assignment |
| Statistics | Show case-pool and benchmark distribution by ontology value |
| Benchmark focus | Show whether values are required, sampled, diagnostic, or ignored in benchmark setup |
Dimensions and ontology values are project foundations. A benchmark does not copy them. **Get Started** assigns benchmark-specific roles to the project values, and **Representation** reports how the selected cases cover them.
Use the table to compare each Dimension's definition with its ontology and current classification footprint before opening the detail view.
### Create or generate a schema
Create a Dimension manually when the axis and vocabulary are already understood. Use the dimension-schema generator when project context or source material should produce a reviewable proposal. Generated proposals can include a definition, why the Dimension matters, proposed ontology members, and warnings.
A proposal is not the active schema. Review each proposed Dimension and value before accepting it. Avoid accepting near-duplicates simply because they use different wording.
> Classification boundary
>
> Creating or editing a Dimension does not instantly classify every existing case. Missing or stale classifications can be queued and monitored separately. Treat unclassified cases as missing evidence, not as an implicit ontology value.
### Design rules
- Make the Dimension answer one stable question. Split axes that mix several independent concerns.
- Give every ontology value a definition that distinguishes it from neighboring values.
- Prefer values that can be applied consistently to real cases.
- Do not use a Dimension to encode case quality, policy approval, or a desired model score.
- Review distributions after changing values. A clean schema can still leave important cases unclassified.
- Delete only after checking Case Pool and benchmark usage; removal changes the project coverage vocabulary.
### Benchmark roles
In benchmark Get Started, each Dimension and ontology value can receive a focus role:
- **Required:** the benchmark is expected to cover this value.
- **Sampled:** include it as part of the desired mix.
- **Diagnostic:** track it for analysis without making it part of the main denominator.
- **Ignored:** exclude it from the benchmark coverage intention.
- **Unset:** no explicit benchmark instruction has been recorded.
Those roles shape Coverage Story generation and interpretation. They do not alter the project-level definition of the value.
{% example-demo title="Example: source condition" %}
The project defines a Source condition Dimension with Current, Superseded, Conflicting, and Missing values. One benchmark marks all four as required; another marks Current as required and the remaining values as diagnostic. The same project vocabulary supports different benchmark intentions without duplicating the Dimension.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema)
- [Work with Project Topics](/docs/coverage-engineering/project-topics)
- [Plan Benchmark Coverage](/docs/coverage-engineering/plan-benchmark-coverage)
- [Analyze dataset representation](/docs/benchmark-datasets/representation)
{% /related-card-grid %}
## Source confidence
Code-backed: the current Dimension list, detail, settings, proposal, classification, examples, and statistics surfaces establish these fields and lifecycle boundaries.
---
id: coverage.case-pool
title: Case Pool
summary: Use the Case Pool to collect, triage, enrich, and promote candidate cases.
kind: concept
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/case-pool
---
# Case Pool
## Definition
Case Pool is the project-level working set for reusable Cases and Case sourcing activity. The current UI routes Case inspection through **Assets → Cases** and keeps **Sourcing Tasks** under Coverage Engineering. Together they let operators inspect Case content and provenance, follow preparation tasks, classify coverage, and select Cases for one or more Benchmarks.
## Why it matters
Cases often arrive before the team knows whether they are clear, representative, or tied to a meaningful coverage need. The Case Pool provides a project boundary where imported, generated, or contributed Cases can be reviewed without treating every item as benchmark evidence.
## Where it appears in the product
Use Assets → Cases for the reusable Case collection. Use Sourcing Tasks to inspect find, synthesize, classification, and preparation activity. From selected Cases, use the supported add-to-benchmark action to change editable Benchmark Dataset membership. Use Benchmark Datasets to inspect the selected set and create Snapshots.
## Artifacts it affects
Each Case keeps backend-issued identity, canonical content, materials, source or contributor context, Coverage Facet assignments, and benchmark inclusion where available. Task state is not Case approval, and benchmark inclusion is not Snapshot membership. Preserve those separate states when reporting progress.
## Operational check
Before selecting a Case, inspect its canonical input, required materials, source trace, coverage assignments, and any supported reference output or evaluator relationship relevant to the intended Benchmark. Check for near duplicates and verify that selection closes a named need rather than merely increasing row count.
Selection is explicit: choose the intended Benchmark in this dialog. The action changes editable dataset membership, not a historical Snapshot.
{% example-demo title="Routing a sourced Case" %}
A sourcing task finds a production-informed question involving a superseded policy attachment. The operator opens the Case in Assets, confirms the current Input Schema and both materials, assigns the source-authority Coverage Facet, and adds it to the support Benchmark's current dataset. The team reviews Representation and creates a new Snapshot later; the selection action alone does not change historical Runs.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Assets](/docs/assets)
- [Importing cases](/docs/operating-manual/import-and-prepare-cases)
- [Synthesize cases](/docs/coverage-engineering/synthesize-cases)
- [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology)
- [Product quickstart](/docs/quickstart)
{% /related-card-grid %}
## Source confidence
Code-backed: current Case Pool navigation separates reusable Assets Cases from Sourcing Tasks, while Case sourcing types expose provenance, coverage targets, classification context, and benchmark inclusion. Benchmark Dataset pages own selection and Snapshot evidence.
---
id: coverage.refresh
title: Refresh coverage after product change
summary: Reconcile coverage facets, Cases, benchmark membership, and Snapshots after the target system or its evidence changes.
kind: task
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/coverage-refresh
---
# Refresh coverage after product change
## When to use it
Refresh coverage when new Cases reveal an unrepresented behavior, source material or product behavior changes, experts qualify an earlier assumption, or a governed Policy changes which situations matter. This is a coordinated workflow across Coverage Engineering—not a single refresh action.
## Prerequisites
- Name the changed signal and the date or version at which it changed.
- Identify the Benchmark whose claims may be affected.
- Preserve the current Snapshot and historical Runs; do not edit them to resemble the new state.
- Decide who can confirm the changed behavior and who owns the resulting Benchmark Version.
### Task steps: Refresh benchmark coverage
1. Open the Benchmark's **Coverage Management** overview and identify which coverage claim is no longer supported.
2. Review **Coverage Facets**. Update Dimensions, ontology values, Project Topics, or Case Construction Patterns only when the behavior model itself changed.
3. Return to the **Case Pool**. Source, upload, draft, or synthesize candidate Cases for the missing or changed region.
4. Inspect the candidates for source context, realistic inputs, duplication, and the intended coverage labels. Keep uncertain Cases out of benchmark use.
5. Use **Coverage Management → Get Started** and the overview to update coverage guidance. Use **Case Review** and **Benchmark Datasets** to change selected Cases deliberately.
6. Create a new Dataset Snapshot and Benchmark Version for the revised evidence boundary.
7. Run a new evaluation when current candidate evidence is required. Compare it with older Runs using the named Benchmark Versions.
## Object and state changes
A refresh may change coverage-facet definitions, Case classifications, candidate Cases, selected benchmark Cases, and the next Snapshot. It does not rewrite an earlier Snapshot or make its Runs invalid. Older results remain evidence for their original version; the new version answers the current coverage question.
If only candidate behavior changed, keep the Benchmark Version fixed and run the new candidate against it. If the Case set, applicable standards, or coverage boundary changed, create a new Benchmark Version before interpreting a new Run as comparable.
## Success criteria
- The changed product reality maps to an explicit coverage facet or documented boundary.
- Candidate Cases have enough source context to be reviewed and are not mistaken for in-use benchmark evidence.
- The new selected set addresses the gap without silently removing still-important behavior.
- The new Snapshot names a reproducible evidence boundary.
- Comparisons distinguish candidate changes from Benchmark Version changes.
## Common failure modes
- Treating refresh as a single button and missing a changed facet, Case set, or Snapshot boundary.
- Rewriting a historical Snapshot instead of creating a new one.
- Adding generated or newly sourced Cases to a Benchmark before review.
- Comparing Runs without naming whether the candidate, Benchmark Version, or both changed.
{% example-demo title="A newly supported exception" %}
A support assistant gains an approved exception path for one account tier. The team adds or revises the account-tier ontology, sources Cases for eligible and ineligible requests, reviews them in Case Review, updates the coverage guidance, and changes the selected Benchmark Dataset. A new Snapshot freezes the revised membership. Previous Runs still describe the old rule; new Runs evaluate the approved exception boundary.
{% /example-demo %}
## Source confidence
Code-backed: Coverage Management, Coverage Facets, the Case Pool, Case Review, Benchmark Datasets, and Dataset Snapshots establish the current sequence and the objects that can change. The decision that a product change requires a refresh remains a team-owned interpretation of evidence.
## Related reference pages
{% related-card-grid title="Continue the workflow" %}
- [Coverage gaps](/docs/coverage-engineering/coverage-gaps)
- [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology)
- [Case Pool](/docs/coverage-engineering/case-pool)
- [Plan benchmark coverage](/docs/coverage-engineering/plan-benchmark-coverage)
- [Benchmark Snapshots](/docs/coverage-engineering/benchmark-snapshots)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Diagnose refresh problems" %}
- [Stale Dimensions](/docs/troubleshooting/stale-dimensions)
- [Unbalanced coverage](/docs/troubleshooting/unbalanced-coverage)
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
{% /related-card-grid %}