# Coverage classification and representation Generated: 2026-11-13T08:01:24.400Z Source build: local Canonical docs: https://teammately.ai/docs --- id: workflow.troubleshooting.coverage title: Coverage classification and representation summary: The Case population or facet analysis does not represent the intended development objective. kind: error product_area: troubleshooting status: stable updated: 2026-11-11 canonical: /docs/troubleshooting/coverage --- # Coverage classification and representation ## Symptom The Case population or facet analysis does not represent the intended development objective. ## Likely causes | What you observe | Focused guide | | --- | --- | | Classification is missing, ambiguous, or incorrect | [Dimension classification troubleshooting](/docs/troubleshooting/dimension-classification) | | The facet vocabulary changed | [Stale Dimensions](/docs/troubleshooting/stale-dimensions) | | Important conditions are underrepresented | [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage) | ## Fix Check the selected population, vocabulary, unclassified counts, and intended proportions. Correct the owning facet, classification, or membership through its normal workflow; inspect the resulting representation before creating a new Snapshot. Follow the focused guide for the actual state you observe; keep accepted records and explicit human decisions intact. ## Prevention Preserve exact object and version identities, review the relevant source context, and inspect the accepted state before comparing outcomes. ## Related task pages {% related-card-grid title="Return to the workflow" %} - [Complete your first alignment loop](/docs/quickstart) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Diagnose the observed state" %} - [Dimension classification troubleshooting](/docs/troubleshooting/dimension-classification) - [Stale Dimensions](/docs/troubleshooting/stale-dimensions) - [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage) {% /related-card-grid %} --- id: troubleshooting.dimension-classification title: Dimension classification troubleshooting summary: Resolve missing, ambiguous, or inconsistent Case classifications before they distort coverage analysis. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/dimension-classification --- # Dimension classification troubleshooting ## Symptoms - Many Cases show no value for an important Dimension. - Similar Cases receive different ontology values without a clear reason. - One Case appears to belong to several mutually exclusive values. - A coverage gap disappears or appears after labels change, although the Case set did not. - Reviewers cannot tell whether **unknown**, **not applicable**, and missing classification mean different things. ## Likely causes - The Dimension definition or ontology values overlap. - Required Case context is absent from the classification view. - The schema changed after existing Cases were classified. - The Dimension bundles independent behavior axes or leaves absence undefined. ## Diagnose the schema before the Cases 1. Open the Dimension and read its definition, ontology values, examples, and origin. 2. Decide whether the values are intended to be mutually exclusive, multi-label, ordered, or merely descriptive. Do not infer this from label names alone. 3. Sample Cases from each value plus unclassified Cases. Compare the full Case context, not only the short input shown in a table. 4. Look for overlapping definitions, missing fallback treatment, context fields unavailable to classification, or a Dimension that bundles more than one behavior axis. 5. Check whether the Dimension or ontology changed after the Cases were classified. ## Fix - Correct an individual Case classification when the schema is clear and the Case was mislabeled. - Improve the Dimension definition or ontology descriptions when reviewers interpret them differently. - Split a Dimension when one label depends on two independent behavior axes. - Add an explicit unknown or not-applicable treatment when absence carries meaning. - Reclassify affected Cases after a schema change, then review coverage plans and Snapshot boundaries before relying on segment results. Do not repair a misleading coverage chart by editing counts or selecting convenient Cases. Correct the Dimension or classification state that produced it. ## Prevention Define the classification rule and unknown treatment before broad use, attach representative examples to each value, and test boundary Cases with more than one informed reviewer. Review affected classifications whenever the Dimension schema changes. ## Verification Have two informed reviewers classify a small boundary sample using only documented context. Agreement helps establish that the schema is usable; disagreement should produce a clearer definition, better context, or an explicit unresolved boundary—not forced consensus. {% example-demo title="Example: overlapping source-support values" %} Cases alternate between **implied support** and **conflicting source** because one document implies compatibility while another denies it. The team clarifies that any authoritative contradiction uses **conflicting source**, adds examples, and reclassifies the affected Cases. They review the benchmark plan before creating a new Snapshot. {% /example-demo %} ## Related task pages {% related-card-grid title="Related workflows" %} - [Dimensions and Ontology](/docs/coverage-engineering/dimensions-ontology) - [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema) - [Coverage dimensions](/docs/object-model/coverage-dimensions) - [Stale Dimensions](/docs/troubleshooting/stale-dimensions) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference" %} - [Coverage dimensions](/docs/object-model/coverage-dimensions) - [Ontology](/docs/object-model/ontology) - [Metadata and context](/docs/reference/metadata-and-context) {% /related-card-grid %} --- id: troubleshooting.stale-dimensions title: Stale Dimensions summary: Refresh coverage Dimensions and ontology values while preserving historical classifications. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/stale-dimensions --- # Stale Dimensions ## Symptoms - New Cases repeatedly fall into **other**, unknown, or no value. - An ontology value refers to a product state or source hierarchy that no longer exists. - Important failures concentrate in metadata that no Dimension represents. - A coverage plan looks balanced under old labels but reviewers describe a new boundary. - Two values have become indistinguishable after a product change. ## Likely causes - Product behavior or source authority changed while the coverage vocabulary did not. - New Cases reveal an axis the existing schema never represented. - Ontology values were renamed or repurposed without reviewing old classifications. - The apparent staleness is actually incomplete Case classification. ## Confirm staleness First distinguish a stale schema from incomplete classification. Sample new and old Cases using the current Dimension definition. If the existing values still describe the behavior and only new Cases are unlabeled, repair classification. If reviewers need a new concept, different source authority, or changed applicability to classify consistently, the Dimension or ontology may be stale. ## Fix 1. Record the change that made the current vocabulary inadequate. 2. Inspect the Dimension definition, values, examples, origin, and where it is used in coverage plans. 3. Decide whether to rename a value, add a value, split the Dimension, replace it, or preserve it with a historical time boundary. 4. Review representative Cases against the proposed schema before broad reclassification. 5. Reclassify affected current Cases and inspect whether coverage gaps or target distributions changed. 6. Update **Coverage Design → Canvas** and the overview deliberately. Do not change selected Dataset membership merely to preserve an old-looking distribution. 7. Create a new Snapshot when the classification or selected snapshot boundary used by the Benchmark changes. Historical Snapshots and Runs should retain their original interpretation. A new Dimension schema can supersede the current planning model without making the old model disappear. ## Prevention Review Dimension definitions alongside product and source changes, keep representative examples for each ontology value, and inspect unclassified or catch-all Cases regularly. Name the schema and Snapshot boundary used when segment results inform a decision. ## Related task pages {% related-card-grid title="Related workflows" %} - [Refresh coverage after product change](/docs/coverage-engineering/coverage-refresh) - [Dimensions and Ontology](/docs/coverage-engineering/dimensions-ontology) - [Detect and route stale correctness results](/docs/governance/staleness-detection) - [Dimension classification troubleshooting](/docs/troubleshooting/dimension-classification) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference" %} - [Coverage dimensions](/docs/object-model/coverage-dimensions) - [Ontology](/docs/object-model/ontology) - [Versions, staleness, and resolution](/docs/object-model/versions-staleness-and-resolution) {% /related-card-grid %} --- id: troubleshooting.unbalanced-coverage title: Unbalanced Coverage summary: Fix benchmarks that overrepresent easy or common cases while missing important behavior. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/unbalanced-coverage --- # Unbalanced Coverage Use this when the Benchmark Dataset overrepresents some behavior and misses other behavior that matters before human review. ## Symptom A benchmark looks strong in aggregate but reviewers or product owners identify important behavior areas with too few cases. Results may be dominated by common easy cases while rare or boundary cases are absent. ## Likely causes - Case import mirrored production volume instead of product risk. - Dimensions or metadata do not represent the behavior areas that matter. - Synthetic cases were added in one area but not reviewed across the full risk boundary. - Coverage was not refreshed after new signals, policy changes, or failures. ## Diagnostic checks - Inspect **Benchmark Datasets → Representation** and the Coverage Design overview by the configured Coverage Facets. - Look for policies or rubrics with very few applicable cases. - Compare production or expert signals with benchmark case distribution. - Inspect whether new cases are candidate-only or actually in use. ## Fix - Add representative, boundary, or adversarial cases for undercovered areas. - Refresh dimensions or metadata before judging balance. - Review and promote candidate cases into the benchmark only after context is complete. - Record remaining gaps as Coverage Stories or explicit benchmark coverage guidance. ## Prevention - Define coverage targets before benchmark runs. - Track candidate and in-use cases separately. - Review coverage after policy, rubric, or product behavior changes. - Do not use aggregate scores without a coverage note. ## Related task pages {% related-card-grid title="Related task pages" %} - [Dataset Representation](/docs/benchmark-datasets/representation) - [Coverage Design Settings](/docs/coverage-design/settings) - [Coverage Stories](/docs/coverage-design/coverage-stories) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Case Pool surface](/docs/object-model/case-pool) - [Coverage dimensions](/docs/object-model/coverage-dimensions) - [Benchmarks](/docs/object-model/benchmarks) {% /related-card-grid %}