# Unclear Cases and unrealistic synthesis
Generated: 2026-11-13T08:01:26.676Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: workflow.troubleshooting.case-quality
title: Unclear Cases and unrealistic synthesis
summary: Experts cannot interpret a Case consistently, or a generated situation does not fit its intended context.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-11
canonical: /docs/troubleshooting/case-quality
---
# Unclear Cases and unrealistic synthesis
## Symptom
Experts cannot interpret a Case consistently, or a generated situation does not fit its intended context.
## Likely causes
| What you observe | Focused guide |
| --- | --- |
| The input lacks controlling context | [Unclear Cases](/docs/troubleshooting/unclear-cases) |
| A generated Case conflicts with its source assumptions | [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases) |
## Fix
Read the input, supplied materials, and intended coverage distinction. Clarify the missing condition or source, then inspect the revised candidate before admitting it to the benchmark. A fluent answer does not repair an incoherent Case.
Follow the focused guide for the actual state you observe; keep accepted records and explicit human decisions intact.
## Prevention
Preserve exact object and version identities, review the relevant source context, and inspect the accepted state before comparing outcomes.
## Related task pages
{% related-card-grid title="Return to the workflow" %}
- [Complete your first alignment loop](/docs/quickstart)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Diagnose the observed state" %}
- [Unclear Cases](/docs/troubleshooting/unclear-cases)
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
{% /related-card-grid %}
---
id: troubleshooting.unclear-cases
title: Unclear Cases
summary: Repair Cases that informed reviewers cannot interpret consistently from the supplied input, context, and output identity.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/unclear-cases
---
# Unclear Cases
## Symptoms
- Reviewers ask what they are supposed to judge.
- The same Case produces incompatible readings because a source, prior turn, time boundary, or user state is absent.
- A candidate answer is visible, but it is unclear whether it is the judged output or reference behavior.
- The Case title or metadata implies context that the review presentation does not contain.
- Experts can answer only by inventing assumptions.
## Likely causes
- A prior conversation turn, source, time boundary, or user state is missing.
- Candidate and reference outputs are swapped or unlabeled.
- One Case combines multiple situations or requested judgments.
- Important instructions live in metadata or private project knowledge instead of visible context.
## Diagnose the Case, not the reviewer
1. Open the Case in its normal review presentation and read only what the assigned expert can see.
2. Identify the judged input and output separately. For conversations, confirm the turn order and which response is under review.
3. List every fact required to make the judgment, then verify each fact is present as Case content, context, source material, or an explicitly linked standard.
4. Check whether metadata is being used as hidden instruction rather than visible context.
5. Compare reviewer rationale. Repeated invented assumptions usually reveal the missing boundary.
## Fix
- Add or map the missing context when it belongs to the real situation.
- Correct the output attachment when candidate and reference behavior were swapped.
- Split a Case that combines two independently judged situations.
- Narrow the question or applicability boundary when multiple interpretations are legitimate.
- Mark the Case unresolved or keep it as candidate material when authoritative source context is unavailable.
- Create a new Case version when the reviewable content changes; do not silently rewrite content captured by an existing Snapshot.
After correction, have a reviewer who did not author the Case explain the situation and requested judgment. If they must rely on private project knowledge, the Case is still incomplete.
## Prevention
Review the Case through the same presentation an expert will see, label output identity explicitly, and include every fact required for judgment. Sample imported and generated Cases before Benchmark selection, and preserve earlier Case content when recording a meaningful change.
{% example-demo title="Example: recommendation without the user's equipment" %}
A Case asks whether a product is compatible but omits the equipment model mentioned in an earlier conversation turn. Reviewers alternate between pass and fail. The operator restores the prior turn, confirms the candidate output identity, versions the Case, and repeats the affected review. The old Snapshot retains the earlier ambiguous version.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Assets](/docs/assets)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Case Versioning](/docs/governance/case-versioning)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Cases](/docs/object-model/cases)
- [Represent conversations in Cases](/docs/object-model/represent-conversations-in-cases)
- [Metadata and context](/docs/reference/metadata-and-context)
{% /related-card-grid %}
---
id: troubleshooting.unrealistic-synthetic-cases
title: Synthetic Cases That Feel Unrealistic
summary: Improve generated candidates whose surface details or behavior assumptions do not represent plausible target-system use.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/unrealistic-synthetic-cases
---
# Synthetic Cases That Feel Unrealistic
## Symptoms
- Generated language sounds artificial or describes a user who would not exist in the product.
- Required context appears conveniently in the prompt even though the real system would retrieve it elsewhere.
- The Case combines incompatible ontology values or violates product constraints.
- Reviewers can identify the intended coverage tuple but would never admit the Case to a Benchmark.
- Many generated candidates repeat the same template with superficial wording changes.
## Likely causes
- The synthesis request lacks representative source Cases or product constraints.
- A coverage tuple combines values that cannot coexist in the target system.
- A Scenario or tuple is too broad, stale, or inconsistent with the target system.
- Generated candidates are being treated as reviewed Benchmark Cases before admission.
## Diagnose the generation boundary
1. Open the intended Benchmark's **Coverage Design → Canvas** and identify the coverage slice used for Case preparation. Use **Assets → Cases** to import real examples or create a reusable Case manually.
2. Compare the generated Case with real in-use Cases from the same behavior region.
3. Separate a bad surface realization from a bad coverage target. Awkward wording may need editing; an impossible product state requires revised Scenario or tuple context.
4. Check whether the generator lacked constraints that operators hold only in private memory.
5. Inspect the synthesis lifecycle and candidate state. A completed generation is still candidate material, not a selected Benchmark Case.
## Fix
- Add representative source Cases or explicit product constraints to the synthesis context.
- Refine the Scenario or tuple when its construction guidance combines incompatible conditions. Keep response-alternative guidance in Comparison Directions.
- Generate from a specific missing coverage tuple rather than asking for generic diversity.
- Edit a promising candidate when its behavioral boundary is sound and only presentation is weak.
- Reject candidates whose premise is impossible or whose source context cannot be justified.
- Review duplicates before adding any generated Case to a Benchmark.
Validate revised candidates with an expert who knows the target workflow. Ask whether the situation could occur, whether the system would receive the shown context, and whether the requested judgment matters. Only then classify and curate the Case for possible benchmark use.
## Prevention
Ground synthesis in reviewed source Cases, explicit product constraints, and a named coverage gap. Review the Scenario and tuple, inspect duplicates, and require Case review before adding generated candidates to a Benchmark or Snapshot.
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Synthesize Cases](/docs/coverage-engineering/synthesize-cases)
- [Scenarios](/docs/coverage-engineering/scenarios)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Case Pool](/docs/coverage-engineering/case-pool)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Case Pool surface](/docs/object-model/case-pool)
- [Cases](/docs/object-model/cases)
- [Coverage dimensions](/docs/object-model/coverage-dimensions)
{% /related-card-grid %}