# Construct controlled variations
Generated: 2026-11-13T08:01:25.390Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: workflow.assets.controlled-variations
title: Construct controlled variations
summary: Vary the condition under investigation while preserving enough shared context to interpret the difference.
kind: task
product_area: assets
status: stable
updated: 2026-11-11
canonical: /docs/assets/controlled-variations
---
# Construct controlled variations
Vary the condition under investigation while preserving enough shared context to interpret the difference.
## Prerequisites
A defined coverage objective, Project Input Schema, and suitable source Cases or synthesis context.
## Steps
1. Identify the Dimension, Topic, or Scenario distinction the variation should exercise.
2. Specify which facts should change and which context must remain fixed. For input variations, follow [Synthesize Cases with Lemon Weave](/docs/coverage-engineering/synthesize-cases): use Canvas → Shots and supply the distinction in Additional instructions. For response alternatives, attach [Comparison Directions](/docs/assets/comparison-directions) when preparing the Expert Contribution.
3. Inspect each prepared candidate for realism, input validity, and the intended difference.
4. Review prepared input candidates in the Canvas preview and confirm selected membership separately. Keep response alternatives attached to their Contribution; comparison alone does not make them canonical Cases or add them to the Benchmark.
## Object and state changes
Generated candidates, expert review alternatives, canonical Cases, and dataset membership remain separate records and decisions.
## Success criteria
A reviewer can attribute a different expected judgment to the intended condition rather than an accidental change in the input.
## Common failure modes
Changing several unrelated conditions at once; accepting a fluent variation that breaks the task’s source assumptions.
## Related reference pages
{% related-card-grid title="Continue this workflow" %}
- [Synthesize Cases with Lemon Weave](/docs/coverage-engineering/synthesize-cases)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Resolve a problem" %}
- [Unclear Cases](/docs/troubleshooting/unclear-cases)
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
{% /related-card-grid %}
---
id: coverage.synthesize-cases
title: Synthesize Cases with Lemon Weave
summary: Generate benchmark-scoped input candidates from explicit coverage needs and the project input/material contract.
kind: task
product_area: coverage_engineering
status: stable
updated: 2026-11-11
canonical: /docs/coverage-engineering/synthesize-cases
---
# Synthesize Cases with Lemon Weave
Synthesize when a named input situation is missing and its expected behavior is clear enough to review.
Production Case uploads are optional. You can supply approved requirements and fictional source context, then generate the input and materials needed for the situation. For the complete data and execution boundary, read [Evaluate Without Uploading Production Cases](/docs/security/synthetic-evaluation).
## Prerequisites
- An explicit coverage requirement or tuple.
- Relevant project facets and source context.
- A Project Input Schema that represents the intended input and materials.
- A reviewer who can assess realism and fit.
## Steps
1. Open the intended Benchmark's Coverage Design → Canvas.
2. Identify the missing situation and inspect existing Cases before generating more.
3. For newly synthesized additions, open **Shots**, start a **New Shot**, choose **Add**, and set **Cases to add**.
4. Select the conditions and supply **Additional instructions**. Inspect the Project Agent Brief, facet context and any **Sample cases** that generation will receive.
5. Choose **Stage shot**, review the preview, then choose **Prepare staged shots** when it is ready.
6. Inspect prepared candidates and required materials in the Canvas preview, approve acceptable Cases, and check current membership and the remaining requirement.
7. Create a new Snapshot when the changed population should define a later evaluation.
Use **Assets → Cases** to import real examples, connect integrations, or create a reusable Case manually.
## Input and material authority
Project Input Schema supplies allowed artifact families, accepted extensions and required material keys. A coverage tuple can request an allowed artifact but cannot override that contract.
Keep retrieved and synthesized source relationships visible. A polished response or a completed material job does not establish that the input demonstrates its intended situation.
Shot additions request new synthesis. Rules preparation can reuse existing Cases while filling its remaining need. Synthesis can use supplied sample Cases and Project context in either workflow. For an experiment that excludes actual records, use the source and context review in [Evaluate Without Uploading Production Cases](/docs/security/synthetic-evaluation).
Cases generated for an Inbox observation are staged as separate items. Their creation does not automatically adopt them or resolve the observation.
## Object and state changes
Generation records accepted planning/input settings and prepares candidates. Review and adoption determine editable membership. Historical Snapshots and Runs stay unchanged.
## Success criteria
- Each candidate demonstrates a named input need.
- The input is realistic and conformant.
- Required materials are available and inspectable.
- Source provenance and adoption state remain distinguishable.
## Common failure modes
- Asking for volume without a coverage purpose.
- Generating around an unresolved correctness standard.
- Assuming accepted extensions require every format.
- Counting generation completion as approved membership.
- Preparing Cases without checking the selected Benchmark and coverage need.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design](/docs/coverage-design)
- [Review Prepared Cases](/docs/coverage-design/case-review)
- [Input Coverage Inbox](/docs/coverage-design/inbox)
- [Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
- [Unclear Cases](/docs/troubleshooting/unclear-cases)
{% /related-card-grid %}
---
id: troubleshooting.unclear-cases
title: Unclear Cases
summary: Repair Cases that informed reviewers cannot interpret consistently from the supplied input, context, and output identity.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/unclear-cases
---
# Unclear Cases
## Symptoms
- Reviewers ask what they are supposed to judge.
- The same Case produces incompatible readings because a source, prior turn, time boundary, or user state is absent.
- A candidate answer is visible, but it is unclear whether it is the judged output or reference behavior.
- The Case title or metadata implies context that the review presentation does not contain.
- Experts can answer only by inventing assumptions.
## Likely causes
- A prior conversation turn, source, time boundary, or user state is missing.
- Candidate and reference outputs are swapped or unlabeled.
- One Case combines multiple situations or requested judgments.
- Important instructions live in metadata or private project knowledge instead of visible context.
## Diagnose the Case, not the reviewer
1. Open the Case in its normal review presentation and read only what the assigned expert can see.
2. Identify the judged input and output separately. For conversations, confirm the turn order and which response is under review.
3. List every fact required to make the judgment, then verify each fact is present as Case content, context, source material, or an explicitly linked standard.
4. Check whether metadata is being used as hidden instruction rather than visible context.
5. Compare reviewer rationale. Repeated invented assumptions usually reveal the missing boundary.
## Fix
- Add or map the missing context when it belongs to the real situation.
- Correct the output attachment when candidate and reference behavior were swapped.
- Split a Case that combines two independently judged situations.
- Narrow the question or applicability boundary when multiple interpretations are legitimate.
- Mark the Case unresolved or keep it as candidate material when authoritative source context is unavailable.
- Create a new Case version when the reviewable content changes; do not silently rewrite content captured by an existing Snapshot.
After correction, have a reviewer who did not author the Case explain the situation and requested judgment. If they must rely on private project knowledge, the Case is still incomplete.
## Prevention
Review the Case through the same presentation an expert will see, label output identity explicitly, and include every fact required for judgment. Sample imported and generated Cases before Benchmark selection, and preserve earlier Case content when recording a meaningful change.
{% example-demo title="Example: recommendation without the user's equipment" %}
A Case asks whether a product is compatible but omits the equipment model mentioned in an earlier conversation turn. Reviewers alternate between pass and fail. The operator restores the prior turn, confirms the candidate output identity, versions the Case, and repeats the affected review. The old Snapshot retains the earlier ambiguous version.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Assets](/docs/assets)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Case Versioning](/docs/governance/case-versioning)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Cases](/docs/object-model/cases)
- [Represent conversations in Cases](/docs/object-model/represent-conversations-in-cases)
- [Metadata and context](/docs/reference/metadata-and-context)
{% /related-card-grid %}
---
id: troubleshooting.unrealistic-synthetic-cases
title: Synthetic Cases That Feel Unrealistic
summary: Improve generated candidates whose surface details or behavior assumptions do not represent plausible target-system use.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/unrealistic-synthetic-cases
---
# Synthetic Cases That Feel Unrealistic
## Symptoms
- Generated language sounds artificial or describes a user who would not exist in the product.
- Required context appears conveniently in the prompt even though the real system would retrieve it elsewhere.
- The Case combines incompatible ontology values or violates product constraints.
- Reviewers can identify the intended coverage tuple but would never admit the Case to a Benchmark.
- Many generated candidates repeat the same template with superficial wording changes.
## Likely causes
- The synthesis request lacks representative source Cases or product constraints.
- A coverage tuple combines values that cannot coexist in the target system.
- A Scenario or tuple is too broad, stale, or inconsistent with the target system.
- Generated candidates are being treated as reviewed Benchmark Cases before admission.
## Diagnose the generation boundary
1. Open the intended Benchmark's **Coverage Design → Canvas** and identify the coverage slice used for Case preparation. Use **Assets → Cases** to import real examples or create a reusable Case manually.
2. Compare the generated Case with real in-use Cases from the same behavior region.
3. Separate a bad surface realization from a bad coverage target. Awkward wording may need editing; an impossible product state requires revised Scenario or tuple context.
4. Check whether the generator lacked constraints that operators hold only in private memory.
5. Inspect the synthesis lifecycle and candidate state. A completed generation is still candidate material, not a selected Benchmark Case.
## Fix
- Add representative source Cases or explicit product constraints to the synthesis context.
- Refine the Scenario or tuple when its construction guidance combines incompatible conditions. Keep response-alternative guidance in Comparison Directions.
- Generate from a specific missing coverage tuple rather than asking for generic diversity.
- Edit a promising candidate when its behavioral boundary is sound and only presentation is weak.
- Reject candidates whose premise is impossible or whose source context cannot be justified.
- Review duplicates before adding any generated Case to a Benchmark.
Validate revised candidates with an expert who knows the target workflow. Ask whether the situation could occur, whether the system would receive the shown context, and whether the requested judgment matters. Only then classify and curate the Case for possible benchmark use.
## Prevention
Ground synthesis in reviewed source Cases, explicit product constraints, and a named coverage gap. Review the Scenario and tuple, inspect duplicates, and require Case review before adding generated candidates to a Benchmark or Snapshot.
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Synthesize Cases](/docs/coverage-engineering/synthesize-cases)
- [Scenarios](/docs/coverage-engineering/scenarios)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Case Pool](/docs/coverage-engineering/case-pool)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Case Pool surface](/docs/object-model/case-pool)
- [Cases](/docs/object-model/cases)
- [Coverage dimensions](/docs/object-model/coverage-dimensions)
{% /related-card-grid %}