Teammately Docs
Docs menu

task

Synthesize cases

Generate candidate cases to fill coverage gaps before adding them to benchmarks or curated datasets.

Synthesize cases

Synthesize candidate cases to fill a known coverage gap without pretending synthetic examples are automatically review context.

When to use it

Use Assets → Cases → Synthesize when creating reusable project Case candidates. Use Coverage Management → Case Foundry when the work begins from a named need in one Benchmark. In either path, generated output is candidate material until reviewed.

Prerequisites

  • A named coverage gap.
  • Policies, rubrics, or dimensions that define the behavior boundary.
  • A review plan for synthetic cases.
  • A way to mark synthetic source and realism concerns.

Role or permission

AI engineers or coverage owners generate candidate cases. Experts or product owners review realism before benchmark promotion.

Steps

Synthesize by tuple screen with selected ontology dimensions and a comparison benchmark selector.

Synthesis starts from a named tuple or coverage gap, not from a generic request for more cases.

  1. Name the Coverage Facet tuple, Coverage Story, or Case Construction Pattern the synthesis should fulfill.
  2. Choose the Assets synthesis path or benchmark-scoped Case Foundry path.
  3. Supply representative source Cases, applicable Comparison Directions, and explicit target-system constraints.
  4. Inspect generated candidates for realism, duplication, source context, and compatibility with the intended Coverage Facets.
  5. Keep impossible, misleading, or context-incomplete candidates out of Benchmark selection.
  6. Use Coverage Management → Case Review for benchmark-scoped preparation and admission.
  7. Select reviewed Cases in Benchmark Datasets and create a Snapshot only when the intended set is ready.

Object and state changes

Completed generation registers candidate Cases in the project Case collection. Case Review and Benchmark Datasets determine whether suitable Cases become selected membership for a Benchmark. Generation completion alone does not change a Snapshot or Benchmark Version.

Success criteria

  • Each synthetic case maps to a named gap or boundary.
  • Reviewers can tell synthetic cases from production-derived cases.
  • Only realistic, context-complete cases affect benchmark evidence.

Common failure modes

  • Synthetic cases are generated for volume rather than a specific gap.
  • Cases combine many variations and obscure the failure reason.
  • Unreviewed synthetic cases enter benchmark evidence.
  • The generated case lacks the context needed for expert judgment.

Related reference pages

Reference / Object ModelCase Pool surfaceUnderstand Case Pool as the Assets view for reusable project Cases, candidates, and explicit Benchmark selection.Reference / Object ModelCoverage dimensionsOrganize cases by the behavior axes that matter to product correctness.Reference / AssetsComparison DirectionsCreate reusable guidance for meaningful candidate-output differences in comparative expert work.

Related troubleshooting pages

Troubleshooting / Recovery LibrarySynthetic Cases That Feel UnrealisticImprove generated candidates whose surface details or behavior assumptions do not represent plausible target-system use.Troubleshooting / Recovery LibraryUnbalanced CoverageFix benchmarks that overrepresent easy or common cases while missing important behavior.Troubleshooting / Recovery LibraryUnclear CasesRepair Cases that informed reviewers cannot interpret consistently from the supplied input, context, and output identity.

Worked example

Fill a stale-source gap

Coverage shows few stale-source cases. The engineer synthesizes examples with old and new policy documents, experts reject unrealistic ones, and only reviewed cases enter the benchmark candidate pool.

Source confidence

Code-backed: the Assets synthesizer defines generation input, candidate cards, and lifecycle; Case Foundry supplies benchmark-scoped coordination; Case Review supplies the preparation and admission boundary. The product does not treat generation completion as Snapshot membership.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Coverage Engineering Case Pool Use the Case Pool to collect, triage, enrich, and promote candidate cases. Coverage Engineering Plan Benchmark Coverage Apply project Coverage Facets to one benchmark, inspect representation, and turn important gaps into concrete case or contribution work. Object Model Dimensions and ontology Learn how dimensions and ontology values define the coverage space for a Teammately project.
AI context