# Review Prepared Cases
Generated: 2026-11-02T05:43:27.608Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: coverage-management.case-review
title: Review Prepared Cases
summary: Inspect generated inputs and materials against their captured coverage purpose before relying on benchmark membership.
kind: task
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design/case-review
---
# Review Prepared Cases
Review prepared Cases for the input situation they actually demonstrate. Preparation success, current inclusion and sufficient coverage are separate conclusions.
## Prerequisites
- Prepared candidates and their captured coverage context.
- Permission to manage benchmark membership.
- The Project Input Schema and required materials used for preparation.
## Inspect candidates
Read the canonical input, source relationship, facet assignments and stated fit to the tuple or requirement. Check required files and values rather than assuming a completed generation produced usable materials.
Remove a candidate that is redundant, unsupported or misplaced through the available controls. Inclusion should reflect a deliberate decision about the current editable dataset. It does not change an older Snapshot.
For a tuple that still lacks suitable Cases, generation can append candidates and regeneration can replace the preparation proposal according to its displayed source mode. Preserve whether material was retrieved, reused or synthesized.
## Materials and compatibility
Project Input Schema owns material keys, accepted families and extensions. A required material must be present and usable. Accepted extensions are alternatives, not a demand to generate every listed format.
Generated-material status and Case status may settle at different times. A partial or failed attachment is not a complete Case. Use the reported failure to correct the owning source or contract rather than silently dropping the required material.
## Object and state changes
Review changes candidate inclusion or editable membership through the owning operation. Additional preparation records another captured request. A Snapshot freezes the selected Cases only when explicitly created.
Cases staged by an Inbox generation have a separate adoption decision in Inbox. Completing generation alone does not add those Cases or resolve the source observation.
## Success criteria
- The input demonstrates its intended situation without invented assumptions.
- Required materials are inspectable and conform to the captured contract.
- Current membership matches the review decision.
- Repeated or generated Cases retain their source relationship.
## Common failure modes
- Accepting a candidate because its card exists.
- Using count completion as proof of distinct coverage.
- Confusing generated material with approved correctness.
- Treating a Case correction as a mutation of historical Runs.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design](/docs/coverage-design)
- [Input Coverage Inbox](/docs/coverage-design/inbox)
- [Case Foundry](/docs/coverage-design/case-foundry)
- [Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Unclear Cases](/docs/troubleshooting/unclear-cases)
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
{% /related-card-grid %}
---
id: coverage-management.overview
title: Coverage Design
summary: Use Canvas to define required input coverage and Inbox to decide how new findings affect benchmark membership.
kind: concept
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design
---
# Coverage Design
Coverage Design applies reusable project facets to one Benchmark. Use **Canvas** to plan and inspect input coverage, and **Inbox** to decide whether new findings should change the selected Cases.
## Definition
A coverage requirement names an input situation and how it should be represented. It can require a minimum Case count or record an observation. Dimension ontology values, Topics and Scenarios identify the intended slice; distribution settings describe its share of the active benchmark population.
Canvas connects those requirements to actual selected Cases. A large total does not establish that an important combination is present, and unknown classification does not establish absence.
## Decision checkpoint
| Need | Use | Result |
| --- | --- | --- |
| Define the intended behavior space | Canvas requirements and settings | Explicit coverage obligations |
| Inspect a slice or combination | Canvas selection and matrix | Matching Cases, unknowns and remaining need |
| Prepare more Cases | Case preparation | Reviewable candidates and materials |
| Act on an expert or Fieldloop finding | Inbox | An explicit adoption decision |
| Freeze the selected population | Benchmark Datasets → Snapshots | An immutable Benchmark Version |
## Work from Canvas
Describe the Benchmark intent, select the relevant facets and inspect count or distribution requirements. Examine Cases behind a count before deciding that a gap is resolved. Distinguish requirements from observations and current membership from available project Cases.
Stories and their tuples describe meaningful situations used in planning and Case preparation. Case Foundry coordinates preparation; Case Review is where prepared inputs and materials are inspected. These mechanisms support Canvas work rather than replacing its current requirement and membership view.
## Materials and input shape
Project Input Schema owns allowed artifact families, accepted formats and required material keys. Coverage Design does not add another Materials configuration. A tuple can require an allowed artifact but cannot remove a required input or enable an unsupported format.
## Act from Inbox
Inbox links attributable input-coverage findings to possible dataset changes. Review each observation or proposed Case separately. Generating a Case stages another item; it does not automatically add membership. A resolved report can contain declined or withdrawn items.
Existing Snapshots and Runs retain their recorded population. Create a new Snapshot when changed membership or standards should define a later evaluation.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design Settings](/docs/coverage-design/settings)
- [Input Coverage Inbox](/docs/coverage-design/inbox)
- [Coverage Stories](/docs/coverage-design/coverage-stories)
- [Review Prepared Cases](/docs/coverage-design/case-review)
- [Benchmark Datasets](/docs/benchmark-datasets)
- [Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
---
id: coverage-design.inbox
title: Input Coverage Inbox
summary: Review attributable input-coverage findings and decide whether to generate, add, remove, or decline Cases.
kind: task
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design/inbox
---
# Input Coverage Inbox
Use **Coverage Design → Inbox** when an expert or Fieldloop identifies an input situation that may change the Benchmark's selected Cases.
## Prerequisites
- A settled source report delivered to the selected Benchmark.
- Permission to manage its Cases.
- Enough source context to decide whether the input need is relevant.
## Read the source
Contribution reports contain effective expert-confirmed coverage observations and attributable newly synthesized Input Cases after their completion and checkpoint reconciliation. Settled results can also be retained from a cancelled Contribution.
Fieldloop reports contain individually identified input-coverage discoveries tied to published sources and Benchmark references. A correctness-only report or an arbitrary paragraph does not become an Inbox item. Lemon Goal reports remain in their own feature.
## Decide each item
A proposed new Case supports **Add to benchmark** or **Do not adopt**. An observation can lead to generating a new Case, adding existing Cases, removing current members, recording that no dataset change is needed, or declining adoption.
Generation requests one new input under the project contract and stages it as a separate item. Inspect that Case and decide adoption. A failed generation or zero returned Cases leaves the observation open.
## Object and state changes
Adoption changes editable membership and records the decision together. Retries preserve the original outcome. An already-present Case is not added twice.
Items can be pending, in progress, applied, dismissed or withdrawn. Reports show **Needs review** while effective unresolved items remain and **Resolved** when none remain. Resolved does not necessarily mean the dataset changed: declined or withdrawn items can also be settled.
Later attributable results can arrive in the same report. Corrected observations remain distinguishable from an older decision. A dismissed item is not reopened merely by rediscovery.
## Success criteria
- Each action refers to the current source item.
- Generated Cases receive a separate adoption decision.
- Actual membership, rather than report viewing or generation completion, determines whether the dataset changed.
- Historical Snapshots remain unchanged.
## Common failure modes
- Treating a correctness concern as input coverage automatically.
- Assuming generation adds a Case.
- Interpreting Resolved as universal acceptance.
- Repeating a settled decision because a report was refreshed.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design](/docs/coverage-design)
- [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts)
- [Fieldloop](/docs/fieldloop)
- [Benchmark Dataset Cases](/docs/benchmark-datasets/cases)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage)
- [Expert Contribution problems](/docs/troubleshooting/expert-contributions)
{% /related-card-grid %}
---
id: coverage-management.case-foundry
title: Case Foundry
summary: Prepare Case candidates for Coverage Design from current requirements and captured project input settings.
kind: task
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design/case-foundry
---
# Case Foundry
Case Foundry coordinates Case preparation for the situations defined in Coverage Design. Read preparation progress separately from reviewed Case quality and benchmark membership.
## Prerequisites
- Coverage intent and requirements that identify the missing situation.
- Relevant Story or tuple planning context.
- Project Input Schema and the materials needed by that contract.
- Permission to prepare and review Cases.
## Prepare candidates
Start from Canvas and the intended coverage slice. Confirm whether the work should reuse source material, synthesize a situation or refine existing candidates through the available preparation controls.
Preparation captures the relevant planning and input settings. Its queued, running, partial, completed, failed or cancelled state describes that accepted operation. A completed operation is not a statement that every current requirement is now satisfied.
Project Input Schema owns allowed artifact families, formats and required material keys. An unavailable required family produces an actionable preparation error. Do not replace required material with text or change the project contract merely to make generation appear complete.
## Review the result
Open Case Review to inspect input content, generated materials, provenance and the situation demonstrated by each candidate. Read added, synthesized or retrieved counts as preparation activity. Confirm actual selected membership in Canvas or Benchmark Datasets.
If requirements or source material changed after preparation started, inspect the captured inputs before requesting another operation. Historical completion does not establish freshness against today's settings.
## Object and state changes
Preparation creates an attributable operation and reviewable candidate material. Review and explicit membership operations determine which Cases enter the editable benchmark. Existing Snapshots remain unchanged.
## Success criteria
- Candidate preparation is tied to a named input need.
- The captured contract and actual required materials agree.
- Partial or failed work remains distinguishable from successful output.
- Reviewed membership, rather than generation volume, explains progress.
## Common failure modes
- Generating around an unresolved correctness question.
- Treating operation completion as Case approval.
- Using obsolete Benchmark material preferences instead of Project Input Schema.
- Assuming an old preparation handled newly edited requirements.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design](/docs/coverage-design)
- [Review Prepared Cases](/docs/coverage-design/case-review)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Benchmark Dataset Cases](/docs/benchmark-datasets/cases)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Synthetic Cases That Feel Unrealistic](/docs/troubleshooting/unrealistic-synthetic-cases)
- [Unclear Cases](/docs/troubleshooting/unclear-cases)
{% /related-card-grid %}
---
id: project-settings.input-schema
title: Project Input Schema
summary: Define the canonical input architecture, case-material fields, and accepted artifact formats for project cases.
kind: reference
product_area: project_settings
status: stable
updated: 2026-10-30
canonical: /docs/project-settings/input-schema
---
# Project Input Schema
## Definition
Project Input Schema is the project-managed contract for future cases. It declares the primary input architecture, optional structured-input schema, named case-material fields, and artifact families or file extensions the project accepts.
The active architectures are **plain text**, **chat**, and **structured**. The schema is a project singleton rather than a versioned benchmark object. When no setting exists, the default accepts one plain-text user message and no case materials or artifacts.
## Fields, states, or lifecycle rules
- `architecture` is `plain_text`, `chat`, or `structured`.
- Structured architecture requires a bounded `structuredInputSchema`.
- `caseMaterialSchema` is a closed, flat object. Each material key has a label, optional description, required flag, type, and any accepted artifact rules.
- Material keys use lowercase letters, digits, and underscores, start with a letter, and remain flat.
- `acceptedArtifacts` declares project-level artifact families. Supported families are image, document, tabular, presentation, source text, and audio.
- A case-material artifact rule must be a subset of the artifact families admitted at the project root.
- Saving a new schema governs future case validation. Operators should inspect existing cases before making a change that would make current content invalid.
## Canonical case content
The primary case payload is `content.input`. Optional supporting values and artifacts live in `content.case_materials`. The product renders `record_content.case_view` so people and execution adapters can inspect the canonical content consistently; that view is a projection rather than an alternate authoring contract.
Static support passed to a Harness uses `case_material_refs`. An executable or queryable environment uses an optional `world_instance_ref`. Do not collapse static documents, images, or values into the world boundary merely because a candidate consumes them during a Run.
{% example-demo title="Example: structured support case" %}
A project selects structured input with `question` and `customer_tier` properties. It declares a required `policy_document` case material that accepts PDF documents and an optional `account_history` tabular material. A case is valid only when its structured input matches the schema and the required document is present in the accepted format.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Project Settings](/docs/project-settings)
- [Product Quickstart](/docs/quickstart)
- [Cases](/docs/assets/cases)
- [Review Screens](/docs/assets/review-screens)
{% /related-card-grid %}
---
id: troubleshooting.unclear-cases
title: Unclear Cases
summary: Repair Cases that informed reviewers cannot interpret consistently from the supplied input, context, and output identity.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-10-30
canonical: /docs/troubleshooting/unclear-cases
---
# Unclear Cases
## Symptoms
- Reviewers ask what they are supposed to judge.
- The same Case produces incompatible readings because a source, prior turn, time boundary, or user state is absent.
- A candidate answer is visible, but it is unclear whether it is the judged output or reference behavior.
- The Case title or metadata implies context that the review presentation does not contain.
- Experts can answer only by inventing assumptions.
## Likely causes
- A prior conversation turn, source, time boundary, or user state is missing.
- Candidate and reference outputs are swapped or unlabeled.
- One Case combines multiple situations or requested judgments.
- Important instructions live in metadata or private project knowledge instead of visible context.
## Diagnose the Case, not the reviewer
1. Open the Case in its normal review presentation and read only what the assigned expert can see.
2. Identify the judged input and output separately. For conversations, confirm the turn order and which response is under review.
3. List every fact required to make the judgment, then verify each fact is present as Case content, context, source material, or an explicitly linked standard.
4. Check whether metadata is being used as hidden instruction rather than visible context.
5. Compare reviewer rationale. Repeated invented assumptions usually reveal the missing boundary.
## Fix
- Add or map the missing context when it belongs to the real situation.
- Correct the output attachment when candidate and reference behavior were swapped.
- Split a Case that combines two independently judged situations.
- Narrow the question or applicability boundary when multiple interpretations are legitimate.
- Mark the Case unresolved or keep it as candidate material when authoritative supporting information is unavailable.
- Create a new Case version when the reviewable content changes; do not silently rewrite supporting information used by an existing Snapshot.
After correction, have a reviewer who did not author the Case explain the situation and requested judgment. If they must rely on private project knowledge, the Case is still incomplete.
## Prevention
Review the Case through the same presentation an expert will see, label output identity explicitly, and include every fact required for judgment. Sample imported and generated Cases before Benchmark selection, and version content changes after supporting information exists.
{% example-demo title="Example: recommendation without the user's equipment" %}
A Case asks whether a product is compatible but omits the equipment model mentioned in an earlier conversation turn. Reviewers alternate between pass and fail. The operator restores the prior turn, confirms the candidate output identity, versions the Case, and repeats the affected review. The old Snapshot retains the earlier ambiguous version.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Assets](/docs/assets)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Case Versioning](/docs/governance/case-versioning)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Cases](/docs/object-model/cases)
- [Represent conversations in Cases](/docs/object-model/represent-conversations-in-cases)
- [Metadata and context](/docs/reference/metadata-and-context)
{% /related-card-grid %}
---
id: troubleshooting.unrealistic-synthetic-cases
title: Synthetic Cases That Feel Unrealistic
summary: Improve generated candidates whose surface details or behavior assumptions do not represent plausible target-system use.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-10-30
canonical: /docs/troubleshooting/unrealistic-synthetic-cases
---
# Synthetic Cases That Feel Unrealistic
## Symptoms
- Generated language sounds artificial or describes a user who would not exist in the product.
- Required context appears conveniently in the prompt even though the real system would retrieve it elsewhere.
- The Case combines incompatible ontology values or violates product constraints.
- Reviewers can identify the intended coverage tuple but would never admit the Case to a Benchmark.
- Many generated candidates repeat the same template with superficial wording changes.
## Likely causes
- The synthesis request lacks representative source Cases or product constraints.
- A coverage tuple combines values that cannot coexist in the target system.
- A Comparison Direction is too broad, stale, or duplicative.
- Generation is being judged as finished benchmark results instead of candidate material.
## Diagnose the generation boundary
1. Identify the entry surface. **Assets → Cases → Synthesize** creates reusable project Case candidates; **Coverage Design → Case Foundry** coordinates candidates for a named benchmark need.
2. Compare the generated Case with real in-use Cases from the same behavior region.
3. Separate a bad surface realization from a bad coverage target. Awkward wording may need editing; an impossible product state requires different inputs or directions.
4. Check whether the generator lacked constraints that operators hold only in private memory.
5. Inspect the synthesis lifecycle and candidate state. A completed generation is still candidate material, not selected Benchmark Dataset supporting information.
## Fix
- Add representative source Cases or explicit product constraints to the synthesis context.
- Narrow, edit, archive, or delete a Comparison Direction that produces implausible combinations.
- Generate from a specific missing coverage tuple rather than asking for generic diversity.
- Edit a promising candidate when its behavioral boundary is sound and only presentation is weak.
- Reject candidates whose premise is impossible or whose source context cannot be justified.
- Review duplicates before adding any generated Case to a Benchmark.
Validate revised candidates with an expert who knows the target workflow. Ask whether the situation could occur, whether the system would receive the shown context, and whether the requested judgment matters. Only then classify and curate the Case for possible benchmark use.
## Prevention
Ground synthesis in reviewed source Cases, explicit product constraints, and a named coverage gap. Keep Comparison Directions narrow, inspect duplicates, and require Case review before adding generated candidates to a Benchmark or Snapshot.
{% example-demo title="Example: impossible refund request" %}
Synthesis produces a Case where an anonymous visitor requests an enterprise-contract refund and supplies internal approval metadata in chat. The tuple is valuable—exception request with conflicting authority—but the realization is impossible. The team adds authentication and channel constraints, narrows the Comparison Direction, and regenerates. The realistic candidate still goes through Case review before selection.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Synthesize Cases](/docs/coverage-engineering/synthesize-cases)
- [Comparison Directions](/docs/assets/comparison-directions)
- [Boundary Cases](/docs/coverage-engineering/boundary-cases)
- [Case Pool](/docs/coverage-engineering/case-pool)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Case Pool surface](/docs/object-model/case-pool)
- [Cases](/docs/object-model/cases)
- [Coverage dimensions](/docs/object-model/coverage-dimensions)
{% /related-card-grid %}