# Input Coverage Inbox
Generated: 2026-11-03T08:09:27.640Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: coverage-design.inbox
title: Input Coverage Inbox
summary: Review attributable input-coverage findings and decide whether to generate, add, remove, or decline Cases.
kind: task
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design/inbox
---
# Input Coverage Inbox
Use **Coverage Design → Inbox** when an expert or Fieldloop identifies an input situation that may change the Benchmark's selected Cases.
## Prerequisites
- A settled source report delivered to the selected Benchmark.
- Permission to manage its Cases.
- Enough source context to decide whether the input need is relevant.
## Read the source
Contribution reports contain effective expert-confirmed coverage observations and attributable newly synthesized Input Cases after their completion and checkpoint reconciliation. Settled results can also be retained from a cancelled Contribution.
Fieldloop reports contain individually identified input-coverage discoveries tied to published sources and Benchmark references. A correctness-only report or an arbitrary paragraph does not become an Inbox item. Lemon Goal reports remain in their own feature.
## Decide each item
A proposed new Case supports **Add to benchmark** or **Do not adopt**. An observation can lead to generating a new Case, adding existing Cases, removing current members, recording that no dataset change is needed, or declining adoption.
Generation requests one new input under the project contract and stages it as a separate item. Inspect that Case and decide adoption. A failed generation or zero returned Cases leaves the observation open.
## Object and state changes
Adoption changes editable membership and records the decision together. Retries preserve the original outcome. An already-present Case is not added twice.
Items can be pending, in progress, applied, dismissed or withdrawn. Reports show **Needs review** while effective unresolved items remain and **Resolved** when none remain. Resolved does not necessarily mean the dataset changed: declined or withdrawn items can also be settled.
Later attributable results can arrive in the same report. Corrected observations remain distinguishable from an older decision. A dismissed item is not reopened merely by rediscovery.
## Success criteria
- Each action refers to the current source item.
- Generated Cases receive a separate adoption decision.
- Actual membership, rather than report viewing or generation completion, determines whether the dataset changed.
- Historical Snapshots remain unchanged.
## Common failure modes
- Treating a correctness concern as input coverage automatically.
- Assuming generation adds a Case.
- Interpreting Resolved as universal acceptance.
- Repeating a settled decision because a report was refreshed.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design](/docs/coverage-design)
- [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts)
- [Fieldloop](/docs/fieldloop)
- [Benchmark Dataset Cases](/docs/benchmark-datasets/cases)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage)
- [Expert Contribution problems](/docs/troubleshooting/expert-contributions)
{% /related-card-grid %}
---
id: coverage-management.overview
title: Coverage Design
summary: Use Canvas to define required input coverage and Inbox to decide how new findings affect benchmark membership.
kind: concept
product_area: coverage_management
status: stable
updated: 2026-10-30
canonical: /docs/coverage-design
---
# Coverage Design
Coverage Design applies reusable project facets to one Benchmark. Use **Canvas** to plan and inspect input coverage, and **Inbox** to decide whether new findings should change the selected Cases.
## Definition
A coverage requirement names an input situation and how it should be represented. It can require a minimum Case count or record an observation. Dimension ontology values, Topics and Scenarios identify the intended slice; distribution settings describe its share of the active benchmark population.
Canvas connects those requirements to actual selected Cases. A large total does not establish that an important combination is present, and unknown classification does not establish absence.
## Decision checkpoint
| Need | Use | Result |
| --- | --- | --- |
| Define the intended behavior space | Canvas requirements and settings | Explicit coverage obligations |
| Inspect a slice or combination | Canvas selection and matrix | Matching Cases, unknowns and remaining need |
| Prepare more Cases | Case preparation | Reviewable candidates and materials |
| Act on an expert or Fieldloop finding | Inbox | An explicit adoption decision |
| Freeze the selected population | Benchmark Datasets → Snapshots | An immutable Benchmark Version |
## Work from Canvas
Describe the Benchmark intent, select the relevant facets and inspect count or distribution requirements. Examine Cases behind a count before deciding that a gap is resolved. Distinguish requirements from observations and current membership from available project Cases.
Stories and their tuples describe meaningful situations used in planning and Case preparation. Case Foundry coordinates preparation; Case Review is where prepared inputs and materials are inspected. These mechanisms support Canvas work rather than replacing its current requirement and membership view.
## Materials and input shape
Project Input Schema owns allowed artifact families, accepted formats and required material keys. Coverage Design does not add another Materials configuration. A tuple can require an allowed artifact but cannot remove a required input or enable an unsupported format.
## Act from Inbox
Inbox links attributable input-coverage findings to possible dataset changes. Review each observation or proposed Case separately. Generating a Case stages another item; it does not automatically add membership. A resolved report can contain declined or withdrawn items.
Existing Snapshots and Runs retain their recorded population. Create a new Snapshot when changed membership or standards should define a later evaluation.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Design Settings](/docs/coverage-design/settings)
- [Input Coverage Inbox](/docs/coverage-design/inbox)
- [Coverage Stories](/docs/coverage-design/coverage-stories)
- [Review Prepared Cases](/docs/coverage-design/case-review)
- [Benchmark Datasets](/docs/benchmark-datasets)
- [Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
---
id: expert-contributions.artifacts
title: Contributed Artifacts
summary: Inspect Policy and Rubric contributions and coverage observations while retaining expert provenance and separate adoption decisions.
kind: reference
product_area: expert_contributions
status: stable
updated: 2026-10-30
canonical: /docs/expert-contributions/contributed-artifacts
---
# Contributed Artifacts
## Definition
Contributed Artifacts presents specialist outputs in **By Policy**, **By Rubric** and **Observations** views. Use it to inspect attributable material and its relationship to the Contribution that produced it.
## Fields, states, or lifecycle rules
- Policy and Rubric records preserve the expert context and artifact decision.
- Coverage observations describe input situations that need attention.
- Contribution completion and artifact approval are separate.
- An observation's confirmation is distinct from applying a dataset change.
- Original task, source and checkpoint relationships remain attributable.
- Current artifact ownership stays with Correctness Governance, Assets or Coverage Design.
## Read an artifact
Begin with its actual content and lifecycle state. Follow the Contribution and relevant responses to understand what was asked and what the expert decided.
A draft proposal, retained candidate and approved standard have different meanings. Comments and interviews can inform a decision without establishing approval of every related object.
Keep a changed applicability boundary distinguishable from an unchanged policy statement. Similar wording does not establish that two artifacts cover the same situation.
## Act on observations
Effective expert-confirmed input-coverage observations and attributable new Input Cases can reach Coverage Design Inbox after the source settles. Inbox owns the later generate/add/remove/no-change/decline decision.
Generating a Case does not adopt it automatically. Likewise, a report can become Resolved through a declined or withdrawn item without changing the dataset.
## Preserve historical interpretation
A current Policy or Rubric can evolve after a Contribution. Existing Benchmark Versions and Runs retain the standard and population they accepted. Correct current material through its owning workflow rather than rewriting an earlier expert answer or completed Run.
Use provenance to explain the decision, not to infer authority from activity volume or a completed status alone.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Contribution Lifecycle and Status](/docs/expert-contributions/lifecycle-and-status)
- [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Input Coverage Inbox](/docs/coverage-design/inbox)
{% /related-card-grid %}
---
id: fieldloop.overview
title: Fieldloop
summary: Connect captured application activity to source-grounded Threads and explicit coverage or expert follow-up.
kind: concept
product_area: fieldloop
status: stable
updated: 2026-10-30
canonical: /docs/fieldloop
---
# Fieldloop
Fieldloop connects captured application activity to questions about correctness and input coverage. Open **Fieldloop → Logs** to inspect captured records and **Threads** to read the resulting inquiries.
## Definition
A source describes where logs come from and how they may be processed. A log preserves captured input, output, messages, trajectories, actions, context or feedback. A Thread presents an inquiry grounded in selected source records and the relevant project context.
A captured log is not an approved standard. A report can identify an issue worth investigating without establishing that a candidate violated an applicable Policy or that every production request was represented.
## Decision checkpoint
| Need | Open | Boundary |
| --- | --- | --- |
| Inspect what was captured | Logs | Original content and source identity |
| Manage a source or inspect intake | Source controls in Logs | Destination and processing permission |
| Read an inquiry | Threads | Report, cited sources and history |
| Act on a missing input situation | Coverage Design → Inbox | Separate adoption decision |
| Ask a specialist to resolve meaning | Expert Contributions | Explicit request and expert judgment |
## Processing and source access
Capture and durable intake precede analysis. Automatic discovery uses the source's explicit processing permission and the selected pipeline. Revoking processing permission stops further screening; it does not convert old reports into different historical facts.
The backend owns model routing. Source settings and task data do not authorize a provider switch or override platform model selection.
## Read Threads
Inspect cited source records, relevant Cases and retrieved standards. A report can retain uncertainty, contrary examples and unresolved questions. Read the exact published material behind a citation rather than interpreting fluent prose as approval.
Coverage and Contribution handoffs remain explicit user choices. Fieldloop does not automatically contact experts, approve Policies or change benchmark membership.
## Preserve the source boundary
Published logs and reports remain attributable to their original captured content. User-requested deletion and background staging cleanup are separate operations. Do not infer a general retention or erasure guarantee from a processing status.
Use Fieldloop findings alongside deliberate benchmark construction. Capture volume alone does not prove production completeness, balanced coverage or correctness.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Capture and Inspect Logs](/docs/fieldloop/ingest-logs)
- [Read Fieldloop Threads](/docs/fieldloop/threads)
- [Input Coverage Inbox](/docs/coverage-design/inbox)
- [API Keys and Delegated Access](/docs/admin-console/api-keys)
- [Integrations](/docs/integrations)
{% /related-card-grid %}
---
id: benchmark-datasets.cases
title: Benchmark Dataset Cases
summary: Inspect benchmark Case membership, coverage traces, references, and scoped bulk actions.
kind: task
product_area: benchmark_datasets
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-datasets/cases
---
# Benchmark Dataset Cases
## Prerequisites
- A selected benchmark and permission to inspect or manage its current dataset.
- Project Cases that conform to the intended Input Schema.
The Cases tab is the benchmark-scoped view of the current editable case set. It shows Case content and membership together with coverage trace, output or reference mapping, and evaluator relationships.
Select one or more rows to request an Expert Contribution, create another benchmark from the selection, remove the Cases from the current benchmark, or download them. Removal changes current membership; it does not delete the reusable Case from project Assets or mutate an existing Snapshot.
## Review before snapshotting
1. Confirm each Case still conforms to Project Input Schema and has the intended materials.
2. Inspect Coverage Facet assignments and source or contributor provenance.
3. Check policy and rubric application, including whether eligible evaluator links are approved.
4. Resolve missing or ambiguous output/reference mapping when the workflow requires reference outputs.
5. Use Representation to check whether the set supports the intended claim.
> Membership is not a frozen evaluation
>
> The editable Cases tab can change. Use a Dataset Snapshot and Benchmark Version when an evaluation, comparison, or Improvement Session must remain reproducible.
## After changing membership
Open Representation and confirm that the change affected the intended facet or evaluator population. Removing redundant Cases can improve balance even when total Case count falls. Adding many near-duplicates can increase count without adding meaningful coverage.
If a selected Case needs content correction, edit it through the owning Case workflow and review every future benchmark that selects it. Existing Snapshots stay unchanged. If the Case reveals an unclear standard, request an Expert Contribution before compensating with more examples.
{% example-demo title="Example: scoped bulk action" %}
An operator selects five Cases tied to an unresolved exception and requests one Expert Contribution. The Cases remain in the current set while the expert works. After the controlling Rubric is clarified, the team reviews membership and creates a new Snapshot with the approved evaluator links.
{% /example-demo %}
## Object and state changes
Selected-row removal changes current benchmark membership; creating another benchmark creates a separate benchmark; requesting a Contribution creates scoped expert work. Downloads and inspection are read-only. No action here mutates an existing Snapshot.
## Success criteria
- Current membership, Case identity, coverage trace, and evaluator relationships are understood.
- Any bulk action affects only the intended selected Cases.
- A new Snapshot is created when changed membership must define a fixed population for future evaluation.
## Common failure modes
- Treating removal from the benchmark as project-level Case deletion.
- Assuming editable membership changed an old Benchmark Version.
- Selecting Cases by visible text while ignoring their durable IDs.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Benchmark Datasets](/docs/benchmark-datasets)
- [Cases](/docs/assets/cases)
- [Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Case upload troubleshooting](/docs/troubleshooting/dataset-upload)
- [Unclear Cases](/docs/troubleshooting/unclear-cases)
- [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage)
{% /related-card-grid %}
---
id: troubleshooting.unbalanced-coverage
title: Unbalanced Coverage
summary: Fix benchmarks that overrepresent easy or common cases while missing important behavior.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-10-30
canonical: /docs/troubleshooting/unbalanced-coverage
---
# Unbalanced Coverage
Use this when benchmark results overrepresents some behavior and misses other behavior that matters before human review.
## Symptom
A benchmark looks strong in aggregate but reviewers or product owners identify important behavior areas with too few cases. Results may be dominated by common easy cases while rare or boundary cases are absent.
## Likely causes
- Case import mirrored production volume instead of product risk.
- Dimensions or metadata do not represent the behavior areas that matter.
- Synthetic cases were added in one area but not reviewed across the full risk boundary.
- Coverage was not refreshed after new signals, policy changes, or failures.
## Diagnostic checks
- Inspect **Benchmark Datasets → Representation** and the Coverage Design overview by the configured Coverage Facets.
- Look for policies or rubrics with very few applicable cases.
- Compare production or expert signals with benchmark case distribution.
- Inspect whether new cases are candidate-only or actually in use.
## Fix
- Add representative, boundary, or adversarial cases for undercovered areas.
- Refresh dimensions or metadata before judging balance.
- Review and promote candidate cases into the benchmark only after context is complete.
- Record remaining gaps as Coverage Stories or explicit benchmark coverage guidance.
## Prevention
- Define coverage targets before benchmark runs.
- Track candidate and in-use cases separately.
- Review coverage after policy, rubric, or product behavior changes.
- Do not use aggregate scores without a coverage note.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Dataset Representation](/docs/benchmark-datasets/representation)
- [Coverage Design Settings](/docs/coverage-design/settings)
- [Coverage Stories](/docs/coverage-design/coverage-stories)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Case Pool surface](/docs/object-model/case-pool)
- [Coverage dimensions](/docs/object-model/coverage-dimensions)
- [Benchmarks](/docs/object-model/benchmarks)
{% /related-card-grid %}
---
id: troubleshooting.expert-contributions
title: Expert Contribution problems
summary: Diagnose contribution access, task routing, checkpoint, synchronization, completion, and artifact-reconciliation problems.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-10-30
canonical: /docs/troubleshooting/expert-contributions
---
# Expert Contribution problems
## Symptom
An expert cannot open the Contribution, reaches the wrong task, sees stale work, cannot submit a response, remains at a checkpoint or waiting state, or completes the experience without the administrator seeing reconciled supporting information.
## Likely causes
- The expert lacks access or opened a link for a different Contribution.
- The task handoff changed and the browser still points to an earlier route.
- A form, chat, interview, or case-review command has not synchronized.
- The task lease, realtime connection, or retry state needs recovery.
- A checkpoint still requires confirmation or correction.
- The expert experience is complete, but contributed artifacts have not yet been reconciled in the administrator workspace.
- The Contribution was cancelled or changed while the expert was working.
## Fix
1. Confirm the Contribution identity and expert assignment in **Expert Contributions → Contributions**.
2. Inspect its current status and task plan rather than relying on the expert's previous URL.
3. Distinguish an outdated bookmark from pending answer delivery. Use the current Contribution entry for an outdated bookmark only after staged and unsent input is protected.
4. Ordinary submission, activation and synchronization recover in the background. Preserve exact pending input; do not repeatedly submit, reload or navigate to make delivery progress. A genuine takeover or revoked assignment remains a separate authority decision.
5. Inspect checkpoints for an unconfirmed or corrected statement.
6. Use **Logs & Status** to distinguish expert completion from reconciliation or artifact-materialization work.
7. Check **Contributed Artifacts** and the owning Correctness Governance, Assets, or Coverage Design surface before concluding that accepted material is missing.
## Prevention
- Keep Contribution objectives and task sequences narrow.
- Verify expert access before sending the request.
- Preserve checkpoints for consequential meaning.
- Use the current Contribution entry route instead of bookmarking an internal task route.
- Teach administrators that task completion, supporting information reconciliation, and artifact governance are separate transitions.
- Avoid submitting the same response repeatedly when the product shows a pending or retryable synchronization state.
{% example-demo title="Example: completed task but missing policy" %}
An expert completes the final checkpoint, but the policy does not yet appear in Correctness Governance. Logs & Status shows that the Contribution is complete while supporting information reconciliation remains active. The administrator waits for reconciliation, then inspects Contributed Artifacts and materializes the policy through its owning workflow instead of asking the expert to repeat the task.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Product Quickstart](/docs/quickstart)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Expert Contributions](/docs/expert-contributions)
- [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts)
- [Permissions](/docs/reference/permissions)
{% /related-card-grid %}