# Connect observed behavior to benchmark coverage Generated: 2026-11-13T08:01:22.761Z Source build: local Canonical docs: https://teammately.ai/docs --- id: workflow.fieldloop.benchmark-coverage title: Connect observed behavior to benchmark coverage summary: Use investigated application behavior to decide which situations need coverage or expert follow-up. kind: task product_area: fieldloop status: stable updated: 2026-11-11 canonical: /docs/fieldloop/benchmark-coverage --- # Connect observed behavior to benchmark coverage Use investigated application behavior to decide which situations need coverage or expert follow-up. ## Prerequisites An accessible Fieldloop Thread with captured sources and an identified coverage question. ## Steps 1. Read the report, original records, contrary examples, and relevant project objects. 2. Distinguish an input-coverage discovery from a correctness claim or an unsupported report paragraph. 3. Inspect supported coverage findings in Coverage Design Inbox. Review the situation and requested preparation before accepting follow-up. 4. Inspect prepared Cases and adopt dataset changes through the normal review path. Request expert judgment separately when expected behavior is unresolved. ## Object and state changes Reading a Thread changes no dataset. A coverage discovery, accepted preparation, reviewed Case, and dataset adoption retain separate states. ## Success criteria The resulting benchmark change has a traceable coverage reason and an explicit adoption decision. ## Common failure modes Treating recent traffic as the full target population; treating report prose as an approved standard or automatic dataset mutation. ## Related reference pages {% related-card-grid title="Continue this workflow" %} - [Inspect Logs and Threads](/docs/fieldloop/threads) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Resolve a problem" %} - [Fieldloop Intake and Processing](/docs/troubleshooting/fieldloop) - [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage) {% /related-card-grid %} --- id: fieldloop.threads title: Inspect Logs and Threads summary: Inspect investigated findings, citations, contrary examples, and focused follow-up work. kind: task product_area: fieldloop status: stable updated: 2026-11-11 canonical: /docs/fieldloop/threads --- # Inspect Logs and Threads A Thread is an inquiry grounded in captured application records. Read the report together with its cited source material and related project objects. ## Prerequisites - Permission to inspect the Thread and its original records. - The intended Project and source scope. - Enough retrieved context to distinguish a hypothesis from an established standard. ## Inspect the report 1. Open Fieldloop → Threads and select the inquiry. 2. Read the current report and its recorded history. 3. Follow citations to their original captured fields and versions. 4. Inspect relevant Cases, Policies or Rubrics where the report relies on them. 5. Preserve contrary examples, uncertainty and missing source context. 6. Choose an explicit follow-up only when the retrieved material supports it. A negative feedback signal alone does not overturn an expert decision. A possible model departure still needs an applicable standard and actual behavior. A source that was not retrieved cannot be inferred from a convincing summary. ## Follow up Prepared actions can route to coverage inspection, further investigation or a Contribution request. Review the destination and requested work before accepting it. Typed input-coverage discoveries can appear in Coverage Design Inbox. Correctness-only findings and arbitrary report paragraphs do not enter that flow. A prepared coverage draft and a dataset adoption decision remain separate. ## Object and state changes Reading a Thread is read-only. An accepted follow-up creates the owning work through its normal workflow. It does not automatically approve a Policy, contact an expert or add/remove Cases. A later publication can update the same inquiry while retaining source provenance and contrary material. Read the published history instead of assuming the newest report contains every earlier source in its opening summary. ## Success criteria - Important conclusions resolve to captured sources and relevant standards. - Interpretations remain distinguishable from the captured facts. - Any follow-up has a clear scope and owner. - Dataset changes receive explicit adoption decisions. ## Common failure modes - Treating report prose as governance approval. - Inferring population completeness from selected records. - Creating a Contribution without checking its intended expert and objective. - Calling an Inbox report resolved merely because generation finished. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Fieldloop](/docs/fieldloop) - [Input Coverage Inbox](/docs/coverage-design/inbox) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Fieldloop Intake and Processing](/docs/troubleshooting/fieldloop) - [Unbalanced Coverage](/docs/troubleshooting/unbalanced-coverage) {% /related-card-grid %} --- id: troubleshooting.fieldloop title: Fieldloop Intake and Processing summary: Resolve source, destination, intake, processing, and Thread issues while preserving captured records. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/fieldloop --- # Fieldloop Intake and Processing ## Symptom A submission has no expected log, a batch reports rejected or conflicting rows, processing is waiting, or captured records have not produced an expected Thread. ## Likely causes - The credential, Project or source destination does not match. - The original submitter's receipt is being read from another principal. - A retry changed content while keeping the same idempotency identity. - Intake is admitted but row processing is not complete. - Source processing consent is absent or revoked. - Provider capacity or a recorded processing failure delayed analysis. - The selected records do not support an inquiry. ## Fix Inspect the original receipt and its row counts first. Confirm the workspace, Project, source and original logical request. Retry an uncertain submission only with its exact content and key. Read Logs for captured content and processing state. Check the source's current permission and the reported error through the normal controls. Keep accepted records and their provenance intact. A missing Thread is not automatically an intake failure. Analysis may legitimately publish no inquiry. Do not invent a report, change source facts or broaden provider access to force a visible result. If a typed input-coverage discovery was published, inspect its related Inbox item separately. Report publication, Case generation and dataset adoption have different outcomes. ## Prevention Retain original source records until admission is known. Keep credentials in the host secret mechanism, preserve exact retry identity and monitor row outcomes rather than HTTP success alone. Configure source permission deliberately. Revoke processing through its owning control when needed; do not treat an access change as authorization to rewrite published history. ## Related task pages {% related-card-grid title="Related task pages" %} - [Fieldloop](/docs/fieldloop) - [Capture and Inspect Logs](/docs/fieldloop/ingest-logs) - [Read Fieldloop Threads](/docs/fieldloop/threads) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [API Keys and Delegated Access](/docs/admin-console/api-keys) - [Input Coverage Inbox](/docs/coverage-design/inbox) {% /related-card-grid %} --- id: troubleshooting.unbalanced-coverage title: Unbalanced Coverage summary: Fix benchmarks that overrepresent easy or common cases while missing important behavior. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/unbalanced-coverage --- # Unbalanced Coverage Use this when the Benchmark Dataset overrepresents some behavior and misses other behavior that matters before human review. ## Symptom A benchmark looks strong in aggregate but reviewers or product owners identify important behavior areas with too few cases. Results may be dominated by common easy cases while rare or boundary cases are absent. ## Likely causes - Case import mirrored production volume instead of product risk. - Dimensions or metadata do not represent the behavior areas that matter. - Synthetic cases were added in one area but not reviewed across the full risk boundary. - Coverage was not refreshed after new signals, policy changes, or failures. ## Diagnostic checks - Inspect **Benchmark Datasets → Representation** and the Coverage Design overview by the configured Coverage Facets. - Look for policies or rubrics with very few applicable cases. - Compare production or expert signals with benchmark case distribution. - Inspect whether new cases are candidate-only or actually in use. ## Fix - Add representative, boundary, or adversarial cases for undercovered areas. - Refresh dimensions or metadata before judging balance. - Review and promote candidate cases into the benchmark only after context is complete. - Record remaining gaps as Coverage Stories or explicit benchmark coverage guidance. ## Prevention - Define coverage targets before benchmark runs. - Track candidate and in-use cases separately. - Review coverage after policy, rubric, or product behavior changes. - Do not use aggregate scores without a coverage note. ## Related task pages {% related-card-grid title="Related task pages" %} - [Dataset Representation](/docs/benchmark-datasets/representation) - [Coverage Design Settings](/docs/coverage-design/settings) - [Coverage Stories](/docs/coverage-design/coverage-stories) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Case Pool surface](/docs/object-model/case-pool) - [Coverage dimensions](/docs/object-model/coverage-dimensions) - [Benchmarks](/docs/object-model/benchmarks) {% /related-card-grid %}