# Runs, missing outputs, and output mapping
Generated: 2026-11-13T08:01:22.578Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: workflow.troubleshooting.execution
title: Runs, missing outputs, and output mapping
summary: An experiment cannot start, has incomplete behavior, or contains unmatched external responses.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-11
canonical: /docs/troubleshooting/execution
---
# Runs, missing outputs, and output mapping
## Symptom
An experiment cannot start, has incomplete behavior, or contains unmatched external responses.
## Likely causes
| What you observe | Focused guide |
| --- | --- |
| Managed execution or evaluation progress is blocked | [Benchmark run troubleshooting](/docs/troubleshooting/benchmark-runs) |
| A response or recorded failure is absent | [Missing Outputs](/docs/troubleshooting/missing-outputs) |
| Imported rows do not match the exact Case manifest | [Output mapping troubleshooting](/docs/troubleshooting/output-mapping) |
## Fix
Identify the Run, candidate, Benchmark Version, and execution path. Separate infrastructure failure, candidate failure, missing submission, mapping rejection, and evaluation completeness before interpreting scores.
Follow the focused guide for the actual state you observe; keep accepted records and explicit human decisions intact.
## Prevention
Preserve exact object and version identities, review the relevant source context, and inspect the accepted state before comparing outcomes.
## Related task pages
{% related-card-grid title="Return to the workflow" %}
- [Complete your first alignment loop](/docs/quickstart)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Diagnose the observed state" %}
- [Benchmark run troubleshooting](/docs/troubleshooting/benchmark-runs)
- [Missing Outputs](/docs/troubleshooting/missing-outputs)
- [Output mapping troubleshooting](/docs/troubleshooting/output-mapping)
{% /related-card-grid %}
---
id: troubleshooting.benchmark-runs
title: Benchmark run troubleshooting
summary: Diagnose a Run that cannot start, has no usable outputs, or produces results that cannot be compared safely.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/benchmark-runs
---
# Benchmark run troubleshooting
## Symptoms
- The evaluation surface has no Benchmark Version to run.
- Output import or mapping cannot identify a Case or output column.
- A Run is created but remains empty, incomplete, or failed.
- Results appear under the wrong candidate label or Harness Version.
- Two Runs show a score change but do not share a comparable snapshot boundary.
## Likely causes
- No immutable Snapshot exists for the intended evaluation set.
- Imported outputs are mapped to the wrong Case or column.
- The saved Harness Version or candidate metadata does not match the evaluated system.
- The Run is partial, failed, or being compared across different Benchmark Versions.
## Check the boundary before the failure
1. Confirm the URL and page identify the intended Project, Benchmark, and Benchmark Version.
2. Open the Version or Snapshot and verify it contains the expected Cases and evaluators. If it does not, repair coverage and create a new Snapshot; do not edit the historical Run.
3. For managed execution, confirm the saved Harness Version and recorded settings. For imported results, confirm that the external candidate context describes the system that produced the submitted responses.
4. If importing outputs, inspect the mapping preview. Match the Case identifier and candidate-output column deliberately; do not use a reference-output column as candidate behavior.
5. Open Run detail and inspect status, Case count, errors, metadata, and per-Case results before trusting aggregates.
## Fix
- **No runnable version:** finish Case selection and create a Snapshot first.
- **Missing imported outputs:** append only missing Cases to an uncompleted Trial using the exact manifest IDs. Correcting an accepted response or adding Cases after completion requires a new Trial in the external Run or a new Run. Preserve the original results.
- **Wrong Harness or metadata:** create a correctly configured Run. Do not relabel completed results to represent a different system.
- **Partial managed failure:** inspect and preserve the existing Run's recorded outcomes. For another independent execution under unchanged conditions, use **Run again**. **Run evaluation** reopens matching work, including failed work. Publish a new Harness Version when its code changes, or create a new Snapshot and Benchmark Version when Case content or criteria change.
- **Confusing comparison:** compare the Benchmark Version, Harness Version, candidate metadata, and evaluator boundary. Qualify or avoid the comparison when more than the intended variable changed.
## Prevention
Create a Snapshot before evaluation. For managed execution, save the exact Harness Version. For imported results, preserve external candidate context and preview the mapping on representative Cases. Before comparing, confirm that every difference between the two Runs is intentional and visible.
## Recovery check
Open the recovered Run and sample several Case results. Confirm the displayed input, candidate output, reference output where present, applicable Rubrics, and metadata all belong together. A completed status alone does not prove correct mapping.
{% example-demo title="Example: scores drop after output import" %}
A team imports a new candidate file and sees a sudden score collapse. Run detail shows that the column containing reference outputs was mapped as candidate output. The operator creates a new Run, maps the actual candidate column, preserves the mistaken Run as an identifiable failed attempt, and compares only the corrected Run with the prior candidate under the same Benchmark Version.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related workflows" %}
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Import External Evaluation Outputs](/docs/benchmark-evaluations/output-mapping)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Run Details and Metadata](/docs/benchmark-evaluations/run-details)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference" %}
- [Benchmarks and versions](/docs/concepts/benchmarks-and-versions)
- [Benchmark snapshots](/docs/coverage-engineering/benchmark-snapshots)
- [Outputs](/docs/object-model/outputs)
{% /related-card-grid %}
---
id: troubleshooting.missing-outputs
title: Missing Outputs
summary: Separate a missing managed Run response from an unmatched row in an imported output-only Run.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/missing-outputs
---
# Missing Outputs
Use this when an Evaluation Run has no response for an intended Case. First identify whether the response was expected from managed Harness execution or an imported output-only Run.
## Symptom
A managed Run is incomplete or has a failed or unevaluable Case attempt, or an imported output-only Run shows an unmatched Case row. An input-only Case by itself is not an error: managed execution is expected to generate its response.
## Likely causes
- Managed execution failed, stopped, or could not evaluate the Case.
- The wrong saved Harness Version or execution settings were selected.
- An external output row has a missing, unknown, or duplicate immutable Case ID.
- The imported file omitted Cases that the output-only Run was intended to cover.
## Diagnostic checks
- Open the Run and check its lifecycle, planned Trials, and failed Cases. For managed execution, confirm the selected Harness Version; for imported results, confirm the external candidate context.
- Confirm that the Case belongs to the immutable Benchmark Version used by the Run.
- For imported outputs, compare external identifiers with the immutable Case IDs shown by the mapping surface.
- Confirm that the intended output column and candidate metadata were selected before import.
## Fix
- For managed execution under unchanged conditions, use **Run again** when an independent execution is intended. **Run evaluation** reopens matching existing work, including a failed Run.
- If the Harness changes, publish the corrected saved Version and select it through **Run evaluation**. Preserve the intended Benchmark Version and review the resulting execution conditions.
- For imported outputs, append only missing Cases to an uncompleted Trial using immutable Case IDs. Correcting an accepted response or adding Cases after completion requires a new Trial in the external Run or a new Run. Preserve the original results and explicitly confirm partial completion when Cases remain absent.
- Keep managed and imported-output diagnoses separate; an imported response does not become a saved Harness Version.
- Change the current Dataset and create a new Snapshot only when Case membership itself is wrong—not merely because a Run response is missing.
A repeated Run retains its own history and does not silently replace the Run marked **Used for score**. See [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation#reuse-and-deliberate-repeats) for reuse and deliberate repetition.
## Prevention
- Save the exact Harness Version before managed execution and review execution settings.
- Export or preserve immutable Case IDs through external generation.
- Validate several mapped rows before creating an output-only Run.
- Use Run Metadata to identify the candidate or external output set without pretending it is a Harness Version.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Import and prepare cases](/docs/operating-manual/import-and-prepare-cases)
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Import External Evaluation Outputs](/docs/benchmark-evaluations/output-mapping)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Outputs](/docs/object-model/outputs)
- [Cases](/docs/assets/cases)
- [Benchmarks](/docs/object-model/benchmarks)
{% /related-card-grid %}
---
id: troubleshooting.output-mapping
title: Output mapping troubleshooting
summary: Fix external output rows that do not join cleanly to immutable Case IDs in an output-only Run.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-11-06
canonical: /docs/troubleshooting/output-mapping
---
# Output mapping troubleshooting
Use this when externally produced responses exist but the mapping preview cannot join them cleanly to the Cases in the selected Benchmark Version.
## Symptom
Rows are unmatched, duplicated, or mapped to the wrong Case ID, or the wrong column was selected as the response. No output-only Run should be trusted until the mapping preview is correct.
## Likely causes
- External rows use row order or display text instead of immutable Case IDs.
- The file contains Case IDs from another Benchmark Version or Dataset export.
- The selected response column or identifier column is wrong.
- Several candidate output sets were combined without distinct metadata.
## Diagnostic checks
- Compare several affected case IDs with the output source IDs.
- Check whether the output set has candidate metadata such as model, prompt, retrieval version, or run identity.
- Confirm the source file was produced from the same immutable Case population.
- Inspect duplicate, blank, and unknown identifier counts in the mapping preview.
## Fix
- Correct the external file and map using immutable Case IDs rather than row position.
- Select the intended response column and keep one candidate output set per output-only Run.
- Create a new output-only Run from the corrected mapping. Existing Runs retain their original mapping.
- Repair missing Case context through the Case workflow, not during output mapping; create a new version boundary when Case content changes.
## Prevention
- Preserve source IDs through import and output generation.
- Sample mapped outputs before expert contribution work and before Evaluation Runs.
- Keep candidate identity in run metadata.
- Treat each imported output-only Run as a record of the submitted responses; it is not a saved Harness Version.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Import and prepare cases](/docs/operating-manual/import-and-prepare-cases)
- [Import External Evaluation Outputs](/docs/benchmark-evaluations/output-mapping)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Outputs](/docs/object-model/outputs)
- [Represent conversations in Cases](/docs/object-model/represent-conversations-in-cases)
- [Benchmark Evaluations](/docs/benchmark-evaluations)
{% /related-card-grid %}