Teammately Docs
Docs menu

error

Output mapping troubleshooting

Fix external output rows that do not join cleanly to immutable Case IDs in an output-only Run.

Output mapping troubleshooting

Use this when externally produced responses exist but the mapping preview cannot join them cleanly to the Cases in the selected Benchmark Version.

Symptom

Rows are unmatched, duplicated, or mapped to the wrong Case ID, or the wrong column was selected as the response. No output-only Run should be trusted until the mapping preview is correct.

Likely causes

  • External rows use row order or display text instead of immutable Case IDs.
  • The file contains Case IDs from another Benchmark Version or Dataset export.
  • The selected response column or identifier column is wrong.
  • Several candidate output sets were combined without distinct metadata.

Diagnostic checks

  • Compare several affected case IDs with the output source IDs.
  • Check whether the output set has candidate metadata such as model, prompt, retrieval version, or run identity.
  • Confirm the source file was produced from the same immutable Case population.
  • Inspect duplicate, blank, and unknown identifier counts in the mapping preview.

Fix

  • Correct the external file and map using immutable Case IDs rather than row position.
  • Select the intended response column and keep one candidate output set per output-only Run.
  • Create a new output-only Run from the corrected mapping. Existing Runs remain evidence under their original mapping.
  • Repair missing Case context through the Case workflow, not during output mapping; create a new version boundary when Case content changes.

Prevention

  • Preserve source IDs through import and output generation.
  • Sample mapped outputs before expert contribution work and before Evaluation Runs.
  • Keep candidate identity in run metadata.
  • Treat each imported output-only Run as its own evidence record; it is not a saved Harness Version.

Related task pages

Task / Operating ManualImport and prepare casesBring cases into the project, conform them to Project Input Schema, inspect materials, and prepare benchmark selection.Task / Benchmark EvaluationsMap External Evaluation OutputsImport reference outputs, map them to immutable benchmark Cases and attempts, and inspect the resulting output-only Run.Task / Benchmark EvaluationsInspect Evaluation ResultsTrace Dashboard and List signals to Run, Case, Policy, Rubric, completeness, and telemetry evidence.

Related reference pages

Reference / Object ModelOutputsDistinguish managed Run responses, imported output-only Runs, and Case-scoped reference outputs.Reference / Object ModelRepresent conversations in CasesPreserve multi-message context inside canonical Case input without inventing a separate Conversation Case object.Concept / Benchmark EvaluationsBenchmark EvaluationsRun and inspect exact Harness Versions against an immutable Benchmark Version through Dashboard, List, Arena, and Compare.

Source confidence

Code-backed: the current output-mapping modal defines identifier and output-column selection, mapping preview, unmatched rows, metadata, and creation of an output-only Run. The Runs workspace keeps that imported evidence distinct from saved Harness Versions.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Benchmark Evaluations Run a Benchmark Evaluation Launch exact active Harness Versions against an immutable Benchmark Version as standard or repeated Run Groups. Benchmark Evaluations Map External Evaluation Outputs Import reference outputs, map them to immutable benchmark Cases and attempts, and inspect the resulting output-only Run. Benchmark Evaluations Inspect Evaluation Results Trace Dashboard and List signals to Run, Case, Policy, Rubric, completeness, and telemetry evidence.
AI context