---
id: troubleshooting.benchmark-runs
title: Benchmark run troubleshooting
summary: Diagnose a Run that cannot start, has no usable outputs, or produces results that cannot be compared safely.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-09-07
canonical: /docs/troubleshooting/benchmark-runs
---

# Benchmark run troubleshooting

## Symptoms

- The evaluation surface has no Benchmark Version to run.
- Output import or mapping cannot identify a Case or output column.
- A Run is created but remains empty, incomplete, or failed.
- Results appear under the wrong candidate label or Harness Version.
- Two Runs show a score change but do not share a comparable evidence boundary.

## Likely causes

- No immutable Snapshot exists for the intended evidence set.
- Imported outputs are mapped to the wrong Case or column.
- The saved Harness Version or candidate metadata does not match the evaluated system.
- The Run is partial, failed, or being compared across different Benchmark Versions.

## Check the boundary before the failure

1. Confirm the URL and page identify the intended Project, Benchmark, and Benchmark Version.
2. Open the Version or Snapshot and verify it contains the expected Cases and evaluators. If it does not, repair coverage and create a new Snapshot; do not edit the historical Run.
3. In Run setup, confirm the saved Harness Version and candidate metadata describe the system that produced the outputs.
4. If importing outputs, inspect the mapping preview. Match the Case identifier and candidate-output column deliberately; do not use a reference-output column as candidate behavior.
5. Open Run detail and inspect status, Case count, errors, metadata, and per-Case results before trusting aggregates.

## Fix

- **No runnable version:** finish Case selection and create a Snapshot first.
- **No mapped outputs:** correct Case identifiers or column mapping, then submit again under the intended Run.
- **Wrong Harness or metadata:** create a correctly configured Run. Do not relabel completed evidence to represent a different system.
- **Partial failure:** preserve successful per-Case evidence when the product does, correct the failed input or execution boundary, and rerun using a clearly named attempt.
- **Confusing comparison:** compare the Benchmark Version, Harness Version, candidate metadata, and evaluator boundary. Qualify or avoid the comparison when more than the intended variable changed.

## Prevention

Create a Snapshot before execution, save the exact Harness Version, preview output mapping on representative Cases, and name candidate metadata consistently. Before comparing, confirm that every difference between the two Runs is intentional and visible.

## Recovery check

Open the recovered Run and sample several Case results. Confirm the displayed input, candidate output, reference output where present, applicable Rubrics, and metadata all belong together. A completed status alone does not prove correct mapping.

{% example-demo title="Example: scores drop after output import" %}
A team imports a new candidate file and sees a sudden score collapse. Run detail shows that the column containing reference outputs was mapped as candidate output. The operator creates a new Run, maps the actual candidate column, preserves the mistaken Run as an identifiable failed attempt, and compares only the corrected Run with the prior candidate under the same Benchmark Version.
{% /example-demo %}

## Source confidence

Code-backed: Runs index, setup, output mapping, Run detail, and metadata-display implementations establish the identifiers, mapping choices, and evidence shown during diagnosis. Backend-provider errors and customer Harness behavior may require additional operational logs outside this page.

## Related task pages

{% related-card-grid title="Related workflows" %}
- [Run an evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Inspect results](/docs/benchmark-evaluations/inspect-results)
- [Compare Harness Versions](/docs/benchmark-evaluations/compare)
- [Run Metadata](/docs/benchmark-evaluations/run-metadata)
{% /related-card-grid %}

## Related reference pages

{% related-card-grid title="Related reference" %}
- [Benchmarks and versions](/docs/concepts/benchmarks-and-versions)
- [Benchmark Snapshots](/docs/coverage-engineering/benchmark-snapshots)
- [Outputs](/docs/object-model/outputs)
{% /related-card-grid %}