---
id: benchmark-evaluations.run
title: Run a Benchmark Evaluation
summary: Launch exact active Harness Versions against an immutable Benchmark Version as standard or repeated Run Groups.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-09-13
canonical: /docs/benchmark-evaluations/run-evaluation
---

# Run a Benchmark Evaluation

Launch a managed evaluation when the immutable Benchmark Version, governed evaluators, and candidate runtimes are ready.

## Prerequisites

- A Benchmark Version backed by the intended Dataset Snapshot.
- Approved eligible Policies and Rubrics.
- At least one saved project Harness Version.
- Prepared Harness runtime and required secret grants.
- A chosen number of Runs for each selected Harness.

## Steps

1. Open **Benchmark Evaluations** for the intended Benchmark Version.
2. Open Evaluation Settings if you need to adjust the machine configuration.
3. Start a Run and select one or more offered Harness Versions. Confirm the exact version labels rather than relying on Harness names alone.
4. Choose the number of Runs for each Harness. Counts may differ; review the total execution volume.
5. Supply any requested Run Metadata. Keep credentials out of descriptive fields.
6. Launch. Each selected Harness creates its own Run Group containing the requested independent Runs, including when the count is one.
7. Follow output and evaluation progress. Distinguish queued, running, complete, failed, cancelled, and incomplete work rather than inferring completion from partial scores.
8. Inspect List, Dashboard, Arena, or Compare only after checking which attempts and Cases are evaluable.

## Evidence created

The launch creates Run Groups and Runs bound to exact Harness and Benchmark Versions. Per-Case outputs and evaluator outcomes accrue separately, so output completion can precede evaluation completion. Provider telemetry can include tokens, cost, and latency when captured; absence of telemetry is not zero usage.

Dashboard aggregates compatible observed Runs across launches. Choose average score, passed at least once, or passed every time where supported. Each Run retains its own outputs and status; inspect the group and individual Runs when work is incomplete.

> No Draft execution
>
> A managed benchmark Run does not evaluate the mutable Harness Draft. Save the candidate and select its exact saved Version when launching.

{% example-demo title="Example: two candidates, three attempts" %}
Harness Versions 6 and 9 are active with `n=3`. One launch creates two Run Groups and six independent Runs against the same Benchmark Version. If one attempt fails preparation, the group reports incomplete evidence instead of silently treating the remaining two as the configured cohort.
{% /example-demo %}

## Object and state changes

Launching creates one Run Group per Harness and one or more independent Runs. Outputs, evaluator outcomes, progress, metadata, and telemetry accrue to those records. A later launch creates new evidence and does not overwrite the cohort.

## Success criteria

- Exact Harness and Benchmark Versions are recorded.
- Each launch group contains the number of Runs requested for that Harness.
- Output and evaluation progress reach an interpretable terminal state.
- Incomplete or failed attempts remain visible.

## Common failure modes

- Selecting the wrong saved Version or assuming Draft execution.
- Reading partial evaluation as a complete cohort.
- Treating absent telemetry as zero usage.

## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Evaluation Execution Settings](/docs/benchmark-evaluations/execution-settings)
- [Harnesses](/docs/assets/harnesses)
- [Run Metadata](/docs/benchmark-evaluations/run-metadata)
{% /related-card-grid %}

## Related troubleshooting pages

{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark runs](/docs/troubleshooting/benchmark-runs)
- [Missing outputs](/docs/troubleshooting/missing-outputs)
- [Authentication](/docs/troubleshooting/authentication)
{% /related-card-grid %}

## Source confidence

Code-backed: the current Run modal, Runs workspace, and Run Group route define selection, group creation, repeated attempts, progress, and evidence identity.