---
id: benchmark-evaluations.execution-settings
title: Evaluation Execution Settings
summary: Configure machines and choose saved Harness Versions and Run counts at launch.
kind: reference
product_area: benchmark_evaluations
status: stable
updated: 2026-09-13
canonical: /docs/benchmark-evaluations/execution-settings
---

# Evaluation Execution Settings

## Definition

Evaluation Settings contains machine settings for managed execution. Harness selection and repetition are choices made when launching an evaluation.

## Fields, states, or lifecycle rules

- Choose an exact saved project Harness Version when launching. Benchmark activation is not required.
- Choose a whole number of Runs from 1 through 50 for each selected Harness. The default is one, and different Harnesses may have different counts.
- Each managed Harness launch creates one Run Group containing the requested Runs, including a one-Run group.
- Existing Runs keep their immutable configuration and evidence. Launching more Runs creates another group rather than rewriting previous membership.

## Reliability choices

Dashboard, Compare, and Arena aggregate compatible observed Runs for each saved Harness Version across launch groups. Average score weights individual Runs equally. Passed at least once and passed every time summarize observed binary case outcomes where the evaluation framework supports them.

Run counts can differ. The product displays the counts and coverage and warns about unequal evidence without requiring another launch. Missing or infrastructure-failed observations are not numerical successes or failures.

## Before launching

Check the exact saved Version, machine selection, and requested execution volume. Runtime preparation and required access still apply. Imported external outputs remain a separate flow because they have no executable Harness Version.

## Related task pages

{% related-card-grid title="Related task pages" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
{% /related-card-grid %}

## Source confidence

Code-backed: Evaluation Settings defines machine configuration; the shared Run modal defines saved-Version selection and per-launch Run counts.