---
id: object-model.benchmarks
title: Benchmarks
summary: Understand a Benchmark as the durable program that owns benchmark-scoped coverage, evidence boundaries, evaluations, and improvement work.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/benchmarks
---

# Benchmarks

## Definition

A Benchmark is the durable project object for one intended evaluation program. It owns benchmark-scoped work across Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve while its selected Cases, governed standards, and candidate systems evolve.

The current Benchmark Dataset is editable. A Dataset Snapshot freezes selected Case membership, and a Benchmark Version provides the immutable boundary consumed by Runs. A Benchmark is therefore not a Snapshot, Benchmark Version, Run, or score.

## Fields, states, or lifecycle rules

- The Benchmark identity persists across changes to its current Dataset, coverage work, standards, and Harness candidates.
- Benchmark Datasets owns selected Cases and immutable Dataset Snapshots.
- A Benchmark Version fixes the evidence boundary used by a Run.
- Benchmark membership should be shaped by coverage work, not by whichever Cases are easiest to run.
- A Run result is weak if the Benchmark Version and saved Harness Version behind it are unclear.
- This page documents object semantics, not public execution, export, rate-limit, or API guarantees.

## Related objects

Benchmarks should be read with [Cases](/docs/assets/cases), [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Coverage Engineering](/docs/coverage-engineering), and [Benchmark Evaluations](/docs/benchmark-evaluations). Use [Run an evaluation](/docs/benchmark-evaluations/run-evaluation) when the next step is execution.

{% example-demo title="Benchmarks boundary" %}
Raw case: A team refreshes coverage after finding unsupported compatibility claims.

Benchmark version: The refreshed version includes new unsupported-claim cases and the approved compatibility rubric.

Run: The candidate is evaluated against that version.

Interpretation: If the score drops, reviewers can see that the benchmark became harder instead of assuming the candidate behavior changed.
{% /example-demo %}

## Source confidence

Code-backed: the Benchmark type and workspace establish durable Benchmark identity; Benchmark Datasets → Snapshots establishes immutable Dataset boundaries; the evaluation-runs route consumes a specific Benchmark Version. The public object definition does not imply an execution or export API.

## Related task pages

{% related-card-grid title="Related task pages" %}
- [Benchmarks](/docs/coverage-engineering/benchmarks)
- [Benchmarks and versions](/docs/concepts/benchmarks-and-versions)
- [Benchmark Evaluations](/docs/benchmark-evaluations)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}