---
id: benchmark-datasets.snapshots
title: Dataset Snapshots
summary: Freeze Cases, evaluator links, and representation facts as an immutable benchmark evidence boundary.
kind: task
product_area: benchmark_datasets
status: stable
updated: 2026-08-22
canonical: /docs/benchmark-datasets/snapshots
---

# Dataset Snapshots

## Prerequisites

- A reviewed current Case set.
- Approved eligible evaluator links and no Snapshot readiness blockers.
- Permission to create benchmark evidence.

A Dataset Snapshot freezes the benchmark's selected Cases, eligible evaluator links, and representation facts at a point in time. The live dataset remains editable; the Snapshot opens read-only **Cases** and **Representation** views.

## Create a Snapshot

The readiness check reports Case count, approved eligible Policy and Rubric counts, and blockers. Resolve every blocker before creation. Record a meaningful Snapshot label, then verify the displayed version, content hash, creation time, and Case count.

Creation does not make weak input trustworthy. Review Case clarity, coverage, materials, and evaluator applicability first. After creation, do not describe later mutable classifications or links as if they were part of the frozen state.

## Evidence rules

- Identify the exact Snapshot or resulting Benchmark Version in every Run and comparison.
- Create a new Snapshot when Case membership, material content, or admitted evaluator relationships change in a way that affects the claim.
- Do not mutate a Snapshot to “fix” historical evidence; correct the live dataset and freeze a new one.
- If historical Representation is unavailable, report that limitation instead of substituting current facts.

{% example-demo title="Example: preserving a coverage expansion" %}
After Case Review adds eight exception-handling Cases, the team verifies approved rubric links and creates a new Snapshot. Runs against the earlier Benchmark Version remain comparable within their old boundary, while new Runs explicitly use the expanded version.
{% /example-demo %}

## Object and state changes

Creation adds a new immutable Snapshot with its own label, version, hash, time, Case membership, evaluator links, and representation facts. It does not lock or copy edits back into the current dataset.

## Success criteria

- Readiness has no blockers.
- Identity fields and Case count match the intended boundary.
- Future Runs cite the resulting exact Benchmark Version.

## Common failure modes

- Snapshotting weak or invalid Cases because readiness passes structurally.
- Treating current classifications as part of an older Snapshot.
- Comparing candidates across moved Snapshot boundaries without disclosure.

## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Benchmark Datasets](/docs/benchmark-datasets)
- [Benchmark versioning](/docs/governance/benchmark-versioning)
- [Reproducibility](/docs/governance/reproducibility)
{% /related-card-grid %}

## Related troubleshooting pages

{% related-card-grid title="Related troubleshooting pages" %}
- [Dataset upload](/docs/troubleshooting/dataset-upload)
- [Benchmark results changed unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
{% /related-card-grid %}

## Source confidence

Code-backed: the active Snapshots route defines readiness, blockers, immutable content, identity fields, and read-only Snapshot inspection.