Teammately Docs
Docs menu

task

Dataset Snapshots

Freeze Cases, evaluator links, and representation facts as an immutable benchmark evidence boundary.

Dataset Snapshots

Prerequisites

  • A reviewed current Case set.
  • Approved eligible evaluator links and no Snapshot readiness blockers.
  • Permission to create benchmark evidence.

A Dataset Snapshot freezes the benchmark's selected Cases, eligible evaluator links, and representation facts at a point in time. The live dataset remains editable; the Snapshot opens read-only Cases and Representation views.

Create a Snapshot

The readiness check reports Case count, approved eligible Policy and Rubric counts, and blockers. Resolve every blocker before creation. Record a meaningful Snapshot label, then verify the displayed version, content hash, creation time, and Case count.

Creation does not make weak input trustworthy. Review Case clarity, coverage, materials, and evaluator applicability first. After creation, do not describe later mutable classifications or links as if they were part of the frozen state.

Evidence rules

  • Identify the exact Snapshot or resulting Benchmark Version in every Run and comparison.
  • Create a new Snapshot when Case membership, material content, or admitted evaluator relationships change in a way that affects the claim.
  • Do not mutate a Snapshot to “fix” historical evidence; correct the live dataset and freeze a new one.
  • If historical Representation is unavailable, report that limitation instead of substituting current facts.

Worked example

Example: preserving a coverage expansion

After Case Review adds eight exception-handling Cases, the team verifies approved rubric links and creates a new Snapshot. Runs against the earlier Benchmark Version remain comparable within their old boundary, while new Runs explicitly use the expanded version.

Object and state changes

Creation adds a new immutable Snapshot with its own label, version, hash, time, Case membership, evaluator links, and representation facts. It does not lock or copy edits back into the current dataset.

Success criteria

  • Readiness has no blockers.
  • Identity fields and Case count match the intended boundary.
  • Future Runs cite the resulting exact Benchmark Version.

Common failure modes

  • Snapshotting weak or invalid Cases because readiness passes structurally.
  • Treating current classifications as part of an older Snapshot.
  • Comparing candidates across moved Snapshot boundaries without disclosure.

Related reference pages

Concept / Benchmark DatasetsBenchmark DatasetsSelect benchmark Cases, inspect representation, and freeze immutable Snapshots for reproducible evidence.Reference / Governance and ReproducibilityBenchmark VersioningPreserve benchmark snapshots so evidence can be compared across target and standard changes.Reference / Governance and ReproducibilityReproducibilityPreserve enough source context to explain and repeat correctness decisions.

Related troubleshooting pages

Troubleshooting / Recovery LibraryCase upload troubleshootingRepair uploads with rejected rows, missing inputs, incorrect column mapping, or Cases that arrive without usable context.Troubleshooting / Recovery LibraryBenchmark Results Changed UnexpectedlyDiagnose result changes across target behavior, benchmark cases, standards, and versions.

Source confidence

Code-backed: the active Snapshots route defines readiness, blockers, immutable content, identity fields, and read-only Snapshot inspection.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Benchmark Datasets Benchmark Datasets Select benchmark Cases, inspect representation, and freeze immutable Snapshots for reproducible evidence. Benchmark Evaluations Run a Benchmark Evaluation Launch exact active Harness Versions against an immutable Benchmark Version as standard or repeated Run Groups. Governance and Reproducibility Reproducibility Preserve enough source context to explain and repeat correctness decisions.
AI context