# Case Versioning Generated: 2026-09-13T04:41:29.735Z Source build: local Canonical docs: https://teammately.ai/docs --- id: governance.case-versioning title: Case Versioning summary: Track meaningful changes to cases, metadata, context, and attached outputs. kind: reference product_area: governance status: stable updated: 2026-08-23 canonical: /docs/governance/case-versioning --- # Case Versioning ## Definition A Case has durable identity and versioned canonical content. The exact Case version records the input shape and Case Materials that an evaluator or Harness received. Customer provenance and classification context can help find and interpret the Case, but they must not obscure which canonical version a historical Benchmark Version froze. ## Fields, states, or lifecycle rules - A change to canonical input or required Case Materials creates a new evidence meaning and must remain distinguishable from the earlier version. - Project Input Schema determines the accepted input kind and named Case Material slots. - Correcting a filename label or customer-owned filter may be operational metadata; do not claim that every metadata edit has the same consequence as changed evaluated content. - Accepted or golden output changes affect reference-output interpretation and should be traced separately from the Case input version. - Existing Snapshots and Runs continue to identify the Case version they used. - A current Case can be selected by several Benchmarks; version review must consider each future dataset that uses it. - Case IDs and version IDs are backend-issued opaque values. ## Related objects Read [Cases](/docs/assets/cases) for canonical Case structure, [Project Input Schema](/docs/project-settings/input-schema) for input and material requirements, and [Dataset Snapshots](/docs/benchmark-datasets/snapshots) for the immutable benchmark boundary. Use Benchmark Versioning when corrected Cases need to enter future evaluation evidence. {% example-demo title="Correcting a controlling document" %} A Case asks whether an opened product is returnable, but its attached policy PDF is discovered to be superseded. Replacing that Case Material changes the evidence presented to the candidate and evaluator, so the corrected Case must remain distinguishable from the old version. Runs against the old Benchmark Version stay interpretable as historical evidence; the corrected Case enters a new Dataset Snapshot for future Runs. {% /example-demo %} ## Source confidence Code-backed: canonical Case content and Project Input Schema services define input and material structure, while Case view types preserve backend-issued Case identity for product surfaces. Exact persistence mechanics remain internal; Snapshot pages define public historical evidence behavior. ## Related task pages {% related-card-grid title="Related task pages" %} - [Cases](/docs/assets/cases) - [Dataset Snapshots](/docs/benchmark-datasets/snapshots) - [Benchmark Versioning](/docs/governance/benchmark-versioning) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.cases title: Cases summary: Understand cases as the situations Teammately uses to represent important AI behavior. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/cases --- # Cases ## Definition Cases are the behavior situations Teammately uses for review, coverage, and benchmark evidence. A case should preserve the input, source context, metadata, and version boundary needed to understand what behavior is being judged. Use this reference when a workflow depends on whether an example is reviewable, benchmark-ready, stale, duplicated, or missing the context a reviewer needs. ## Fields, states, or lifecycle rules - A case is not the same thing as an output; the case holds the situation and context, while the output is the behavior being judged. - A case can be useful for coverage planning before it is ready for benchmark use. - Case versions matter when source context, metadata, or expected behavior changes. - A benchmark result is weak if the failed output cannot be traced back to the exact case version. - Exact import schema and export behavior belong to source-backed integration pages, not this object reference. ## Related objects Cases should be read with [Outputs](/docs/object-model/outputs), [Case versions](/docs/governance/case-versioning), [Case pool](/docs/object-model/case-pool), and [Benchmarks](/docs/object-model/benchmarks). Use [Importing cases](/docs/operating-manual/import-and-prepare-cases) when the next step is bringing source examples into Teammately. {% example-demo title="Cases boundary" %} Raw record: A transcript includes a customer question, assistant answer, account context, and later human correction. Case: The reviewable unit preserves the customer question and account context. Output: The candidate answer is attached separately so reviewers know which behavior is being judged. Interpretation: If the candidate fails later, the run result can point to the exact case and output rather than to a vague transcript. {% /example-demo %} ## Source confidence Code-backed: Case view types and the Cases asset route establish the product object and its presentation; internal Case-content types establish how input and contextual material remain separate from outputs. These implementation sources support object semantics, not a customer-facing serialization contract. ## Related task pages {% related-card-grid title="Related task pages" %} - [Case model overview](/docs/assets) - [Case Pool](/docs/coverage-engineering/case-pool) - [Case pool](/docs/object-model/case-pool) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: benchmark-datasets.snapshots title: Dataset Snapshots summary: Freeze Cases, evaluator links, and representation facts as an immutable benchmark evidence boundary. kind: task product_area: benchmark_datasets status: stable updated: 2026-08-22 canonical: /docs/benchmark-datasets/snapshots --- # Dataset Snapshots ## Prerequisites - A reviewed current Case set. - Approved eligible evaluator links and no Snapshot readiness blockers. - Permission to create benchmark evidence. A Dataset Snapshot freezes the benchmark's selected Cases, eligible evaluator links, and representation facts at a point in time. The live dataset remains editable; the Snapshot opens read-only **Cases** and **Representation** views. ## Create a Snapshot The readiness check reports Case count, approved eligible Policy and Rubric counts, and blockers. Resolve every blocker before creation. Record a meaningful Snapshot label, then verify the displayed version, content hash, creation time, and Case count. Creation does not make weak input trustworthy. Review Case clarity, coverage, materials, and evaluator applicability first. After creation, do not describe later mutable classifications or links as if they were part of the frozen state. ## Evidence rules - Identify the exact Snapshot or resulting Benchmark Version in every Run and comparison. - Create a new Snapshot when Case membership, material content, or admitted evaluator relationships change in a way that affects the claim. - Do not mutate a Snapshot to “fix” historical evidence; correct the live dataset and freeze a new one. - If historical Representation is unavailable, report that limitation instead of substituting current facts. {% example-demo title="Example: preserving a coverage expansion" %} After Case Review adds eight exception-handling Cases, the team verifies approved rubric links and creates a new Snapshot. Runs against the earlier Benchmark Version remain comparable within their old boundary, while new Runs explicitly use the expanded version. {% /example-demo %} ## Object and state changes Creation adds a new immutable Snapshot with its own label, version, hash, time, Case membership, evaluator links, and representation facts. It does not lock or copy edits back into the current dataset. ## Success criteria - Readiness has no blockers. - Identity fields and Case count match the intended boundary. - Future Runs cite the resulting exact Benchmark Version. ## Common failure modes - Snapshotting weak or invalid Cases because readiness passes structurally. - Treating current classifications as part of an older Snapshot. - Comparing candidates across moved Snapshot boundaries without disclosure. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Benchmark Datasets](/docs/benchmark-datasets) - [Benchmark versioning](/docs/governance/benchmark-versioning) - [Reproducibility](/docs/governance/reproducibility) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Dataset upload](/docs/troubleshooting/dataset-upload) - [Benchmark results changed unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly) {% /related-card-grid %} ## Source confidence Code-backed: the active Snapshots route defines readiness, blockers, immutable content, identity fields, and read-only Snapshot inspection. --- id: governance.benchmark-versioning title: Benchmark Versioning summary: Preserve benchmark snapshots so evidence can be compared across target and standard changes. kind: reference product_area: governance status: stable updated: 2026-08-23 canonical: /docs/governance/benchmark-versioning --- # Benchmark Versioning ## Definition A Benchmark Version is the immutable evidence boundary used by Runs. It identifies the frozen dataset state and admitted evaluator relationships that make a result interpretable. The Benchmark remains a durable program; its versions preserve successive evidence boundaries as Cases, materials, coverage, Policies, or Rubrics change. ## Fields, states, or lifecycle rules - Create a new Snapshot and resulting Benchmark Version when changed evidence would alter what a Run claims to test. - Existing Runs remain attached to their original Benchmark Version. - Current Dataset edits do not mutate a historical version. - A new Harness Version alone does not require a new Benchmark Version; candidate and evidence versions move independently. - Comparisons within one Benchmark Version isolate candidate differences more cleanly. - Cross-version comparisons must name the changed Cases, evaluators, or representation boundary. - Version identity does not prove that coverage is sufficient or that every admitted Rubric is correct. ## Related objects Use [Dataset Snapshots](/docs/benchmark-datasets/snapshots) to create the frozen dataset boundary. Use [Benchmark Evaluations](/docs/benchmark-evaluations) to inspect Runs inside one exact version, and [Compare Harness Versions](/docs/benchmark-evaluations/compare) to interpret candidate movement without hiding version changes. {% example-demo title="Separating candidate change from standard change" %} Harness Version 12 improves retrieval and is evaluated against Benchmark Version 4, the same boundary used for Version 11. That comparison isolates candidate behavior. Later, experts approve a stricter source-authority Rubric and the dataset gains conflict Cases. The team creates Benchmark Version 5 and reports subsequent Runs under that new boundary instead of presenting the lower score as a regression against unchanged evidence. {% /example-demo %} ## Source confidence Code-backed: Benchmark Datasets → Snapshots preserves immutable Dataset boundaries, and the version-scoped Evaluation route binds Runs to one selected Benchmark Version. Coverage quality and downstream decisions remain outside version identity itself. ## Related task pages {% related-card-grid title="Related task pages" %} - [Benchmark snapshots](/docs/coverage-engineering/benchmark-snapshots) - [Benchmarks](/docs/object-model/benchmarks) - [Compare Harness Versions](/docs/benchmark-evaluations/compare) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %}