{"query":"Benchmarks and versions","corpusVersion":"local","generatedAt":"2026-09-14T05:51:08.661Z","results":[{"blockId":"concepts.benchmarks-versions#benchmarks-and-versions","pageId":"concepts.benchmarks-versions","title":"Benchmarks and versions","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#benchmarks-and-versions","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"benchmarks-and-versions","kind":"concept","productArea":"object_model","score":1933.6247875047427,"reasons":["search_match","title_match","display_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Benchmarks and versions"},{"blockId":"concepts.benchmarks-versions#version-sanity-check","pageId":"concepts.benchmarks-versions","title":"Version sanity check","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#version-sanity-check","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"version-sanity-check","kind":"concept","productArea":"object_model","score":802.3825010091368,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Version sanity check\n\n| Before comparing runs, confirm... | Otherwise... |\n| --- | --- |\n| The same benchmark version was used, or the version change is intentional and named. | Score movement may be caused by changed cases or standards. |\n| The Dataset Snapshot and governed evaluator boundary are identifiable. | A comparison can hide the evidence that actually changed. |\n| The saved Harness Version, settings, and Run Metadata are attached. | The result cannot explain which candidate configuration was evaluated. |"},{"blockId":"benchmark-evaluations.compare#compare-harness-versions","pageId":"benchmark-evaluations.compare","title":"Compare Harness Versions","pageTitle":"Compare Harness Versions","url":"https://teammately.ai/docs/benchmark-evaluations/compare.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/compare#compare-harness-versions","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/compare.md","sectionId":"compare-harness-versions","kind":"task","productArea":"benchmark_evaluations","score":747.063994593775,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Compare Harness Versions\n\nCompare shows aggregate observed evidence for saved Harness Versions. Choose the Versions, compatible evaluation configuration, and measurement to compare. Use **Inspect individual Run comparisons** for the detailed result matrix."},{"blockId":"concepts.benchmarks-versions#related-reference-pages","pageId":"concepts.benchmarks-versions","title":"Related reference pages","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#related-reference-pages","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"related-reference-pages","kind":"concept","productArea":"object_model","score":657.392961720175,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Related reference pages\n\n{% related-card-grid title=\"Related reference pages\" %}\n- [Benchmarks](/docs/object-model/benchmarks)\n- [Dataset Snapshots](/docs/benchmark-datasets/snapshots)\n- [Benchmark Versioning](/docs/governance/benchmark-versioning)\n- [Reproducibility](/docs/governance/reproducibility)\n{% /related-card-grid %}"},{"blockId":"concepts.benchmarks-versions#artifacts-it-affects","pageId":"concepts.benchmarks-versions","title":"Artifacts it affects","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#artifacts-it-affects","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"artifacts-it-affects","kind":"concept","productArea":"object_model","score":649.5216963131428,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Artifacts it affects\n\nThe Benchmark connects Dataset membership, Snapshots, coverage work, Contributions, Benchmark Versions, Runs, comparisons, and Improvement Sessions. Project Cases, Coverage Facets, Policies, Rubrics, and Harnesses remain reusable project objects rather than becoming owned by the Benchmark.\n\n{% example-demo title=\"Coverage refresh version\" %}\nReviewers discover an unsupported compatibility boundary after a candidate passes Benchmark Version 3. They add reviewed Cases to the current Dataset and create a new Snapshot and Benchmark Version 4. The original Run remains evidence for Version 3. A new Run against Version 4 answers the expanded question, and any customer-owned human review context names both versions.\n{% /example-demo %}"},{"blockId":"concepts.benchmarks-versions#where-it-appears-in-the-product","pageId":"concepts.benchmarks-versions","title":"Where it appears in the product","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#where-it-appears-in-the-product","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"where-it-appears-in-the-product","kind":"concept","productArea":"object_model","score":644.4923829588697,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Where it appears in the product\n\nUse the Benchmark selector to enter a Benchmark workspace. **Benchmark Datasets** owns selected Cases and Snapshots. **Coverage Management** explains representation and gaps. **Expert Contributions** scopes specialist work. **Benchmark Evaluations** runs and compares saved Harness Versions. **Improve** explores evidence-backed candidate changes."},{"blockId":"concepts.benchmarks-versions#related-workflows","pageId":"concepts.benchmarks-versions","title":"Related workflows","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#related-workflows","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"related-workflows","kind":"concept","productArea":"object_model","score":642.8905273476008,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Related workflows\n\n{% related-card-grid title=\"Related workflows\" %}\n- [Run a benchmark](/docs/benchmark-evaluations/run-evaluation)\n- [Compare Harness Versions](/docs/benchmark-evaluations/compare)\n- [Read run results](/docs/benchmark-evaluations/inspect-results)\n- [Work with Benchmark Datasets](/docs/benchmark-datasets)\n- [Refresh coverage after product change](/docs/coverage-engineering/coverage-refresh)\n{% /related-card-grid %}"},{"blockId":"concepts.benchmarks-versions#definition","pageId":"concepts.benchmarks-versions","title":"Definition","pageTitle":"Benchmarks and versions","url":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","humanUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions#definition","markdownUrl":"https://teammately.ai/docs/concepts/benchmarks-and-versions.md","sectionId":"definition","kind":"concept","productArea":"object_model","score":633.8897543493707,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Definition\n\nA Benchmark is the durable workspace for one evaluation program. Its current Dataset and coverage work can change over time. A Dataset Snapshot freezes selected Case membership; a Benchmark Version fixes the evidence boundary used by an Evaluation Run."}]}