{"query":"Benchmark run troubleshooting","corpusVersion":"local","generatedAt":"2026-09-13T04:38:00.448Z","results":[{"blockId":"troubleshooting.benchmark-runs#benchmark-run-troubleshooting","pageId":"troubleshooting.benchmark-runs","title":"Benchmark run troubleshooting","pageTitle":"Benchmark run troubleshooting","url":"https://teammately.ai/docs/troubleshooting/benchmark-runs.md","humanUrl":"https://teammately.ai/docs/troubleshooting/benchmark-runs#benchmark-run-troubleshooting","markdownUrl":"https://teammately.ai/docs/troubleshooting/benchmark-runs.md","sectionId":"benchmark-run-troubleshooting","kind":"error","productArea":"troubleshooting","score":1749.9182629375177,"reasons":["search_match","title_match","display_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Benchmark run troubleshooting"},{"blockId":"benchmark-evaluations.run#related-troubleshooting-pages","pageId":"benchmark-evaluations.run","title":"Related troubleshooting pages","pageTitle":"Run a Benchmark Evaluation","url":"https://teammately.ai/docs/benchmark-evaluations/run-evaluation.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/run-evaluation#related-troubleshooting-pages","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/run-evaluation.md","sectionId":"related-troubleshooting-pages","kind":"task","productArea":"benchmark_evaluations","score":732.7966885748243,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Related troubleshooting pages\n\n{% related-card-grid title=\"Related troubleshooting pages\" %}\n- [Benchmark runs](/docs/troubleshooting/benchmark-runs)\n- [Missing outputs](/docs/troubleshooting/missing-outputs)\n- [Authentication](/docs/troubleshooting/authentication)\n{% /related-card-grid %}"},{"blockId":"benchmark-evaluations.run-metadata#benchmark-run-metadata","pageId":"benchmark-evaluations.run-metadata","title":"Benchmark Run Metadata","pageTitle":"Benchmark Run Metadata","url":"https://teammately.ai/docs/benchmark-evaluations/run-metadata.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/run-metadata#benchmark-run-metadata","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/run-metadata.md","sectionId":"benchmark-run-metadata","kind":"reference","productArea":"benchmark_evaluations","score":608.981734472532,"reasons":["search_match","term_match"],"markdown":"# Benchmark Run Metadata"},{"blockId":"benchmark-evaluations.run#run-a-benchmark-evaluation","pageId":"benchmark-evaluations.run","title":"Run a Benchmark Evaluation","pageTitle":"Run a Benchmark Evaluation","url":"https://teammately.ai/docs/benchmark-evaluations/run-evaluation.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/run-evaluation#run-a-benchmark-evaluation","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/run-evaluation.md","sectionId":"run-a-benchmark-evaluation","kind":"task","productArea":"benchmark_evaluations","score":605.1231181838976,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Run a Benchmark Evaluation\n\nLaunch a managed evaluation when the immutable Benchmark Version, governed evaluators, and candidate runtimes are ready."},{"blockId":"troubleshooting.overview#related-troubleshooting-pages","pageId":"troubleshooting.overview","title":"Related troubleshooting pages","pageTitle":"Troubleshooting","url":"https://teammately.ai/docs/troubleshooting.md","humanUrl":"https://teammately.ai/docs/troubleshooting#related-troubleshooting-pages","markdownUrl":"https://teammately.ai/docs/troubleshooting.md","sectionId":"related-troubleshooting-pages","kind":"concept","productArea":"troubleshooting","score":580.3609255369747,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Related troubleshooting pages\n\n{% related-card-grid title=\"Related troubleshooting pages\" %}\n- [Access troubleshooting](/docs/troubleshooting/authentication)\n- [Benchmark run troubleshooting](/docs/troubleshooting/benchmark-runs)\n- [Case upload troubleshooting](/docs/troubleshooting/dataset-upload)\n- [Permissions troubleshooting](/docs/troubleshooting/permissions)\n- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)\n{% /related-card-grid %}\n\n{% example-demo title=\"Unexpected benchmark change\" %}\nA score changes after a Dataset refresh even though the saved Harness Version is unchanged. The team confirms that the second Run used a new Benchmark Version with additional boundary Cases. It treats the result as evidence under an expanded benchmark, not as a candidate regression, and compares Case-level Rubric outcomes within each named boundary.\n{% /example-demo %}"},{"blockId":"troubleshooting.benchmark-runs#fix","pageId":"troubleshooting.benchmark-runs","title":"Fix","pageTitle":"Benchmark run troubleshooting","url":"https://teammately.ai/docs/troubleshooting/benchmark-runs.md","humanUrl":"https://teammately.ai/docs/troubleshooting/benchmark-runs#fix","markdownUrl":"https://teammately.ai/docs/troubleshooting/benchmark-runs.md","sectionId":"fix","kind":"error","productArea":"troubleshooting","score":550.7650853417869,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Fix\n\n- **No runnable version:** finish Case selection and create a Snapshot first.\n- **No mapped outputs:** correct Case identifiers or column mapping, then submit again under the intended Run.\n- **Wrong Harness or metadata:** create a correctly configured Run. Do not relabel completed evidence to represent a different system.\n- **Partial failure:** preserve successful per-Case evidence when the product does, correct the failed input or execution boundary, and rerun using a clearly named attempt.\n- **Confusing comparison:** compare the Benchmark Version, Harness Version, candidate metadata, and evaluator boundary. Qualify or avoid the comparison when more than the intended variable changed."},{"blockId":"troubleshooting.output-mapping#output-mapping-troubleshooting","pageId":"troubleshooting.output-mapping","title":"Output mapping troubleshooting","pageTitle":"Output mapping troubleshooting","url":"https://teammately.ai/docs/troubleshooting/output-mapping.md","humanUrl":"https://teammately.ai/docs/troubleshooting/output-mapping#output-mapping-troubleshooting","markdownUrl":"https://teammately.ai/docs/troubleshooting/output-mapping.md","sectionId":"output-mapping-troubleshooting","kind":"error","productArea":"troubleshooting","score":544.9297948128361,"reasons":["search_match","term_match"],"markdown":"# Output mapping troubleshooting\n\nUse this when externally produced responses exist but the mapping preview cannot join them cleanly to the Cases in the selected Benchmark Version."},{"blockId":"troubleshooting.benchmark-runs#symptoms","pageId":"troubleshooting.benchmark-runs","title":"Symptoms","pageTitle":"Benchmark run troubleshooting","url":"https://teammately.ai/docs/troubleshooting/benchmark-runs.md","humanUrl":"https://teammately.ai/docs/troubleshooting/benchmark-runs#symptoms","markdownUrl":"https://teammately.ai/docs/troubleshooting/benchmark-runs.md","sectionId":"symptoms","kind":"error","productArea":"troubleshooting","score":541.6823254698919,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Symptoms\n\n- The evaluation surface has no Benchmark Version to run.\n- Output import or mapping cannot identify a Case or output column.\n- A Run is created but remains empty, incomplete, or failed.\n- Results appear under the wrong candidate label or Harness Version.\n- Two Runs show a score change but do not share a comparable evidence boundary."}]}