{"query":"Evaluation Results","corpusVersion":"local","generatedAt":"2026-11-03T08:05:14.976Z","results":[{"blockId":"benchmark-evaluations.compare#evaluation-results","pageId":"benchmark-evaluations.compare","title":"Evaluation Results","pageTitle":"Evaluation Results","url":"https://teammately.ai/docs/benchmark-evaluations/results.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/results#evaluation-results","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/results.md","sectionId":"evaluation-results","kind":"task","productArea":"benchmark_evaluations","score":774.0174086845907,"reasons":["search_match","title_match","display_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Evaluation Results\n\nResults presents a matrix of selected Runs and the Cases or criteria they evaluated. Use one Run for inspection or several comparable Runs to locate gains, regressions and missing judgments."},{"blockId":"public-api.benchmark-results#example-read-evaluation-results","pageId":"public-api.benchmark-results","title":"Example: Read evaluation results","pageTitle":"Retrieve Benchmark Results","url":"https://teammately.ai/docs/public-api/benchmark-results.md","humanUrl":"https://teammately.ai/docs/public-api/benchmark-results#example-read-evaluation-results","markdownUrl":"https://teammately.ai/docs/public-api/benchmark-results.md","sectionId":"example-read-evaluation-results","kind":"api","productArea":"public_api","score":659.5838013856136,"reasons":["search_match","title_match","display_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"### Example: Read evaluation results\n\n```sh\ncurl --fail-with-body \\\n  --header \"Authorization: Bearer ${TEAMMATELY_API_KEY}\" \\\n  \"${BENCHMARK_URL}/runs/${RUN_ID}/evaluation-results?limit=20&offset=0\"\n```\n\n```python\nimport os\nfrom urllib.parse import quote, urlencode\nfrom urllib.request import HTTPRedirectHandler, Request, build_opener\n\nclass NoRedirect(HTTPRedirectHandler):\n    def redirect_request(self, req, fp, code, msg, headers, newurl):\n        return None\n\nopener = build_opener(NoRedirect())\napi_key = os.environ[\"TEAMMATELY_API_KEY\"]\n\nurl = os.environ[\"BENCHMARK_URL\"] + \"/runs/\" + quote(os.environ[\"RUN_ID\"], safe=\"\") + \"/evaluation-results\"\nurl += \"?\" + urlencode({\"limit\": \"20\", \"offset\": \"0\"})\nheaders = {\"Authorization\": \"Bearer \" + api_key}\nrequest = Request(url, headers=headers, method=\"GET\")\nwith opener.open(request) as response:\n    print(response.read().decode(\"utf-8\"))\n```\n\n```typescript\nconst apiKey = process.env.TEAMMATELY_API_KEY!;\n\n{\n  const url = new URL(process.env.BENCHMARK_URL! + \"/runs/\" + encodeURIComponent(process.env.RUN_ID!) + \"/evaluation-results\");\n  url.searchParams.set(\"limit\", \"20\");\n  url.searchParams.set(\"offset\", \"0\");\n  const response = await fetch(url, {\n    method: \"GET\",\n    redirect: \"error\",\n    headers: {\n      Authorization: `Bearer ${apiKey}`,\n    },\n  });\n  if (!response.ok) throw new Error(`HTTP ${response.status}`);\n  console.log(await response.text());\n}\n```\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    \"io\"\n    \"log\"\n    \"net/http\"\n    \"net/url\"\n    \"os\"\n)\n\nfunc main() {\n    if err := run(context.Background()); err != nil { log.Fatal(err) }\n}\n\nfunc run(ctx context.Context) error {\n    client := &http.Client{\n        CheckRedirect: func(req *http.Request, via []*http.Request) error {\n            return http.ErrUseLastResponse\n        },\n    }\n    {\n        target := os.Getenv(\"BENCHMARK_URL\") + \"/runs/\" + url.PathEscape(os.Getenv(\"RUN_ID\")) + \"/evaluation-results\"\n        query := url.Values{}\n        query.Set(\"limit\", \"20\")\n        query.Set(\"offset\", \"0\")\n        target += \"?\" + query.Encode()\n        request, err := http.NewRequestWithContext(ctx, \"GET\", target, nil)\n        if err != nil { return err }\n        request.Header.Set(\"Authorization\", \"Bearer \" + os.Getenv(\"TEAMMATELY_API_KEY\"))\n        response, err := client.Do(request)\n        if err != nil { return err }\n        payloadBytes, readErr := io.ReadAll(response.Body)\n        response.Body.Close()\n        if readErr != nil { return readErr }\n        fmt.Println(string(payloadBytes))\n        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf(\"HTTP %d\", response.StatusCode) }\n    }\n    return nil\n}\n```\n\nUse the child Trial's `runId`. The `data` payload holds the page. `limit` accepts 1–100 (default 20); `offset` is nonnegative. Page through the returned population instead of treating one page as all results.\n\n| Query | Selection |\n| --- | --- |\n| `result` | `passed`, `failed` or `missing` rubric observations |\n| `case_id`, `policy_id` | One Case or Policy |\n| `case_rubric_pair_id` | One Case/Rubric pair |\n| `dimension_id`, `ontology_id` | Declared coverage population |\n\nThe same filters apply to `evaluation-context`. Unknown/repeated parameters are rejected. Run and instruction reads take no query parameters."},{"blockId":"benchmark-evaluations.compare#steps","pageId":"benchmark-evaluations.compare","title":"Steps","pageTitle":"Evaluation Results","url":"https://teammately.ai/docs/benchmark-evaluations/results.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/results#steps","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/results.md","sectionId":"steps","kind":"task","productArea":"benchmark_evaluations","score":567.0250000520981,"reasons":["search_match","display_title_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Steps\n\n1. Open **Benchmark Evaluations → Results** and confirm the Benchmark Version.\n2. Select the visible Runs. Managed and imported Runs can both appear; an imported Run does not need a Harness Version to be inspected.\n3. Choose Cases, Policies, Rubrics, Dimension ontology values, Topics, Topic Groups or Scenarios as the row grouping.\n4. Read completeness before comparing scores. An empty or incomplete result is not a failed judgment.\n5. Open a Case or rubric detail to inspect the response, criterion, observed judgment and rationale.\n6. For repeated Trials, inspect the selected aggregation rule and the individual Trial outcomes.\n7. Route an actual candidate problem to Harness improvement, a standard problem to Correctness Governance, or an input-coverage problem to Coverage Design."},{"blockId":"benchmark-evaluations.inspect#inspect-evaluation-results","pageId":"benchmark-evaluations.inspect","title":"Inspect Evaluation Results","pageTitle":"Inspect Evaluation Results","url":"https://teammately.ai/docs/benchmark-evaluations/inspect-results.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/inspect-results#inspect-evaluation-results","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/inspect-results.md","sectionId":"inspect-evaluation-results","kind":"task","productArea":"benchmark_evaluations","score":564.3401374314755,"reasons":["search_match","title_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Inspect Evaluation Results\n\nStart with identity and completeness, then explain the score from the actual response and applicable Rubrics."},{"blockId":"improve.candidates-frontier#read-results","pageId":"improve.candidates-frontier","title":"Read Results","pageTitle":"Candidates and Results","url":"https://teammately.ai/docs/improve/candidates-and-frontier.md","humanUrl":"https://teammately.ai/docs/improve/candidates-and-frontier#read-results","markdownUrl":"https://teammately.ai/docs/improve/candidates-and-frontier.md","sectionId":"read-results","kind":"reference","productArea":"improve","score":447.0262143430345,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Read Results\n\nOpen the Session's **Results** panel to inspect candidate outcomes and comparison context. Its Evaluations view links the recorded executions. Check completeness and criteria before interpreting a lead.\n\nA candidate can improve one target while violating another constraint. Keep that tradeoff visible. A high overall score cannot replace a missing judgment on a protected criterion.\n\nThe retained result is not automatically the newest candidate. An unsuccessful direction can still explain why the agent tried another change, while an incomparable result cannot justify a numerical improvement claim."},{"blockId":"public-api.evaluation-outputs#example-read-the-manifest-and-submit-first-results","pageId":"public-api.evaluation-outputs","title":"Example: Read the manifest and submit first results","pageTitle":"Submit External Evaluation Outputs","url":"https://teammately.ai/docs/public-api/evaluation-outputs.md","humanUrl":"https://teammately.ai/docs/public-api/evaluation-outputs#example-read-the-manifest-and-submit-first-results","markdownUrl":"https://teammately.ai/docs/public-api/evaluation-outputs.md","sectionId":"example-read-the-manifest-and-submit-first-results","kind":"api","productArea":"public_api","score":430.4484673752884,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"### Example: Read the manifest and submit first results\n\n```sh\ncurl --fail-with-body \\\n  --header \"Authorization: Bearer ${TEAMMATELY_API_KEY}\" \\\n  \"${BENCHMARK_URL}/case-manifest\"\n\ncurl --fail-with-body \\\n  --header \"Authorization: Bearer ${TEAMMATELY_API_KEY}\" \\\n  --header 'Content-Type: application/json' \\\n  --header \"Idempotency-Key: ${RESULT_OPERATION_KEY}\" \\\n  --data-binary @first-results.json \\\n  \"${BENCHMARK_URL}/run-groups\"\n```\n\n```python\nimport os\nfrom pathlib import Path\nfrom urllib.parse import quote\nfrom urllib.request import HTTPRedirectHandler, Request, build_opener\n\nclass NoRedirect(HTTPRedirectHandler):\n    def redirect_request(self, req, fp, code, msg, headers, newurl):\n        return None\n\nopener = build_opener(NoRedirect())\napi_key = os.environ[\"TEAMMATELY_API_KEY\"]\n\nurl = os.environ[\"BENCHMARK_URL\"] + \"/case-manifest\"\nheaders = {\"Authorization\": \"Bearer \" + api_key}\nrequest = Request(url, headers=headers, method=\"GET\")\nwith opener.open(request) as response:\n    print(response.read().decode(\"utf-8\"))\n\nurl = os.environ[\"BENCHMARK_URL\"] + \"/run-groups\"\nheaders = {\"Authorization\": \"Bearer \" + api_key}\nheaders[\"Idempotency-Key\"] = os.environ[\"RESULT_OPERATION_KEY\"]\nheaders[\"Content-Type\"] = \"application/json\"\nbody = Path(\"first-results.json\").read_bytes()\nrequest = Request(url, data=body, headers=headers, method=\"POST\")\nwith opener.open(request) as response:\n    print(response.read().decode(\"utf-8\"))\n```\n\n```typescript\nimport { readFile } from \"node:fs/promises\";\n\nconst apiKey = process.env.TEAMMATELY_API_KEY!;\n\n{\n  const url = new URL(process.env.BENCHMARK_URL! + \"/case-manifest\");\n  const response = await fetch(url, {\n    method: \"GET\",\n    redirect: \"error\",\n    headers: {\n      Authorization: `Bearer ${apiKey}`,\n    },\n  });\n  if (!response.ok) throw new Error(`HTTP ${response.status}`);\n  console.log(await response.text());\n}\n\n{\n  const url = new URL(process.env.BENCHMARK_URL! + \"/run-groups\");\n  const response = await fetch(url, {\n    method: \"POST\",\n    redirect: \"error\",\n    headers: {\n      Authorization: `Bearer ${apiKey}`,\n      \"Idempotency-Key\": process.env.RESULT_OPERATION_KEY!,\n      \"Content-Type\": \"application/json\",\n    },\n    body: new Uint8Array(await readFile(\"first-results.json\")).buffer,\n  });\n  if (!response.ok) throw new Error(`HTTP ${response.status}`);\n  console.log(await response.text());\n}\n```\n\n```go\npackage main\n\nimport (\n    \"bytes\"\n    \"context\"\n    \"fmt\"\n    \"io\"\n    \"log\"\n    \"net/http\"\n    \"os\"\n)\n\nfunc main() {\n    if err := run(context.Background()); err != nil { log.Fatal(err) }\n}\n\nfunc run(ctx context.Context) error {\n    client := &http.Client{\n        CheckRedirect: func(req *http.Request, via []*http.Request) error {\n            return http.ErrUseLastResponse\n        },\n    }\n    {\n        target := os.Getenv(\"BENCHMARK_URL\") + \"/case-manifest\"\n        request, err := http.NewRequestWithContext(ctx, \"GET\", target, nil)\n        if err != nil { return err }\n        request.Header.Set(\"Authorization\", \"Bearer \" + os.Getenv(\"TEAMMATELY_API_KEY\"))\n        response, err := client.Do(request)\n        if err != nil { return err }\n        payloadBytes, readErr := io.ReadAll(response.Body)\n        response.Body.Close()\n        if readErr != nil { return readErr }\n        fmt.Println(string(payloadBytes))\n        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf(\"HTTP %d\", response.StatusCode) }\n    }\n    {\n        target := os.Getenv(\"BENCHMARK_URL\") + \"/run-groups\"\n        payload, err := os.ReadFile(\"first-results.json\")\n        if err != nil { return err }\n        request, err := http.NewRequestWithContext(ctx, \"POST\", target, bytes.NewReader(payload))\n        if err != nil { return err }\n        request.Header.Set(\"Authorization\", \"Bearer \" + os.Getenv(\"TEAMMATELY_API_KEY\"))\n        request.Header.Set(\"Idempotency-Key\", os.Getenv(\"RESULT_OPERATION_KEY\"))\n        request.Header.Set(\"Content-Type\", \"application/json\")\n        response, err := client.Do(request)\n        if err != nil { return err }\n        payloadBytes, readErr := io.ReadAll(response.Body)\n        response.Body.Close()\n        if readErr != nil { return readErr }\n        fmt.Println(string(payloadBytes))\n        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf(\"HTTP %d\", response.StatusCode) }\n    }\n    return nil\n}\n```\n\nRetain Group and child Run IDs. Append additional batches under the child's `results` route with a key per logical batch. Recover a lost response with its original bytes and key.\n\nRead import state before completing. Send `{\"allow_partial\":false}` to `POST /run-groups/{groupId}/runs/{runId}/complete`; use `true` only for intentionally missing Cases. A failed evaluation-admission retry uses `retry:true` and retains the original partial policy."},{"blockId":"benchmark-evaluations.overview#benchmark-evaluations","pageId":"benchmark-evaluations.overview","title":"Benchmark Evaluations","pageTitle":"Benchmark Evaluations","url":"https://teammately.ai/docs/benchmark-evaluations.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations#benchmark-evaluations","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations.md","sectionId":"benchmark-evaluations","kind":"concept","productArea":"benchmark_evaluations","score":422.1872487591107,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Benchmark Evaluations\n\nBenchmark Evaluations connects a fixed Benchmark Version to candidate responses, rubric judgments, scores and recorded resource use. Open **Runs** to launch or inspect execution, **Results** to inspect and compare outcomes, or **Dashboard** to read Progress, Leaderboard and Frontier."},{"blockId":"playbooks.alongside-existing-evals#using-teammately-alongside-existing-evaluation-infrastructure","pageId":"playbooks.alongside-existing-evals","title":"Using Teammately Alongside Existing Evaluation Infrastructure","pageTitle":"Using Teammately Alongside Existing Evaluation Infrastructure","url":"https://teammately.ai/docs/playbooks/using-teammately-alongside-existing-evaluation-infrastructure.md","humanUrl":"https://teammately.ai/docs/playbooks/using-teammately-alongside-existing-evaluation-infrastructure#using-teammately-alongside-existing-evaluation-infrastructure","markdownUrl":"https://teammately.ai/docs/playbooks/using-teammately-alongside-existing-evaluation-infrastructure.md","sectionId":"using-teammately-alongside-existing-evaluation-infrastructure","kind":"recipe","productArea":"playbooks","score":381.61453111439045,"reasons":["search_match","term_match"],"markdown":"# Using Teammately Alongside Existing Evaluation Infrastructure\n\nUse this playbook when a team already has tests, traces, dashboards, or offline evals and wants Teammately to add expert-grounded correctness results rather than replace everything."}]}