# Retrieve Benchmark Results

Generated: 2026-11-02T05:44:14.713Z
Source build: local
Canonical docs: https://teammately.ai/docs

<doc id="public-api.benchmark-results" slug="public-api/benchmark-results" url="https://teammately.ai/docs/public-api/benchmark-results.md" title="Retrieve Benchmark Results" updated="2026-10-31" status="stable">
---
id: public-api.benchmark-results
title: Retrieve Benchmark Results
summary: Read scoped individual Run context, complete rubric results, instructions, and comparable Run differences.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/benchmark-results
---

# Retrieve Benchmark Results

## Definition

Under the organization/project/Benchmark/Version prefix, current gateway routes include:

```http
GET /version-context
GET /runs/{runId}
GET /runs/{runId}/evaluation-context
GET /runs/{runId}/evaluation-results
GET /runs/{runId}/evaluation-instructions
GET /comparison/{runId}/summary
GET /comparison/{runId}/cases
```

External Group/child reads additionally use their returned nested links.

## Read the exact population

Verify that the individual Run belongs to the selected Benchmark Version. Read available Case outputs, rubric judgments, native interpretation, rationale and incomplete state. Page through the authoritative filtered result population rather than treating the first page as all results.

Instructions and context orient the coding agent; neither is a mutation or a completed evaluation.

## Compare

Comparisons preserve individual Run grain and recorded conditions. Missing outputs, profile drift or unavailable native results can prevent a definitive comparison. A later independent Run does not silently replace the reference chosen before outcomes were known.

Record the exact starting and candidate Run IDs. Do not interpret legacy compatibility columns as mandatory binary rules when the current Regime supplies native point/gate meaning.

## Read a bounded page

### Example: Read evaluation results

```sh
curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  "${BENCHMARK_URL}/runs/${RUN_ID}/evaluation-results?limit=20&offset=0"
```

```python
import os
from urllib.parse import quote, urlencode
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/runs/" + quote(os.environ["RUN_ID"], safe="") + "/evaluation-results"
url += "?" + urlencode({"limit": "20", "offset": "0"})
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/runs/" + encodeURIComponent(process.env.RUN_ID!) + "/evaluation-results");
  url.searchParams.set("limit", "20");
  url.searchParams.set("offset", "0");
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/runs/" + url.PathEscape(os.Getenv("RUN_ID")) + "/evaluation-results"
        query := url.Values{}
        query.Set("limit", "20")
        query.Set("offset", "0")
        target += "?" + query.Encode()
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Use the child Trial's `runId`. The `data` payload holds the page. `limit` accepts 1–100 (default 20); `offset` is nonnegative. Page through the returned population instead of treating one page as all results.

| Query | Selection |
| --- | --- |
| `result` | `passed`, `failed` or `missing` rubric observations |
| `case_id`, `policy_id` | One Case or Policy |
| `case_rubric_pair_id` | One Case/Rubric pair |
| `dimension_id`, `ontology_id` | Declared coverage population |

The same filters apply to `evaluation-context`. Unknown/repeated parameters are rejected. Run and instruction reads take no query parameters.

## Compare two Trials

### Example: Compare two Trials

```sh
curl --fail-with-body --get \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  --data-urlencode "target_run_id=${TARGET_RUN_ID}" \
  "${BENCHMARK_URL}/comparison/${BASELINE_RUN_ID}/summary"
```

```python
import os
from urllib.parse import quote, urlencode
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/comparison/" + quote(os.environ["BASELINE_RUN_ID"], safe="") + "/summary"
url += "?" + urlencode({"target_run_id": os.environ["TARGET_RUN_ID"]})
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/comparison/" + encodeURIComponent(process.env.BASELINE_RUN_ID!) + "/summary");
  url.searchParams.set("target_run_id", process.env.TARGET_RUN_ID!);
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/comparison/" + url.PathEscape(os.Getenv("BASELINE_RUN_ID")) + "/summary"
        query := url.Values{}
        query.Set("target_run_id", os.Getenv("TARGET_RUN_ID"))
        target += "?" + query.Encode()
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Use `/cases` for paged changes, with `limit` and `offset`. Both child IDs must belong to the scope. Inspect comparability and incompleteness before interpreting a delta.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Result Serialization](/docs/public-api/result-schema)
- [Submit External Evaluation Outputs](/docs/public-api/evaluation-outputs)
- [Evaluation Results](/docs/benchmark-evaluations/results)
{% /related-card-grid %}
</doc>

<doc id="public-api.result-schema" slug="public-api/result-schema" url="https://teammately.ai/docs/public-api/result-schema.md" title="Result Serialization" updated="2026-10-31" status="stable">
---
id: public-api.result-schema
title: Result Serialization
summary: Interpret actual scoped responses, criteria, native judgments, point scores, completeness, and nullable telemetry without a fabricated global envelope.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/result-schema
---

# Result Serialization

## Definition

Evaluation access preserves selected public Run fields, Case content and exact references, actual outputs and rationales, frozen criteria/treatment context, native interpretation, coverage/completeness and captured telemetry.

The current access contract is `evaluation_access.v1`. Its owning service defines per-endpoint envelopes and pagination. Do not serialize every response as an earlier proposed universal Run/Policy/Rubric object.

## Judgments and scores

Preserve binary judgment, unavailable interpretation and the configured point/gate calculation separately. Current scores use exact fractions and can be negative. A completed mandatory-gate failure contributes zero; missing judgments remain incomplete and keep the full planned population unrankable.

Manual corrections retain original AI judgment and attributable change metadata. They do not rewrite the criterion or submitted response.

## Outputs and telemetry

External response items contain exactly one native JSON output or execution failure. Accepted text whitespace and JSON null remain meaningful submitted values. Optional trajectories describe captured model/tool/agent steps, not hidden reasoning.

Latency, usage and cost are nullable captured facts. An absent field does not mean zero. Preserve units and currency identity rather than inventing a normalized total.

## Evaluation page fields

The `data` payload contains:

| Field | Interpretation |
| --- | --- |
| `schema_version` | `evaluation_access.v1` |
| `run` | Scoped execution and pinned identity |
| `rows` | Current page of observations |
| `total`, `limit`, `offset` | Filtered population and current page |
| `grain` | `case_rubric_observation` when supplied |
| `selection`, `sampling` | Selection context when supplied |
| `links`, `next_page` | Navigation when supplied |

`evaluation-context` also carries procedure/context information. See `BenchmarkEvaluationPage`, `BenchmarkEvaluationContext` and `BenchmarkComparison` in the [OpenAPI](https://api.teammately.ai/v1/openapi.yaml). Native scores, completeness and HTTP success have separate meanings.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Retrieve Benchmark Results](/docs/public-api/benchmark-results)
- [API Conventions](/docs/public-api/conventions)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
</doc>

<doc id="public-api.evaluation-outputs" slug="public-api/evaluation-outputs" url="https://teammately.ai/docs/public-api/evaluation-outputs.md" title="Submit External Evaluation Outputs" updated="2026-10-31" status="stable">
---
id: public-api.evaluation-outputs
title: Submit External Evaluation Outputs
summary: Create an external Run from actual first results, append missing Case responses, and explicitly complete its Trial for evaluation.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/evaluation-outputs
---

# Submit External Evaluation Outputs

## Definition

Use the organization/project/Benchmark/Version prefix and retrieve `GET /case-manifest` for exact frozen Cases. Product Runs and Trials retain the API's existing `run-groups` and child `runs` wire identities.

## Create with actual results

```http
POST /run-groups
Idempotency-Key: <logical-command-key>
```

```json
{
  "name": "Application comparison",
  "results": [
    {
      "case_id": "<issued-case-id>",
      "output": "<response produced by the application>"
    }
  ]
}
```

The placeholder ID must be replaced with an issued manifest Case ID. This command atomically creates the Group and first child from real results; it does not reserve an empty reference Run.

## Append and complete

Use the returned IDs and links. `POST /run-groups/{groupId}/runs` starts another child Trial in that external Group. GET/POST `/run-groups/{groupId}/runs/{runId}/results` reads or appends Case results.

Each item contains exactly one output or failure. Optional fields include exact membership/version references, trajectory, latency, usage and cost. Native JSON and whitespace are retained. Accepted Case results cannot be replaced.

`POST .../complete` accepts `allow_partial`. Missing Cases require explicit partial completion. Repeated completion returns the recorded outcome; an explicit failed-admission retry does not change accepted content.

## Boundaries

Managed and controlled-evaluation Groups reject externally submitted responses. Corrections require a new child Trial. The existing external Group can accept that new child through its registered route; a new Group is also an explicit option. Accepted execution failures remain unavailable response judgments rather than fabricated rubric failures.

## Request sequence

Configure `BENCHMARK_URL` with the [quickstart](/docs/public-api/quickstart). Prepare `first-results.json` using the body above, an actual manifest Case ID and its application response.

### Example: Read the manifest and submit first results

```sh
curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  "${BENCHMARK_URL}/case-manifest"

curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  --header 'Content-Type: application/json' \
  --header "Idempotency-Key: ${RESULT_OPERATION_KEY}" \
  --data-binary @first-results.json \
  "${BENCHMARK_URL}/run-groups"
```

```python
import os
from pathlib import Path
from urllib.parse import quote
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/case-manifest"
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))

url = os.environ["BENCHMARK_URL"] + "/run-groups"
headers = {"Authorization": "Bearer " + api_key}
headers["Idempotency-Key"] = os.environ["RESULT_OPERATION_KEY"]
headers["Content-Type"] = "application/json"
body = Path("first-results.json").read_bytes()
request = Request(url, data=body, headers=headers, method="POST")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
import { readFile } from "node:fs/promises";

const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/case-manifest");
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}

{
  const url = new URL(process.env.BENCHMARK_URL! + "/run-groups");
  const response = await fetch(url, {
    method: "POST",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Idempotency-Key": process.env.RESULT_OPERATION_KEY!,
      "Content-Type": "application/json",
    },
    body: new Uint8Array(await readFile("first-results.json")).buffer,
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "bytes"
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/case-manifest"
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/run-groups"
        payload, err := os.ReadFile("first-results.json")
        if err != nil { return err }
        request, err := http.NewRequestWithContext(ctx, "POST", target, bytes.NewReader(payload))
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        request.Header.Set("Idempotency-Key", os.Getenv("RESULT_OPERATION_KEY"))
        request.Header.Set("Content-Type", "application/json")
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Retain Group and child Run IDs. Append additional batches under the child's `results` route with a key per logical batch. Recover a lost response with its original bytes and key.

Read import state before completing. Send `{"allow_partial":false}` to `POST /run-groups/{groupId}/runs/{runId}/complete`; use `true` only for intentionally missing Cases. A failed evaluation-admission retry uses `retry:true` and retains the original partial policy.

## Inspect the result

Read `GET /runs/{runId}` for judging progress, then [retrieve results](/docs/public-api/benchmark-results). Missing output, execution failure and an unfavorable rubric judgment are different outcomes. For a full local-file walkthrough, see [CLI external outputs](/docs/cli/external-outputs).
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Retrieve Benchmark Results](/docs/public-api/benchmark-results)
- [Result Serialization](/docs/public-api/result-schema)
- [API Conventions](/docs/public-api/conventions)
{% /related-card-grid %}
</doc>

<doc id="benchmark-evaluations.compare" slug="benchmark-evaluations/results" url="https://teammately.ai/docs/benchmark-evaluations/results.md" title="Evaluation Results" updated="2026-10-30" status="stable">
---
id: benchmark-evaluations.compare
title: Evaluation Results
summary: Inspect Case and rubric outcomes and compare exact managed or imported Runs within a Benchmark Version.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/results
---

# Evaluation Results

Results presents a matrix of selected Runs and the Cases or criteria they evaluated. Use one Run for inspection or several comparable Runs to locate gains, regressions and missing judgments.

## Prerequisites

- At least one Run for the intended Benchmark Version.
- Permission to inspect its Cases, responses and judgments.
- Comparable recorded conditions when interpreting differences.

## Steps

1. Open **Benchmark Evaluations → Results** and confirm the Benchmark Version.
2. Select the visible Runs. Managed and imported Runs can both appear; an imported Run does not need a Harness Version to be inspected.
3. Choose Cases, Policies, Rubrics, Dimension ontology values, Topics, Topic Groups or Scenarios as the row grouping.
4. Read completeness before comparing scores. An empty or incomplete result is not a failed judgment.
5. Open a Case or rubric detail to inspect the response, criterion, observed judgment and rationale.
6. For repeated Trials, inspect the selected aggregation rule and the individual Trial outcomes.
7. Route an actual candidate problem to Harness improvement, a standard problem to Correctness Governance, or an input-coverage problem to Coverage Design.

## Representative and historical Runs

Default candidate results use the Run selected before its outcomes were known. **Used for score** identifies that reference. Other executions remain in Run history and can be inspected without silently replacing the representative result.

Selecting matrix columns changes the analysis view. It does not activate a Harness, execute a candidate or declare a winner.

## Object and state changes

Filtering, row grouping and inspection are read-only. An authorized project manager can separately use **Correct judgment** on an exact rubric result. That operation records a correction and preserves the original AI judgment; refreshed aggregates come from the authoritative result snapshot.

## Success criteria

- Every compared column has an identified Run and Benchmark Version.
- Differences resolve to actual Cases and applicable criteria.
- Missing outcomes stay distinct from negative scores or failed gates.
- Manual corrections remain attributable.

## Common failure modes

- Treating an imported Run as an executable Harness.
- Combining later successful Trials into an earlier reference score.
- Reading a filtered population as the entire benchmark.
- Changing a rubric judgment when the real problem is an incorrect Case or standard.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Evaluation Dashboard](/docs/benchmark-evaluations/dashboard)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}

## Related troubleshooting pages

{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
- [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics)
{% /related-card-grid %}
</doc>