# Submit External Evaluation Outputs

Generated: 2026-11-02T05:42:52.607Z
Source build: local
Canonical docs: https://teammately.ai/docs

<doc id="public-api.evaluation-outputs" slug="public-api/evaluation-outputs" url="https://teammately.ai/docs/public-api/evaluation-outputs.md" title="Submit External Evaluation Outputs" updated="2026-10-31" status="stable">
---
id: public-api.evaluation-outputs
title: Submit External Evaluation Outputs
summary: Create an external Run from actual first results, append missing Case responses, and explicitly complete its Trial for evaluation.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/evaluation-outputs
---

# Submit External Evaluation Outputs

## Definition

Use the organization/project/Benchmark/Version prefix and retrieve `GET /case-manifest` for exact frozen Cases. Product Runs and Trials retain the API's existing `run-groups` and child `runs` wire identities.

## Create with actual results

```http
POST /run-groups
Idempotency-Key: <logical-command-key>
```

```json
{
  "name": "Application comparison",
  "results": [
    {
      "case_id": "<issued-case-id>",
      "output": "<response produced by the application>"
    }
  ]
}
```

The placeholder ID must be replaced with an issued manifest Case ID. This command atomically creates the Group and first child from real results; it does not reserve an empty reference Run.

## Append and complete

Use the returned IDs and links. `POST /run-groups/{groupId}/runs` starts another child Trial in that external Group. GET/POST `/run-groups/{groupId}/runs/{runId}/results` reads or appends Case results.

Each item contains exactly one output or failure. Optional fields include exact membership/version references, trajectory, latency, usage and cost. Native JSON and whitespace are retained. Accepted Case results cannot be replaced.

`POST .../complete` accepts `allow_partial`. Missing Cases require explicit partial completion. Repeated completion returns the recorded outcome; an explicit failed-admission retry does not change accepted content.

## Boundaries

Managed and controlled-evaluation Groups reject externally submitted responses. Corrections require a new child Trial. The existing external Group can accept that new child through its registered route; a new Group is also an explicit option. Accepted execution failures remain unavailable response judgments rather than fabricated rubric failures.

## Request sequence

Configure `BENCHMARK_URL` with the [quickstart](/docs/public-api/quickstart). Prepare `first-results.json` using the body above, an actual manifest Case ID and its application response.

### Example: Read the manifest and submit first results

```sh
curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  "${BENCHMARK_URL}/case-manifest"

curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  --header 'Content-Type: application/json' \
  --header "Idempotency-Key: ${RESULT_OPERATION_KEY}" \
  --data-binary @first-results.json \
  "${BENCHMARK_URL}/run-groups"
```

```python
import os
from pathlib import Path
from urllib.parse import quote
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/case-manifest"
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))

url = os.environ["BENCHMARK_URL"] + "/run-groups"
headers = {"Authorization": "Bearer " + api_key}
headers["Idempotency-Key"] = os.environ["RESULT_OPERATION_KEY"]
headers["Content-Type"] = "application/json"
body = Path("first-results.json").read_bytes()
request = Request(url, data=body, headers=headers, method="POST")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
import { readFile } from "node:fs/promises";

const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/case-manifest");
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}

{
  const url = new URL(process.env.BENCHMARK_URL! + "/run-groups");
  const response = await fetch(url, {
    method: "POST",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Idempotency-Key": process.env.RESULT_OPERATION_KEY!,
      "Content-Type": "application/json",
    },
    body: new Uint8Array(await readFile("first-results.json")).buffer,
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "bytes"
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/case-manifest"
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/run-groups"
        payload, err := os.ReadFile("first-results.json")
        if err != nil { return err }
        request, err := http.NewRequestWithContext(ctx, "POST", target, bytes.NewReader(payload))
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        request.Header.Set("Idempotency-Key", os.Getenv("RESULT_OPERATION_KEY"))
        request.Header.Set("Content-Type", "application/json")
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Retain Group and child Run IDs. Append additional batches under the child's `results` route with a key per logical batch. Recover a lost response with its original bytes and key.

Read import state before completing. Send `{"allow_partial":false}` to `POST /run-groups/{groupId}/runs/{runId}/complete`; use `true` only for intentionally missing Cases. A failed evaluation-admission retry uses `retry:true` and retains the original partial policy.

## Inspect the result

Read `GET /runs/{runId}` for judging progress, then [retrieve results](/docs/public-api/benchmark-results). Missing output, execution failure and an unfavorable rubric judgment are different outcomes. For a full local-file walkthrough, see [CLI external outputs](/docs/cli/external-outputs).
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Retrieve Benchmark Results](/docs/public-api/benchmark-results)
- [Result Serialization](/docs/public-api/result-schema)
- [API Conventions](/docs/public-api/conventions)
{% /related-card-grid %}
</doc>

<doc id="public-api.benchmark-results" slug="public-api/benchmark-results" url="https://teammately.ai/docs/public-api/benchmark-results.md" title="Retrieve Benchmark Results" updated="2026-10-31" status="stable">
---
id: public-api.benchmark-results
title: Retrieve Benchmark Results
summary: Read scoped individual Run context, complete rubric results, instructions, and comparable Run differences.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/benchmark-results
---

# Retrieve Benchmark Results

## Definition

Under the organization/project/Benchmark/Version prefix, current gateway routes include:

```http
GET /version-context
GET /runs/{runId}
GET /runs/{runId}/evaluation-context
GET /runs/{runId}/evaluation-results
GET /runs/{runId}/evaluation-instructions
GET /comparison/{runId}/summary
GET /comparison/{runId}/cases
```

External Group/child reads additionally use their returned nested links.

## Read the exact population

Verify that the individual Run belongs to the selected Benchmark Version. Read available Case outputs, rubric judgments, native interpretation, rationale and incomplete state. Page through the authoritative filtered result population rather than treating the first page as all results.

Instructions and context orient the coding agent; neither is a mutation or a completed evaluation.

## Compare

Comparisons preserve individual Run grain and recorded conditions. Missing outputs, profile drift or unavailable native results can prevent a definitive comparison. A later independent Run does not silently replace the reference chosen before outcomes were known.

Record the exact starting and candidate Run IDs. Do not interpret legacy compatibility columns as mandatory binary rules when the current Regime supplies native point/gate meaning.

## Read a bounded page

### Example: Read evaluation results

```sh
curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  "${BENCHMARK_URL}/runs/${RUN_ID}/evaluation-results?limit=20&offset=0"
```

```python
import os
from urllib.parse import quote, urlencode
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/runs/" + quote(os.environ["RUN_ID"], safe="") + "/evaluation-results"
url += "?" + urlencode({"limit": "20", "offset": "0"})
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/runs/" + encodeURIComponent(process.env.RUN_ID!) + "/evaluation-results");
  url.searchParams.set("limit", "20");
  url.searchParams.set("offset", "0");
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/runs/" + url.PathEscape(os.Getenv("RUN_ID")) + "/evaluation-results"
        query := url.Values{}
        query.Set("limit", "20")
        query.Set("offset", "0")
        target += "?" + query.Encode()
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Use the child Trial's `runId`. The `data` payload holds the page. `limit` accepts 1–100 (default 20); `offset` is nonnegative. Page through the returned population instead of treating one page as all results.

| Query | Selection |
| --- | --- |
| `result` | `passed`, `failed` or `missing` rubric observations |
| `case_id`, `policy_id` | One Case or Policy |
| `case_rubric_pair_id` | One Case/Rubric pair |
| `dimension_id`, `ontology_id` | Declared coverage population |

The same filters apply to `evaluation-context`. Unknown/repeated parameters are rejected. Run and instruction reads take no query parameters.

## Compare two Trials

### Example: Compare two Trials

```sh
curl --fail-with-body --get \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  --data-urlencode "target_run_id=${TARGET_RUN_ID}" \
  "${BENCHMARK_URL}/comparison/${BASELINE_RUN_ID}/summary"
```

```python
import os
from urllib.parse import quote, urlencode
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/comparison/" + quote(os.environ["BASELINE_RUN_ID"], safe="") + "/summary"
url += "?" + urlencode({"target_run_id": os.environ["TARGET_RUN_ID"]})
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/comparison/" + encodeURIComponent(process.env.BASELINE_RUN_ID!) + "/summary");
  url.searchParams.set("target_run_id", process.env.TARGET_RUN_ID!);
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/comparison/" + url.PathEscape(os.Getenv("BASELINE_RUN_ID")) + "/summary"
        query := url.Values{}
        query.Set("target_run_id", os.Getenv("TARGET_RUN_ID"))
        target += "?" + query.Encode()
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Use `/cases` for paged changes, with `limit` and `offset`. Both child IDs must belong to the scope. Inspect comparability and incompleteness before interpreting a delta.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Result Serialization](/docs/public-api/result-schema)
- [Submit External Evaluation Outputs](/docs/public-api/evaluation-outputs)
- [Evaluation Results](/docs/benchmark-evaluations/results)
{% /related-card-grid %}
</doc>

<doc id="public-api.result-schema" slug="public-api/result-schema" url="https://teammately.ai/docs/public-api/result-schema.md" title="Result Serialization" updated="2026-10-31" status="stable">
---
id: public-api.result-schema
title: Result Serialization
summary: Interpret actual scoped responses, criteria, native judgments, point scores, completeness, and nullable telemetry without a fabricated global envelope.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/result-schema
---

# Result Serialization

## Definition

Evaluation access preserves selected public Run fields, Case content and exact references, actual outputs and rationales, frozen criteria/treatment context, native interpretation, coverage/completeness and captured telemetry.

The current access contract is `evaluation_access.v1`. Its owning service defines per-endpoint envelopes and pagination. Do not serialize every response as an earlier proposed universal Run/Policy/Rubric object.

## Judgments and scores

Preserve binary judgment, unavailable interpretation and the configured point/gate calculation separately. Current scores use exact fractions and can be negative. A completed mandatory-gate failure contributes zero; missing judgments remain incomplete and keep the full planned population unrankable.

Manual corrections retain original AI judgment and attributable change metadata. They do not rewrite the criterion or submitted response.

## Outputs and telemetry

External response items contain exactly one native JSON output or execution failure. Accepted text whitespace and JSON null remain meaningful submitted values. Optional trajectories describe captured model/tool/agent steps, not hidden reasoning.

Latency, usage and cost are nullable captured facts. An absent field does not mean zero. Preserve units and currency identity rather than inventing a normalized total.

## Evaluation page fields

The `data` payload contains:

| Field | Interpretation |
| --- | --- |
| `schema_version` | `evaluation_access.v1` |
| `run` | Scoped execution and pinned identity |
| `rows` | Current page of observations |
| `total`, `limit`, `offset` | Filtered population and current page |
| `grain` | `case_rubric_observation` when supplied |
| `selection`, `sampling` | Selection context when supplied |
| `links`, `next_page` | Navigation when supplied |

`evaluation-context` also carries procedure/context information. See `BenchmarkEvaluationPage`, `BenchmarkEvaluationContext` and `BenchmarkComparison` in the [OpenAPI](https://api.teammately.ai/v1/openapi.yaml). Native scores, completeness and HTTP success have separate meanings.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Retrieve Benchmark Results](/docs/public-api/benchmark-results)
- [API Conventions](/docs/public-api/conventions)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
</doc>

<doc id="public-api.conventions" slug="public-api/conventions" url="https://teammately.ai/docs/public-api/conventions.md" title="API Conventions" updated="2026-10-31" status="stable">
---
id: public-api.conventions
title: API Conventions
summary: Use organization-scoped paths, operation-specific envelopes, issued IDs, exact retry identity, and explicit completion.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/conventions
---

# API Conventions

## Definition

The resource prefix is `/v1/organizations/{organizationKey}/projects/{projectId}`. Benchmark operations continue beneath the selected Benchmark and immutable Version. Paths and IDs are scoped; a readable title is not a join key.

## Envelopes and pagination

Case lists return `message` and `data: {columns, rows}`; detail returns `message` and `result`. Fieldloop and benchmark operations use their documented `data` payloads. Parse the called operation's envelope.

Case reads use limit/offset pagination, defaulting to 10 rows and accepting a positive limit through 1,000. Other readers have their own pagination. Preserve returned continuation information instead of assuming a cursor shape universally.

Case reads emit ETag and support If-None-Match. Concurrency and mutation preconditions belong to the particular operation.

## Writes and retries

Use an Idempotency-Key where the operation requires it. Preserve exact request content and logical identity, but follow that operation's recovery contract. Changed content is a different command.

External results append only to the issued Trial. Completion freezes accepted responses and admits evaluation separately. A failed admission retry does not replace those responses.

## IDs and errors

Use database/API-issued opaque IDs. Example placeholders are not usable IDs. Keep the original HTTP status, structured code and actionable reason; no global error envelope is promised across legacy and forwarded families.

## HTTP outcomes

| Outcome | Next step |
| --- | --- |
| `200` or `201` | Inspect the resource or per-item outcomes |
| `202` | Retain the issued identity and inspect processing state |
| `304` | Use the representation associated with the matching ETag |
| `400` or `422` | Correct the request using its code and reason |
| `401` or `403` | Correct credentials or scope |
| `409` or `412` | Inspect immutable state, idempotency or preconditions |
| `429` | Honor `Retry-After` when supplied |
| `502`, `503` or `504` | Preserve command identity before recovering an uncertain write |

Not every endpoint emits every status. The [OpenAPI](https://api.teammately.ai/v1/openapi.yaml) lists operation-specific responses; one rate-limit response does not define a universal quota.

## Recover without duplicating work

Fieldloop and benchmark submission workflows support their documented exact-content recovery. Preserve original bytes/key and inspect the returned receipt. A client timeout does not undo admitted work.

Case imports and batch Case reference writes use duplicate-key rejection: repeating a key returns `409`, not a replay of the original response. Retain the issued import identity when available and read its status. If a response is lost before its identity is known, reconcile through your administrator or support before submitting with a new key; a fresh key can create another operation. Do not assume duplicate rejection proves whether the first handler completed.

See [SDK Delivery and Recovery](/docs/sdks/delivery-and-recovery) for the Fieldloop client path.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Public API](/docs/public-api)
- [API Authentication](/docs/public-api/authentication)
- [Submit External Evaluation Outputs](/docs/public-api/evaluation-outputs)
{% /related-card-grid %}
</doc>