# Result Serialization

Generated: 2026-11-02T06:36:39.722Z
Source build: local
Canonical docs: https://teammately.ai/docs

<doc id="public-api.result-schema" slug="public-api/result-schema" url="https://teammately.ai/docs/public-api/result-schema.md" title="Result Serialization" updated="2026-10-31" status="stable">
---
id: public-api.result-schema
title: Result Serialization
summary: Interpret actual scoped responses, criteria, native judgments, point scores, completeness, and nullable telemetry without a fabricated global envelope.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/result-schema
---

# Result Serialization

## Definition

Evaluation access preserves selected public Run fields, Case content and exact references, actual outputs and rationales, frozen criteria/treatment context, native interpretation, coverage/completeness and captured telemetry.

The current access contract is `evaluation_access.v1`. Its owning service defines per-endpoint envelopes and pagination. Do not serialize every response as an earlier proposed universal Run/Policy/Rubric object.

## Judgments and scores

Preserve binary judgment, unavailable interpretation and the configured point/gate calculation separately. Current scores use exact fractions and can be negative. A completed mandatory-gate failure contributes zero; missing judgments remain incomplete and keep the full planned population unrankable.

Manual corrections retain original AI judgment and attributable change metadata. They do not rewrite the criterion or submitted response.

## Outputs and telemetry

External response items contain exactly one native JSON output or execution failure. Accepted text whitespace and JSON null remain meaningful submitted values. Optional trajectories describe captured model/tool/agent steps, not hidden reasoning.

Latency, usage and cost are nullable captured facts. An absent field does not mean zero. Preserve units and currency identity rather than inventing a normalized total.

## Evaluation page fields

The `data` payload contains:

| Field | Interpretation |
| --- | --- |
| `schema_version` | `evaluation_access.v1` |
| `run` | Scoped execution and pinned identity |
| `rows` | Current page of observations |
| `total`, `limit`, `offset` | Filtered population and current page |
| `grain` | `case_rubric_observation` when supplied |
| `selection`, `sampling` | Selection context when supplied |
| `links`, `next_page` | Navigation when supplied |

`evaluation-context` also carries procedure/context information. See `BenchmarkEvaluationPage`, `BenchmarkEvaluationContext` and `BenchmarkComparison` in the [OpenAPI](https://api.teammately.ai/v1/openapi.yaml). Native scores, completeness and HTTP success have separate meanings.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Retrieve Benchmark Results](/docs/public-api/benchmark-results)
- [API Conventions](/docs/public-api/conventions)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
</doc>

<doc id="public-api.benchmark-results" slug="public-api/benchmark-results" url="https://teammately.ai/docs/public-api/benchmark-results.md" title="Retrieve Benchmark Results" updated="2026-10-31" status="stable">
---
id: public-api.benchmark-results
title: Retrieve Benchmark Results
summary: Read scoped individual Run context, complete rubric results, instructions, and comparable Run differences.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/benchmark-results
---

# Retrieve Benchmark Results

## Definition

Under the organization/project/Benchmark/Version prefix, current gateway routes include:

```http
GET /version-context
GET /runs/{runId}
GET /runs/{runId}/evaluation-context
GET /runs/{runId}/evaluation-results
GET /runs/{runId}/evaluation-instructions
GET /comparison/{runId}/summary
GET /comparison/{runId}/cases
```

External Group/child reads additionally use their returned nested links.

## Read the exact population

Verify that the individual Run belongs to the selected Benchmark Version. Read available Case outputs, rubric judgments, native interpretation, rationale and incomplete state. Page through the authoritative filtered result population rather than treating the first page as all results.

Instructions and context orient the coding agent; neither is a mutation or a completed evaluation.

## Compare

Comparisons preserve individual Run grain and recorded conditions. Missing outputs, profile drift or unavailable native results can prevent a definitive comparison. A later independent Run does not silently replace the reference chosen before outcomes were known.

Record the exact starting and candidate Run IDs. Do not interpret legacy compatibility columns as mandatory binary rules when the current Regime supplies native point/gate meaning.

## Read a bounded page

### Example: Read evaluation results

```sh
curl --fail-with-body \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  "${BENCHMARK_URL}/runs/${RUN_ID}/evaluation-results?limit=20&offset=0"
```

```python
import os
from urllib.parse import quote, urlencode
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/runs/" + quote(os.environ["RUN_ID"], safe="") + "/evaluation-results"
url += "?" + urlencode({"limit": "20", "offset": "0"})
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/runs/" + encodeURIComponent(process.env.RUN_ID!) + "/evaluation-results");
  url.searchParams.set("limit", "20");
  url.searchParams.set("offset", "0");
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/runs/" + url.PathEscape(os.Getenv("RUN_ID")) + "/evaluation-results"
        query := url.Values{}
        query.Set("limit", "20")
        query.Set("offset", "0")
        target += "?" + query.Encode()
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Use the child Trial's `runId`. The `data` payload holds the page. `limit` accepts 1–100 (default 20); `offset` is nonnegative. Page through the returned population instead of treating one page as all results.

| Query | Selection |
| --- | --- |
| `result` | `passed`, `failed` or `missing` rubric observations |
| `case_id`, `policy_id` | One Case or Policy |
| `case_rubric_pair_id` | One Case/Rubric pair |
| `dimension_id`, `ontology_id` | Declared coverage population |

The same filters apply to `evaluation-context`. Unknown/repeated parameters are rejected. Run and instruction reads take no query parameters.

## Compare two Trials

### Example: Compare two Trials

```sh
curl --fail-with-body --get \
  --header "Authorization: Bearer ${TEAMMATELY_API_KEY}" \
  --data-urlencode "target_run_id=${TARGET_RUN_ID}" \
  "${BENCHMARK_URL}/comparison/${BASELINE_RUN_ID}/summary"
```

```python
import os
from urllib.parse import quote, urlencode
from urllib.request import HTTPRedirectHandler, Request, build_opener

class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        return None

opener = build_opener(NoRedirect())
api_key = os.environ["TEAMMATELY_API_KEY"]

url = os.environ["BENCHMARK_URL"] + "/comparison/" + quote(os.environ["BASELINE_RUN_ID"], safe="") + "/summary"
url += "?" + urlencode({"target_run_id": os.environ["TARGET_RUN_ID"]})
headers = {"Authorization": "Bearer " + api_key}
request = Request(url, headers=headers, method="GET")
with opener.open(request) as response:
    print(response.read().decode("utf-8"))
```

```typescript
const apiKey = process.env.TEAMMATELY_API_KEY!;

{
  const url = new URL(process.env.BENCHMARK_URL! + "/comparison/" + encodeURIComponent(process.env.BASELINE_RUN_ID!) + "/summary");
  url.searchParams.set("target_run_id", process.env.TARGET_RUN_ID!);
  const response = await fetch(url, {
    method: "GET",
    redirect: "error",
    headers: {
      Authorization: `Bearer ${apiKey}`,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  console.log(await response.text());
}
```

```go
package main

import (
    "context"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
)

func main() {
    if err := run(context.Background()); err != nil { log.Fatal(err) }
}

func run(ctx context.Context) error {
    client := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        },
    }
    {
        target := os.Getenv("BENCHMARK_URL") + "/comparison/" + url.PathEscape(os.Getenv("BASELINE_RUN_ID")) + "/summary"
        query := url.Values{}
        query.Set("target_run_id", os.Getenv("TARGET_RUN_ID"))
        target += "?" + query.Encode()
        request, err := http.NewRequestWithContext(ctx, "GET", target, nil)
        if err != nil { return err }
        request.Header.Set("Authorization", "Bearer " + os.Getenv("TEAMMATELY_API_KEY"))
        response, err := client.Do(request)
        if err != nil { return err }
        payloadBytes, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil { return readErr }
        fmt.Println(string(payloadBytes))
        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf("HTTP %d", response.StatusCode) }
    }
    return nil
}
```

Use `/cases` for paged changes, with `limit` and `offset`. Both child IDs must belong to the scope. Inspect comparability and incompleteness before interpreting a delta.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Result Serialization](/docs/public-api/result-schema)
- [Submit External Evaluation Outputs](/docs/public-api/evaluation-outputs)
- [Evaluation Results](/docs/benchmark-evaluations/results)
{% /related-card-grid %}
</doc>

<doc id="public-api.conventions" slug="public-api/conventions" url="https://teammately.ai/docs/public-api/conventions.md" title="API Conventions" updated="2026-10-31" status="stable">
---
id: public-api.conventions
title: API Conventions
summary: Use organization-scoped paths, operation-specific envelopes, issued IDs, exact retry identity, and explicit completion.
kind: api
product_area: public_api
status: stable
updated: 2026-10-31
canonical: /docs/public-api/conventions
---

# API Conventions

## Definition

The resource prefix is `/v1/organizations/{organizationKey}/projects/{projectId}`. Benchmark operations continue beneath the selected Benchmark and immutable Version. Paths and IDs are scoped; a readable title is not a join key.

## Envelopes and pagination

Case lists return `message` and `data: {columns, rows}`; detail returns `message` and `result`. Fieldloop and benchmark operations use their documented `data` payloads. Parse the called operation's envelope.

Case reads use limit/offset pagination, defaulting to 10 rows and accepting a positive limit through 1,000. Other readers have their own pagination. Preserve returned continuation information instead of assuming a cursor shape universally.

Case reads emit ETag and support If-None-Match. Concurrency and mutation preconditions belong to the particular operation.

## Writes and retries

Use an Idempotency-Key where the operation requires it. Preserve exact request content and logical identity, but follow that operation's recovery contract. Changed content is a different command.

External results append only to the issued Trial. Completion freezes accepted responses and admits evaluation separately. A failed admission retry does not replace those responses.

## IDs and errors

Use database/API-issued opaque IDs. Example placeholders are not usable IDs. Keep the original HTTP status, structured code and actionable reason; no global error envelope is promised across legacy and forwarded families.

## HTTP outcomes

| Outcome | Next step |
| --- | --- |
| `200` or `201` | Inspect the resource or per-item outcomes |
| `202` | Retain the issued identity and inspect processing state |
| `304` | Use the representation associated with the matching ETag |
| `400` or `422` | Correct the request using its code and reason |
| `401` or `403` | Correct credentials or scope |
| `409` or `412` | Inspect immutable state, idempotency or preconditions |
| `429` | Honor `Retry-After` when supplied |
| `502`, `503` or `504` | Preserve command identity before recovering an uncertain write |

Not every endpoint emits every status. The [OpenAPI](https://api.teammately.ai/v1/openapi.yaml) lists operation-specific responses; one rate-limit response does not define a universal quota.

## Recover without duplicating work

Fieldloop and benchmark submission workflows support their documented exact-content recovery. Preserve original bytes/key and inspect the returned receipt. A client timeout does not undo admitted work.

Case imports and batch Case reference writes use duplicate-key rejection: repeating a key returns `409`, not a replay of the original response. Retain the issued import identity when available and read its status. If a response is lost before its identity is known, reconcile through your administrator or support before submitting with a new key; a fresh key can create another operation. Do not assume duplicate rejection proves whether the first handler completed.

See [SDK Delivery and Recovery](/docs/sdks/delivery-and-recovery) for the Fieldloop client path.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Public API](/docs/public-api)
- [API Authentication](/docs/public-api/authentication)
- [Submit External Evaluation Outputs](/docs/public-api/evaluation-outputs)
{% /related-card-grid %}
</doc>

<doc id="correctness.scoring" slug="correctness-governance/scoring" url="https://teammately.ai/docs/correctness-governance/scoring.md" title="Rubric Points and Benchmark Scores" updated="2026-10-30" status="stable">
---
id: correctness.scoring
title: Rubric Points and Benchmark Scores
summary: Interpret earned points, penalties, mandatory gates, negative percentages, and incomplete benchmark results.
kind: reference
product_area: correctness_governance
status: stable
updated: 2026-10-30
canonical: /docs/correctness-governance/scoring
---

# Rubric Points and Benchmark Scores

## Definition

A binary rubric judgment describes observed behavior. The Regime determines how that judgment contributes to a Case and Benchmark score. Positive points, penalties and mandatory gates have distinct meanings.

## Fields, states, or lifecycle rules

- Positive points are earned when the relevant Rubric passes.
- Penalty points are deducted when its Rubric fails.
- A mandatory gate determines eligibility in a gated framework.
- Missing or incompatible judgments remain incomplete.
- Exact recorded fractions are displayed as percentages.

## Case score

For current point-based configuration, the Case score is:

```text
(earned positive points - triggered penalty points)
  / total applicable positive points
```

If there are no applicable positive points, the denominator is the total applicable penalty points. If that denominator is also zero, the Case is unscored.

A penalty-only Case ranges from 0% when no penalty triggers to -100% when all trigger. A mixed Case can score below -100%. There is no per-Case floor, cap or output scaling in the current configuration.

{% example-demo title="A valid negative score" %}
A Case has 2 applicable positive points and 6 penalty points. It earns both positive points and triggers every penalty. Its score is (2 - 6) / 2, or -200%. The negative result preserves the configured consequence rather than being clamped to zero.
{% /example-demo %}

## Gates and incomplete judgments

A completed mandatory-gate failure contributes zero to the benchmark score. Its rubric calculation remains available for diagnosis. Missing judgments do not become zero merely because another gate failed.

Benchmark and repeated-Trial means retain the entire frozen population. An unresolved Case or Trial makes the evaluation incomplete and unrankable. Technical execution failure is not a valid rubric failure.

## History and configuration

Project Settings → Regime configures point ranges, increments and guidance. New settings affect future captured work. Earlier Benchmark Versions retain their accepted scoring configuration; historical scaling rules are interpreted under that configuration.

Use **Correct judgment** for an attributable correction to an exact result. Use the governing Policy, Rubric or Regime settings when the intended standard is what needs to change.
## Related task pages

{% related-card-grid title="Related task pages" %}
- [Regime Settings](/docs/project-settings/regime)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Write Binary Rubrics](/docs/correctness-governance/binary-rubrics)
{% /related-card-grid %}
</doc>