Teammately Docs
Docs menu

task

Connect Model Outputs

Map externally produced outputs to immutable Benchmark Cases and create an output-only reference Run.

Connect Model Outputs

Prerequisites

  • An immutable Benchmark Version containing the Cases to evaluate.
  • One externally produced output per required Case.
  • Durable Case IDs from that Benchmark Version.
  • A reference label that identifies the external system or candidate.
  • Measured latency, usage, or cost only when the source system actually recorded it.

Before and after

beforeafter
Outputs exist in an external file or systemOutputs are mapped to exact immutable Benchmark Cases
Candidate identity is customer-owned contextA Teammately reference Run preserves both its own ID and the external reference label
No Teammately evaluation state existsRubric evaluation proceeds as a separate Run lifecycle
Missing telemetry may be ambiguousUnmeasured telemetry remains absent rather than becoming zero

Map outputs

Connect external model outputs

  1. Open the intended Benchmark Version and go to Benchmark Evaluations → Runs.
  2. Choose Import reference outputs and name the external system or candidate clearly.
  3. Download the mapping template for the current Benchmark Version. Keep case_id unchanged; use input and context columns only to verify the match.
  4. Populate output for each Case. Add latency, usage, or cost columns only for values measured by the producing system.
  5. Upload the file and inspect unknown Case IDs, missing Benchmark Cases, duplicates, and output previews.
  6. Resolve every mapping error. Do not join on input text or force an output onto a similar-looking Case.
  7. Confirm the mapping and inspect the output-only reference Run.
  8. Wait for evaluation to complete before interpreting result summaries or failures.

Object and state changes

The workflow creates an ordinary output-only reference Run for one immutable Benchmark Version and associates submitted outputs with its Cases. It can also store measured telemetry supplied with those outputs.

It does not create or save a Harness Version, change Benchmark membership, mutate Cases, approve reference responses, or make the external candidate available to Improve as an executable Harness.

Success criteria

  • Every output joins through the exact Case ID from the intended Benchmark Version.
  • The reference label distinguishes this output set from other Runs.
  • Unknown, missing, duplicate, inserted, and updated counts are understood before interpretation.
  • Unmeasured telemetry is absent.
  • The output-only Run is not presented as a managed Harness candidate.
  • Result interpretation waits for evaluation completion.

Common failure modes

  • Reusing IDs from the editable Project Case collection instead of the frozen Benchmark Version.
  • Joining on input text, row order, or a customer ID without verifying the Teammately Case ID.
  • Uploading outputs for two candidate versions under one reference label.
  • Reporting missing latency or cost as zero.
  • Treating successful mapping as successful evaluation.
  • Assuming the reference Run can enter Harness Compare, Arena, or Improve as an executable candidate.

Worked example

Retrieval candidate outputs

A retrieval team evaluates a new indexing configuration outside Teammately. It exports one answer per frozen Benchmark Case and preserves its own generation ID. In the mapping template, each answer joins on case_id; the generation ID remains correlation context and measured latency is included.

The imported set becomes a reference Run. Teammately evaluates its outputs against the Benchmark Version's Policy and Rubric evidence, while the indexing configuration itself remains outside Teammately as a non-Harness system.

Related reference pages

Task / Benchmark EvaluationsMap External Evaluation OutputsImport reference outputs, map them to immutable benchmark Cases and attempts, and inspect the resulting output-only Run.Concept / Benchmark EvaluationsBenchmark EvaluationsRun and inspect exact Harness Versions against an immutable Benchmark Version through Dashboard, List, Arena, and Compare.Task / Benchmark DatasetsDataset SnapshotsFreeze Cases, evaluator links, and representation facts as an immutable benchmark evidence boundary.Concept / Data IntegrationsIntegrationsMove Cases, source materials, external outputs, and benchmark evidence across Teammately's supported product boundaries.

Related troubleshooting pages

Troubleshooting / Recovery LibraryOutput mapping troubleshootingFix external output rows that do not join cleanly to immutable Case IDs in an output-only Run.Troubleshooting / Recovery LibraryMissing OutputsSeparate a missing managed Run response from an unmatched row in an imported output-only Run.Troubleshooting / Recovery LibraryBenchmark Results Changed UnexpectedlyDiagnose result changes across target behavior, benchmark cases, standards, and versions.

Source confidence

Code-backed: the current external-reference Run and output-mapping surfaces define the immutable Case join, template fields, reference-Run boundary, and result transition. Exact customer API serialization remains publication-gated separately.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Benchmark Evaluations Map External Evaluation Outputs Import reference outputs, map them to immutable benchmark Cases and attempts, and inspect the resulting output-only Run. Benchmark Evaluations Run a Benchmark Evaluation Launch exact active Harness Versions against an immutable Benchmark Version as standard or repeated Run Groups. Benchmark Datasets Dataset Snapshots Freeze Cases, evaluator links, and representation facts as an immutable benchmark evidence boundary. Recovery Library Output mapping troubleshooting Fix external output rows that do not join cleanly to immutable Case IDs in an output-only Run.
AI context