# Outputs Generated: 2026-09-13T04:34:25.735Z Source build: local Canonical docs: https://teammately.ai/docs --- id: object-model.outputs title: Outputs summary: Distinguish managed Run responses, imported output-only Runs, and Case-scoped reference outputs. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/outputs --- # Outputs Understand the outputs Teammately reviews, compares, and turns into correctness evidence. ## Definition An output is response evidence associated with a Case, but its identity depends on the workflow that produced it. A managed Evaluation Run generates a response from a saved Harness Version. An imported output-only Run maps externally produced responses to immutable Case IDs. A Case may also expose a golden or reference response where the current surface supports it. Do not collapse these into one generic “accepted output” lifecycle. Imported outputs are not Harness Versions, Run responses are not automatically reference answers, and reviewer comments are not outputs. ## Fields, states, or lifecycle rules - A managed Run response is bound to the Run, saved Harness Version, Benchmark Version, and Case. - An imported output-only Run joins external rows to immutable Case IDs and retains imported candidate metadata. - Output-only Runs can be scored and inspected, but cannot be activated, optimized in Improve, or used as Harness columns in Compare or Arena. - A golden or reference output is Case-scoped example evidence where exposed; it does not become a global Policy. - Mapping errors can make evidence misleading even when the external response data is unchanged. - This page documents object boundaries, not model execution, provider behavior, or public output schemas. ## Related objects Read outputs with [Cases](/docs/object-model/cases), [Reference and golden outputs](/docs/object-model/reference-and-golden-outputs), [Run an evaluation](/docs/benchmark-evaluations/run-evaluation), and [Map external outputs](/docs/benchmark-evaluations/output-mapping). {% example-demo title="Target model outputs boundary" %} Case: A user asks whether a product is compatible with equipment they own. Candidate output: The assistant says the product will work. Interpretation: The benchmark failure belongs to the candidate output, but the reviewer still needs the case context and applicable rubric to explain why it failed. {% /example-demo %} ## Source confidence Code-backed: Benchmark Dataset output mapping and Evaluation Run detail distinguish imported output-only evidence from managed Run responses; review and Case-content sources preserve the Case boundary. Provider execution and customer serialization are outside this reference. ## Related task pages {% related-card-grid title="Related task pages" %} - [Run an evaluation](/docs/benchmark-evaluations/run-evaluation) - [Map external outputs](/docs/benchmark-evaluations/output-mapping) - [Connect model outputs](/docs/integrations/connect-model-outputs) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) {% /related-card-grid %} --- id: assets.cases title: Cases summary: Understand canonical project cases, their input and materials, and how they become members of benchmark datasets. kind: reference product_area: assets status: stable updated: 2026-08-22 canonical: /docs/assets/cases --- # Cases ## Definition A Case is a project-level situation used for expert contribution, benchmark coverage, or candidate evaluation. It has canonical input content and may include declared supporting materials. Cases live in the Assets pool and can be selected into one or more benchmark datasets. The Project Input Schema determines how the primary input and materials are represented. A benchmark snapshot determines which selected cases belong to one versioned evidence boundary. ## Fields, states, or lifecycle rules - Canonical primary input is stored under `content.input`. - Optional supporting values or artifacts are stored under `content.case_materials` according to the project's declared keys. - `record_content.case_view` is a rendered projection used for inspection and delivery; it is not a second editable payload. - Inputs can use plain-text, chat, or structured architecture as configured by the project. - Materials can include admitted artifact families and must satisfy any required-field and file-extension rules. - A project Case is not automatically part of every benchmark. Benchmark Datasets owns selection and snapshots. - Generated or imported Cases should be reviewed for clarity, source authority, and schema conformance before they are trusted as benchmark evidence. - Static case materials and executable Worlds remain separate. A document supplied to a Harness does not become a world merely because it affects execution. ## Case identity and change Treat the persisted case identity as opaque. Do not construct IDs in client code or documentation. When case content changes materially, benchmark interpretation must use a snapshot or version boundary that makes the selected content clear. Responses produced by a Harness are evaluation outputs attached to a Run. They are not the primary case input. Expert-authored acceptable examples can inform standards or contribution work, but the current evaluation contract should remain explicit about which candidate produced each response. {% example-demo title="Example: multimaterial case" %} A chat case asks whether an exception applies. Its required `current_policy` PDF and optional `account_history` table are stored as case materials admitted by Project Input Schema. The rendered case view presents the conversation and both materials. A benchmark snapshot selects the case, and a Run records the evaluated Harness response separately. {% /example-demo %} ## Source confidence Code-backed: the active Assets Cases route and case-content services define canonical input, case materials, and the rendered case view. Public import or export APIs are outside this reference unless separately documented. ## Related task pages {% related-card-grid title="Related task pages" %} - [Configure Project Input Schema](/docs/project-settings/input-schema) - [Work with Benchmark Datasets](/docs/benchmark-datasets) - [Manage benchmark coverage](/docs/coverage-management) {% /related-card-grid %} --- id: benchmark-evaluations.output-mapping title: Map External Evaluation Outputs summary: Import reference outputs, map them to immutable benchmark Cases and attempts, and inspect the resulting output-only Run. kind: task product_area: benchmark_evaluations status: stable updated: 2026-08-22 canonical: /docs/benchmark-evaluations/output-mapping --- # Map External Evaluation Outputs ## Prerequisites - External outputs for the exact Benchmark Version. - Durable Case IDs and, for repeated groups, an attempt-assignment plan. - Permission to create or update the output-only Run. Output mapping admits responses produced outside the managed Harness runtime as an output-only Run. Use upload or the displayed API path, then map every row to an immutable Case in the current Benchmark Version. ## Row contract The mapping template uses `case_id`, `input`, `context`, and `output`. Optional fields can carry latency, usage, and cost. `case_id` is the reliable join key; input and context help operators verify that the external row represents the intended immutable Case. For repeated Run Groups, assign an attempt explicitly or use automatic assignment when the incoming rows can be distributed unambiguously. Never combine two external attempts into one output simply to satisfy the configured sample count. ## Steps 1. Open Runs and start the external or reference-output mapping flow. 2. Download or inspect the template for the current Benchmark Version. 3. Populate exact Case IDs and outputs. Preserve the source system's telemetry only when it is measured. 4. Upload or submit through the displayed API workflow and review the preview. 5. Resolve unknown Cases, missing benchmark Cases, duplicates, or ambiguous attempt assignments. 6. Commit the mapping and inspect inserted, updated, missing, and unknown counts. 7. Follow evaluation progress and inspect the output-only Run from List. > Reference output boundary > > An imported output-only Run can be scored and inspected, but it is not a saved Harness Version. It cannot be activated, optimized in Improve, or used as a Harness column in Compare or Arena. ## Common mistakes - Inventing Case IDs or joining only on input text. - Reporting missing telemetry as zero. - Mapping current editable Cases instead of the immutable Benchmark Version. - Ignoring updated rows when the operation was expected to insert only. - Assuming a successful upload proves that Rubric evaluation is complete. ## Object and state changes Committing inserts or updates mapped output rows and creates or updates the scoped output-only Run and attempt assignment. It does not create a Harness Version or modify immutable Cases. ## Success criteria - Every admitted row maps to the intended Case and attempt. - Inserted, updated, missing, and unknown counts are understood. - Evaluation completion remains separate from upload completion. ## Common failure modes - Joining on text while ignoring Case IDs. - Overwriting an attempt unintentionally. - Presenting the reference Run as an executable candidate. ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Benchmark Evaluations](/docs/benchmark-evaluations) - [Dataset Snapshots](/docs/benchmark-datasets/snapshots) - [Connect model outputs](/docs/integrations/connect-model-outputs) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Related troubleshooting pages" %} - [Output mapping](/docs/troubleshooting/output-mapping) - [Missing outputs](/docs/troubleshooting/missing-outputs) - [Dataset upload](/docs/troubleshooting/dataset-upload) {% /related-card-grid %} ## Source confidence Code-backed: the current output-mapping modal and Runs workspace define the row template, immutable Case mapping, attempt handling, result counts, and output-only Run boundary. --- id: object-model.reference-outputs title: Reference and golden outputs summary: Understand Case-scoped example responses without treating them as generic approval state or candidate behavior. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/reference-and-golden-outputs --- # Reference and golden outputs ## Definition A golden or reference output is a Case-scoped example response available on supported review, mapping, or Snapshot surfaces. It can help a reviewer understand intended behavior for that Case, but it is not a global Policy, a saved Harness Version, or a generic customer approval state. Use this reference only where the product explicitly exposes a golden or reference response. Do not rename every expert answer, reviewed response, or imported candidate output as an “accepted output.” ## Fields, states, or lifecycle rules - Keep the reference response attached to the Case and evidence context that justifies it. - A candidate Run response does not become a reference response merely because it passed a Rubric. - Reviewer comments and Contribution rationale remain attributable review input, not output state. - A reference-output change can require a new Case, Snapshot, or Benchmark Version boundary when it affects governed interpretation. - Policies and Rubrics remain the reusable correctness standard; a reference output illustrates one Case. - This page does not define an export format or public output schema. ## Related objects Read reference outputs with [Outputs](/docs/object-model/outputs), [Cases](/docs/object-model/cases), [Correctness Governance](/docs/correctness-governance), and [Expert Contributions](/docs/expert-contributions). {% example-demo title="Case-scoped reference response" %} A compatibility Case includes a reference response that says the available source does not confirm support and directs the user to an explicit compatibility check. The response illustrates desired behavior for that Case. The reusable requirement still belongs in the applicable Policy and Rubric. {% /example-demo %} ## Source confidence Code-backed: output mapping, Snapshot artifacts, linked Policy Cases, and Contribution review surfaces expose golden or reference responses as Case-scoped evidence with explicit review or overwrite actions. The sources do not establish a universal accepted-output lifecycle. ## Related task pages {% related-card-grid title="Related task pages" %} - [Map external outputs](/docs/benchmark-evaluations/output-mapping) - [Review prepared Cases](/docs/coverage-management/case-review) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) {% /related-card-grid %}