# Object model Generated: 2026-09-13T04:33:03.628Z Source build: local Canonical docs: https://teammately.ai/docs --- id: object-model.overview title: Object model summary: Understand how project foundations, benchmark artifacts, contributions, evaluations, and improvement sessions connect. kind: reference product_area: reference status: stable updated: 2026-09-07 canonical: /docs/object-model --- # Object model Use this reference when a workflow depends on artifact ownership, identity, version history, or provenance. Teammately keeps reusable project foundations separate from benchmark-scoped evidence so a later reader can reconstruct what the agent knew, what was evaluated, who supplied judgment, and why a candidate was retained. > Preserve scope and version > > An artifact name alone is not sufficient evidence. Preserve its project or benchmark scope, version or snapshot identity, lifecycle state, and provenance whenever it affects evaluation or improvement. ## Definition A **Project** owns reusable foundations: the Project Agent Brief, Reference blocks, policies, rubrics, coverage facets, Cases, Harnesses, Review Screen configuration, Comparison Directions, and Project Input Schema. These objects can support more than one benchmark. A **Benchmark** owns a particular correctness boundary. Its workspace selects Cases into Dataset snapshots, organizes benchmark-scoped coverage work, requests Expert Contributions, and creates Benchmark versions. A Benchmark version is the fixed boundary against which a saved Harness version is evaluated. An **Evaluation Run** records the response, Rubric outcomes, mapping, settings, and metadata produced by one saved Harness Version against one Benchmark Version. It does not currently expose execution trajectories. An **Improvement Session** pins target evidence through a Goal Contract, evaluates candidates, retains canonical receipts and safe narrated session activity, and updates a Current frontier. The frontier is evidence-backed session state, not an unnamed replacement for the saved Harness. ## Decision checkpoint | Question | Object boundary to preserve | Common mistake | | --- | --- | --- | | What did the agent understand? | Project Agent Brief and Reference block generation | Treating the latest source file as the knowledge used by an earlier run | | What behavior was evaluated? | Case, Dataset snapshot, Benchmark version, and saved Harness version | Describing only a prompt and score | | What standard applied? | Policy, applicability, Rubric, and provenance | Letting a Rubric stand in for the Policy it checks | | Who supplied judgment? | Contribution, Task, Checkpoint, expert, and Contributed artifact | Treating an unattributed comment as governed truth | | Why was a candidate retained? | Improvement Session, Goal Contract, evaluation receipt, and Current frontier | Treating an unsaved draft as the evaluated candidate | ## Fields, states, or lifecycle rules ### Project foundations - **Project Agent Brief:** Published project understanding used to orient agents. - **Reference block:** Indexed knowledge with source and generation identity. - **Policy and Rubric:** Governed meaning and executable judgment criteria, with applicability and provenance. - **Coverage Facet:** Reusable Dimensions, Project Topics, and Case Construction Patterns that describe important behavior space. - **Case:** Canonical `content.input`, optional `content.case_materials`, metadata, and optional executable world reference. - **Harness:** An agent configuration with mutable Draft state and immutable saved versions used for evaluation. - **Project Input Schema:** The project-level architecture that governs canonical Case input and materials. ### Benchmark artifacts - **Dataset snapshot:** A reproducible selection and representation of benchmark Cases. - **Coverage Story:** Benchmark-scoped intent that connects coverage structure to concrete case work. - **Expert Contribution:** A benchmark-scoped request containing Tasks, context, statuses, and optional Checkpoints. - **Contributed artifact:** A policy, Rubric, Case, or coverage observation supplied through a Contribution with attributable provenance. - **Benchmark version:** The fixed evaluation boundary used by Runs and Improvement Sessions. - **Evaluation Run:** One execution with candidate, benchmark, response, Rubric outcomes, settings, mapping, and metadata identity. - **Improvement Session:** A goal-directed candidate exploration process with pinned evidence, receipts, trajectories, and frontier state. {% artifact-map title="How correctness artifacts connect" %} {% /artifact-map %} ## Static materials and executable worlds Case materials and executable worlds are intentionally different. `content.case_materials` holds canonical material data or references that can accompany the input. A `world_instance_ref` identifies an executable or queryable environment with a separate capability and lifecycle boundary. Rendering either for inspection does not create another authoring source. {% example-demo title="From contribution to frontier" %} A benchmark Contribution asks a procurement specialist to resolve three ambiguous Cases. The expert confirms a Policy and binary Rubric at a Checkpoint. The artifacts are reconciled into Correctness Governance while retaining their Contribution provenance. The Cases enter a Dataset snapshot and the standard enters Benchmark version 6. An Evaluation Run tests Harness version 11 and exposes two failures. An Improvement Session pins those failures, evaluates versions 12 and 13, and retains version 13 in the Current frontier with canonical evaluation receipts. {% /example-demo %} ## Source confidence Code-backed: current navigation establishes project and benchmark scopes; canonical case-content code establishes the input and materials boundary; the active Improve session contract establishes Goal Contract, candidate, receipt, and frontier concepts. This page describes public object semantics, not a public API or export schema. ## Related task pages {% related-card-grid title="Related task pages" %} - [First correctness loop](/docs/operating-manual/first-correctness-loop) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation) - [Start an Improvement Session](/docs/improve/start-improvement-session) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships) - [Cases](/docs/assets/cases) - [Harnesses](/docs/assets/harnesses) - [Glossary](/docs/reference/glossary) {% /related-card-grid %} --- id: object-model.applicability-logic title: Applicability logic summary: Explain when a policy or rubric should be used for a case, output, or coverage segment. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/applicability-logic --- # Applicability logic ## Definition Applicability logic is the boundary that decides whether a policy or rubric should be considered for a specific case, output, or coverage segment. It prevents a good standard from being applied to the wrong behavior. Use this reference when a benchmark result is hard to explain because a standard seems relevant in some cases but not others. The question is not only whether the policy is correct; it is whether the policy was eligible to judge that output. ## Fields, states, or lifecycle rules - Applicability sits between the case/output and the policy/rubric that may judge it. - Weak applicability makes benchmark failures noisy: a candidate can fail a good rubric on a case where the rule should not have applied. - Strong applicability names the behavior condition, source context, or case segment that brings the standard into scope. - Applicability changes can make old benchmark evidence stale because the same output may be judged by a different standard boundary. - This page explains the public object relationship, not a public rule language, API schema, or export contract. ## Related objects Applicability logic should be read with [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Cases](/docs/object-model/cases), and [Coverage Dimensions](/docs/object-model/coverage-dimensions). Use the Correctness Governance workflow to write the boundary and troubleshooting when the observed Case set is wrong. {% example-demo title="Applicability logic boundary" %} Raw case: A user asks whether a product works with equipment they already own. Policy: Compatibility claims require explicit source support. Applicability logic: The policy applies only when the answer recommends, validates, or compares a product for a concrete use context. Benchmark interpretation: If the case only asks for a product description, the compatibility rubric should not judge it. If the answer claims the product will work with the user's equipment, the rule is in scope. {% /example-demo %} ## Source confidence Code-backed: Policy and Rubric types and their detail routes expose applicability fields and the links among governed standards, Cases, and evaluation checks. This page describes those product relationships; it does not define a portable rule language. ## Related task pages {% related-card-grid title="Related task pages" %} - [Work with Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Fix weak applicability logic](/docs/troubleshooting/weak-applicability-logic) - [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.benchmarks title: Benchmarks summary: Understand a Benchmark as the durable program that owns benchmark-scoped coverage, evidence boundaries, evaluations, and improvement work. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/benchmarks --- # Benchmarks ## Definition A Benchmark is the durable project object for one intended evaluation program. It owns benchmark-scoped work across Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve while its selected Cases, governed standards, and candidate systems evolve. The current Benchmark Dataset is editable. A Dataset Snapshot freezes selected Case membership, and a Benchmark Version provides the immutable boundary consumed by Runs. A Benchmark is therefore not a Snapshot, Benchmark Version, Run, or score. ## Fields, states, or lifecycle rules - The Benchmark identity persists across changes to its current Dataset, coverage work, standards, and Harness candidates. - Benchmark Datasets owns selected Cases and immutable Dataset Snapshots. - A Benchmark Version fixes the evidence boundary used by a Run. - Benchmark membership should be shaped by coverage work, not by whichever Cases are easiest to run. - A Run result is weak if the Benchmark Version and saved Harness Version behind it are unclear. - This page documents object semantics, not public execution, export, rate-limit, or API guarantees. ## Related objects Benchmarks should be read with [Cases](/docs/assets/cases), [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Coverage Engineering](/docs/coverage-engineering), and [Benchmark Evaluations](/docs/benchmark-evaluations). Use [Run an evaluation](/docs/benchmark-evaluations/run-evaluation) when the next step is execution. {% example-demo title="Benchmarks boundary" %} Raw case: A team refreshes coverage after finding unsupported compatibility claims. Benchmark version: The refreshed version includes new unsupported-claim cases and the approved compatibility rubric. Run: The candidate is evaluated against that version. Interpretation: If the score drops, reviewers can see that the benchmark became harder instead of assuming the candidate behavior changed. {% /example-demo %} ## Source confidence Code-backed: the Benchmark type and workspace establish durable Benchmark identity; Benchmark Datasets → Snapshots establishes immutable Dataset boundaries; the evaluation-runs route consumes a specific Benchmark Version. The public object definition does not imply an execution or export API. ## Related task pages {% related-card-grid title="Related task pages" %} - [Benchmarks](/docs/coverage-engineering/benchmarks) - [Benchmarks and versions](/docs/concepts/benchmarks-and-versions) - [Benchmark Evaluations](/docs/benchmark-evaluations) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.case-pool title: Case Pool surface summary: Understand Case Pool as the Assets view for reusable project Cases, candidates, and explicit Benchmark selection. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/case-pool --- # Case Pool surface ## Definition **Case Pool** is a tab under **Assets → Cases**, not a separate Case object type. It shows reusable project Cases and candidate material that can be inspected before explicit selection into a Benchmark Dataset. Use this reference when deciding whether a Case should stay in the project collection, be prepared further, or be selected for a Benchmark. **Sourcing Tasks** is the adjacent job and audit surface; it is not another Case state. ## Fields, states, or lifecycle rules - Case Pool can contain useful Cases that are not selected into any Benchmark. - Candidate and in-use labels describe curation and membership context; they do not create a different persisted Case type. - Coverage dimensions and ontology values help explain what behavior the pool represents. - Adding a Case to a current Benchmark Dataset changes mutable membership. A new Dataset Snapshot is required to freeze that change as an evidence boundary. - This page does not define public upload schemas, export contracts, or retention behavior. ## Related objects Read Case Pool with [Cases](/docs/object-model/cases), [Candidate and In-Use Cases](/docs/coverage-engineering/candidate-and-in-use-cases), [Coverage Dimensions](/docs/object-model/coverage-dimensions), and [Benchmark Dataset Cases](/docs/benchmark-datasets/cases). Use [Case Pool](/docs/coverage-engineering/case-pool) for operating instructions. {% example-demo title="Case pool boundary" %} Raw case: A new support transcript shows a refund exception question that reviewers have not seen before. Case pool state: The example is a candidate case because it may close an exception-handling coverage gap. Interpretation: It should not affect benchmark scores until source context, output identity, coverage labels, and benchmark membership are clear. {% /example-demo %} ## Source confidence Code-backed: the Assets Cases routes expose Case Pool and Sourcing Tasks tabs, candidate and in-use views, and an explicit add-to-Benchmark action. Case Pool is therefore a curation surface, not automatic Benchmark membership, a separate Case schema, or a public storage contract. ## Related task pages {% related-card-grid title="Related task pages" %} - [Case Pool](/docs/coverage-engineering/case-pool) - [Candidate and In-Use Cases](/docs/coverage-engineering/candidate-and-in-use-cases) - [Cases](/docs/object-model/cases) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.cases title: Cases summary: Understand cases as the situations Teammately uses to represent important AI behavior. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/cases --- # Cases ## Definition Cases are the behavior situations Teammately uses for review, coverage, and benchmark evidence. A case should preserve the input, source context, metadata, and version boundary needed to understand what behavior is being judged. Use this reference when a workflow depends on whether an example is reviewable, benchmark-ready, stale, duplicated, or missing the context a reviewer needs. ## Fields, states, or lifecycle rules - A case is not the same thing as an output; the case holds the situation and context, while the output is the behavior being judged. - A case can be useful for coverage planning before it is ready for benchmark use. - Case versions matter when source context, metadata, or expected behavior changes. - A benchmark result is weak if the failed output cannot be traced back to the exact case version. - Exact import schema and export behavior belong to source-backed integration pages, not this object reference. ## Related objects Cases should be read with [Outputs](/docs/object-model/outputs), [Case versions](/docs/governance/case-versioning), [Case pool](/docs/object-model/case-pool), and [Benchmarks](/docs/object-model/benchmarks). Use [Importing cases](/docs/operating-manual/import-and-prepare-cases) when the next step is bringing source examples into Teammately. {% example-demo title="Cases boundary" %} Raw record: A transcript includes a customer question, assistant answer, account context, and later human correction. Case: The reviewable unit preserves the customer question and account context. Output: The candidate answer is attached separately so reviewers know which behavior is being judged. Interpretation: If the candidate fails later, the run result can point to the exact case and output rather than to a vague transcript. {% /example-demo %} ## Source confidence Code-backed: Case view types and the Cases asset route establish the product object and its presentation; internal Case-content types establish how input and contextual material remain separate from outputs. These implementation sources support object semantics, not a customer-facing serialization contract. ## Related task pages {% related-card-grid title="Related task pages" %} - [Case model overview](/docs/assets) - [Case Pool](/docs/coverage-engineering/case-pool) - [Case pool](/docs/object-model/case-pool) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.coverage-dimensions title: Coverage dimensions summary: Organize cases by the behavior axes that matter to product correctness. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/coverage-dimensions --- # Coverage dimensions ## Definition Coverage dimensions are the axes used to explain what behavior space a case set represents. A dimension can describe source freshness, request type, risk level, product area, policy boundary, or another classification that matters for review and benchmark interpretation. Use this reference when a benchmark score is not enough and the team needs to ask which kinds of behavior are represented or missing. ## Fields, states, or lifecycle rules - Dimensions should describe meaningful behavior axes, not arbitrary tags. - Ontology values should keep each dimension's labels consistent enough for coverage planning. - Coverage dimensions can reveal untested segments even when aggregate benchmark scores look strong. - Changing a dimension schema can change how old benchmark evidence is interpreted. - This page describes object semantics, not a public schema contract. ## Related objects Coverage dimensions should be read with [Ontology](/docs/object-model/ontology), [Case pool](/docs/object-model/case-pool), [Benchmarks](/docs/object-model/benchmarks), and [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology). {% example-demo title="Coverage dimensions boundary" %} Dimension: Source support level. Ontology values: Explicit support, implied support, conflicting source, no source. Interpretation: A compatibility benchmark can show whether failures concentrate in cases where the source does not explicitly support the claim. {% /example-demo %} ## Source confidence Code-backed: Dimension and classification types plus the Dimensions and Ontology list and detail routes establish Dimensions, ontology values, origin, examples, and Case classification. Whether a Dimension is meaningful or complete remains a coverage-design judgment. ## Related task pages {% related-card-grid title="Related task pages" %} - [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology) - [Dimensions and ontology](/docs/concepts/dimensions-and-ontology) - [Ontology](/docs/object-model/ontology) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.ontology title: Ontology summary: Use ontology values to classify cases consistently within each coverage dimension. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/ontology --- # Ontology ## Definition Ontology values are the controlled labels used inside a coverage dimension. They keep case classification consistent enough for teams to plan coverage, compare benchmark segments, and explain gaps. Use this reference when free-form tags are making coverage hard to read or when a benchmark segment needs stable vocabulary before results are trusted. Ontology values remain project-scoped foundations. ## Fields, states, or lifecycle rules - Ontology values belong to dimensions; they should not drift into unrelated free-form tags. - Values should be specific enough to separate meaningful behavior, but not so narrow that coverage becomes fragmented. - Changing ontology values can change segment-level benchmark interpretation. - Ontology should support coverage decisions, not replace expert judgment or policies. - This page does not define a public taxonomy API. ## Related objects Ontology should be read with [Coverage dimensions](/docs/object-model/coverage-dimensions), [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology), and [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema). {% example-demo title="Ontology boundary" %} Dimension: Source support level. Ontology values: Explicit support, implied support, conflicting source, no source. Interpretation: Segment results are easier to trust because the team knows which support condition each case represents. {% /example-demo %} ## Source confidence Code-backed: Dimension types and the Dimension settings, ontology-row, and proposal components establish ontology members as labels owned by a Dimension and used for classification. Vocabulary quality and cross-reviewer consistency require project review. ## Related task pages {% related-card-grid title="Related task pages" %} - [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology) - [Coverage dimensions](/docs/object-model/coverage-dimensions) - [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.outputs title: Outputs summary: Distinguish managed Run responses, imported output-only Runs, and Case-scoped reference outputs. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/outputs --- # Outputs Understand the outputs Teammately reviews, compares, and turns into correctness evidence. ## Definition An output is response evidence associated with a Case, but its identity depends on the workflow that produced it. A managed Evaluation Run generates a response from a saved Harness Version. An imported output-only Run maps externally produced responses to immutable Case IDs. A Case may also expose a golden or reference response where the current surface supports it. Do not collapse these into one generic “accepted output” lifecycle. Imported outputs are not Harness Versions, Run responses are not automatically reference answers, and reviewer comments are not outputs. ## Fields, states, or lifecycle rules - A managed Run response is bound to the Run, saved Harness Version, Benchmark Version, and Case. - An imported output-only Run joins external rows to immutable Case IDs and retains imported candidate metadata. - Output-only Runs can be scored and inspected, but cannot be activated, optimized in Improve, or used as Harness columns in Compare or Arena. - A golden or reference output is Case-scoped example evidence where exposed; it does not become a global Policy. - Mapping errors can make evidence misleading even when the external response data is unchanged. - This page documents object boundaries, not model execution, provider behavior, or public output schemas. ## Related objects Read outputs with [Cases](/docs/object-model/cases), [Reference and golden outputs](/docs/object-model/reference-and-golden-outputs), [Run an evaluation](/docs/benchmark-evaluations/run-evaluation), and [Map external outputs](/docs/benchmark-evaluations/output-mapping). {% example-demo title="Target model outputs boundary" %} Case: A user asks whether a product is compatible with equipment they own. Candidate output: The assistant says the product will work. Interpretation: The benchmark failure belongs to the candidate output, but the reviewer still needs the case context and applicable rubric to explain why it failed. {% /example-demo %} ## Source confidence Code-backed: Benchmark Dataset output mapping and Evaluation Run detail distinguish imported output-only evidence from managed Run responses; review and Case-content sources preserve the Case boundary. Provider execution and customer serialization are outside this reference. ## Related task pages {% related-card-grid title="Related task pages" %} - [Run an evaluation](/docs/benchmark-evaluations/run-evaluation) - [Map external outputs](/docs/benchmark-evaluations/output-mapping) - [Connect model outputs](/docs/integrations/connect-model-outputs) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) {% /related-card-grid %} --- id: object-model.policies title: Policies summary: Define policies as reusable statements of what correct AI behavior requires. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/policies --- # Policies ## Definition Policies are reusable statements of what correct behavior requires. They preserve expert judgment as a governed standard that can later be tested by applicability logic and rubrics. Use this reference when a team needs to know whether a rule is a draft suggestion, an approved correctness standard, or a standard whose version may affect benchmark evidence. ## Fields, states, or lifecycle rules - A policy names the rule; a rubric tests the rule. - A policy needs applicability logic before the team can know which cases it should judge. - Policy approval state matters before benchmark use. - Policy revisions can make older benchmark evidence stale or require comparison notes. - This page does not define legal, compliance, retention, or external policy-management guarantees. ## Related objects Policies should be read with [Applicability logic](/docs/object-model/applicability-logic), [Rubrics](/docs/object-model/rubrics), [Policy versions](/docs/governance/policy-versioning), and [Human Approval Boundaries](/docs/governance/human-approval-boundaries). Use [Create a policy](/docs/correctness-governance/policies-and-rubrics) for the operating workflow. {% example-demo title="Policies boundary" %} Expert judgment: Answers should not claim compatibility unless source data explicitly supports the claim. Policy: Compatibility claims require explicit source support. Interpretation: The policy explains the standard; linked applicability and rubrics decide when and how a specific output is judged. {% /example-demo %} ## Source confidence Code-backed: Policy types and list/detail routes expose identity, description, applicability, linked Cases and Rubrics, version facts, and approval state. The product object is a governed correctness standard, not an external legal-policy system. ## Related task pages {% related-card-grid title="Related task pages" %} - [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Applicability logic](/docs/object-model/applicability-logic) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.reference-outputs title: Reference and golden outputs summary: Understand Case-scoped example responses without treating them as generic approval state or candidate behavior. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/reference-and-golden-outputs --- # Reference and golden outputs ## Definition A golden or reference output is a Case-scoped example response available on supported review, mapping, or Snapshot surfaces. It can help a reviewer understand intended behavior for that Case, but it is not a global Policy, a saved Harness Version, or a generic customer approval state. Use this reference only where the product explicitly exposes a golden or reference response. Do not rename every expert answer, reviewed response, or imported candidate output as an “accepted output.” ## Fields, states, or lifecycle rules - Keep the reference response attached to the Case and evidence context that justifies it. - A candidate Run response does not become a reference response merely because it passed a Rubric. - Reviewer comments and Contribution rationale remain attributable review input, not output state. - A reference-output change can require a new Case, Snapshot, or Benchmark Version boundary when it affects governed interpretation. - Policies and Rubrics remain the reusable correctness standard; a reference output illustrates one Case. - This page does not define an export format or public output schema. ## Related objects Read reference outputs with [Outputs](/docs/object-model/outputs), [Cases](/docs/object-model/cases), [Correctness Governance](/docs/correctness-governance), and [Expert Contributions](/docs/expert-contributions). {% example-demo title="Case-scoped reference response" %} A compatibility Case includes a reference response that says the available source does not confirm support and directs the user to an explicit compatibility check. The response illustrates desired behavior for that Case. The reusable requirement still belongs in the applicable Policy and Rubric. {% /example-demo %} ## Source confidence Code-backed: output mapping, Snapshot artifacts, linked Policy Cases, and Contribution review surfaces expose golden or reference responses as Case-scoped evidence with explicit review or overwrite actions. The sources do not establish a universal accepted-output lifecycle. ## Related task pages {% related-card-grid title="Related task pages" %} - [Map external outputs](/docs/benchmark-evaluations/output-mapping) - [Review prepared Cases](/docs/coverage-management/case-review) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) {% /related-card-grid %} --- id: object-model.conversation-inputs title: Represent conversations in Cases summary: Preserve multi-message context inside canonical Case input without inventing a separate Conversation Case object. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/represent-conversations-in-cases --- # Represent conversations in Cases ## Definition A canonical Case input can represent a multi-message interaction when correctness depends on prior turns, accumulated context, or state carried across an exchange. “Conversation” describes the shape of the Case input; it is not a separate first-class object or a public schema guarantee. Use this reference when review or evaluation depends on exactly which messages were available before the response being judged. ## Fields, states, or lifecycle rules - Preserve the message order and roles needed to interpret the Case. - Identify the response being judged separately from prior user or assistant messages. - Store supporting documents or structured material according to the Project Input Schema instead of flattening everything into message text. - Removing, reordering, or summarizing material can change the Case and should follow the normal Case version boundary. - Output mapping joins an external response to the Case ID; it does not repair missing conversational context. - This page does not define a public conversation serialization contract. ## Related objects Read this representation with [Cases](/docs/object-model/cases), [Outputs](/docs/object-model/outputs), [Project Input Schema](/docs/project-settings/input-schema), and [Metadata and Context](/docs/reference/metadata-and-context). {% example-demo title="Plan context carried across turns" %} A user first identifies an enterprise plan, then asks whether a feature is available. The later response can only be judged correctly if the earlier plan message remains part of the canonical Case input. If that message is removed, create a new Case version rather than trying to restore it during output mapping. {% /example-demo %} ## Source confidence Code-backed: Case-content services support structured project input and material fields, while record presentation and Case-view types preserve the context used for review. The sources support conversation-shaped Case input but not a distinct Conversation Case object or public import schema. ## Related task pages {% related-card-grid title="Related task pages" %} - [Configure the Project Input Schema](/docs/project-settings/input-schema) - [Import and prepare Cases](/docs/operating-manual/import-and-prepare-cases) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) - [Troubleshoot unclear Cases](/docs/troubleshooting/unclear-cases) {% /related-card-grid %} --- id: object-model.rubrics title: Rubrics summary: Turn policies into binary, reviewable checks that explain pass and fail evidence. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/rubrics --- # Rubrics ## Definition Rubrics turn policies into reviewable checks for a specific output. A good rubric makes pass, fail, or incomplete evidence explainable without replacing the policy it tests. Use this reference when a benchmark failure needs to show exactly which condition a candidate output satisfied or violated. ## Fields, states, or lifecycle rules - Rubrics should be tied to a policy and an applicability boundary. - A binary rubric should be narrow enough to judge one output clearly. - Linked Cases make the intended boundary concrete; they do not replace the Rubric's applicability rule. - Overlapping rubrics can make failures hard to interpret. - Rubric versions matter when pass/fail logic changes between benchmark runs. - This page describes rubric semantics, not a public scoring API. ## Related objects Rubrics should be read with [Policies](/docs/object-model/policies), [Applicability logic](/docs/object-model/applicability-logic), [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics), and [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics). {% example-demo title="Rubrics boundary" %} Policy: Compatibility claims require explicit source support. Rubric: Pass only if the answer cites explicit compatibility support or states that compatibility is uncertain. Interpretation: A candidate output that simply says "yes, it works" fails because the rubric names the missing evidence. {% /example-demo %} ## Source confidence Code-backed: Rubric types and list/detail routes expose criteria, Policy and Case links, applicability, and versioned evaluation context. The reference explains product semantics and does not promise a public scoring schema. ## Related task pages {% related-card-grid title="Related task pages" %} - [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics) - [Create a rubric](/docs/correctness-governance/policies-and-rubrics) - [Policies](/docs/object-model/policies) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.versions-staleness-resolution title: Versions, staleness, and resolution summary: Track how correctness objects evolve and how teams resolve conflicting evidence. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/versions-staleness-and-resolution --- # Versions, staleness, and resolution ## Definition Versions, staleness, and resolution describe how correctness artifacts evolve without making old evidence ambiguous. Cases, Policies, Rubrics, Benchmark Versions, supported Case-scoped reference outputs, and customer-owned review context can change at different times; version boundaries explain which evidence belongs to which state. Use this reference when a result changed unexpectedly, a policy was revised, a case was refreshed, or reviewers need to know whether older benchmark evidence still applies. ## Fields, states, or lifecycle rules - Versions preserve what changed and what evidence was produced before the change. - Staleness means older evidence may no longer reflect the current case, standard, coverage, or candidate boundary. - Resolution work should name whether the fix belongs to a case, output, policy, rubric, coverage plan, benchmark version, or run metadata. - Comparisons are weak when artifact versions are hidden. - This page describes public object semantics, not retention, audit-log completeness, or compliance guarantees. ## Related objects Versions, staleness, and resolution should be read with [Versioning and Staleness](/docs/governance/versioning-and-staleness), [Benchmark versioning](/docs/governance/benchmark-versioning), [Case versioning](/docs/governance/case-versioning), and [Policy Conflicts and Revisions](/docs/governance/conflict-resolution). {% example-demo title="Versions, staleness, and resolution boundary" %} State change: Reviewers revise a compatibility policy after finding unsupported-claim failures. Benchmark evidence: Runs against the old policy remain interpretable, but they should not be summarized as current evidence without naming the old policy version. Interpretation: The resolution note explains whether to rerun, revise the benchmark version, or preserve the old result as historical context. {% /example-demo %} ## Source confidence Code-backed: Benchmark, Policy, and Rubric types carry version facts; Benchmark Datasets → Snapshots and Policy activity preserve named historical boundaries. Cross-object staleness and conflict resolution are explicit review decisions rather than a universal automatic state. ## Related task pages {% related-card-grid title="Related task pages" %} - [Versioning and Staleness](/docs/governance/versioning-and-staleness) - [Conflict Resolution](/docs/governance/conflict-resolution) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: object-model.workspaces-projects-systems title: Workspaces, projects, and target systems summary: Model organizational boundaries, product boundaries, and the AI system being governed. kind: reference product_area: object_model status: stable updated: 2026-09-07 canonical: /docs/object-model/workspaces-projects-and-target-systems --- # Workspaces, projects, and target systems ## Definition Workspaces, projects, and target systems define where organizational access, product-specific correctness work, and the AI behavior under evaluation are separated. A workspace groups people and administration; a project holds the cases, standards, coverage, and benchmark evidence for a specific target behavior. Use this reference when a reader needs to know whether an artifact belongs to an organization boundary, a project boundary, or the target system being evaluated. ## Fields, states, or lifecycle rules - Workspace boundaries should not be used to infer project-level correctness decisions. - Project boundaries keep cases, policies, rubrics, coverage, benchmark versions, and review context tied to a specific target behavior. - Target-system identity matters for benchmark-level run metadata and comparison interpretation. - Cross-project reuse should not imply cross-project approval. - This page does not make billing, tenancy, deployment, or compliance claims. ## Related objects Workspaces, projects, and target systems should be read with [Product map](/docs/getting-oriented/product-map), [Product boundaries](/docs/introduction/product-boundaries), [Permissions](/docs/reference/permissions), and [Run metadata](/docs/benchmark-evaluations/run-metadata). {% example-demo title="Workspaces, projects, and target systems boundary" %} Scenario: The same company evaluates a support assistant and an internal policy-search assistant. Project boundary: Each assistant has its own cases, policies, rubrics, benchmark versions, and benchmark-level run metadata. Interpretation: A passing benchmark in the support project should not imply the policy-search assistant has approved evidence. {% /example-demo %} ## Source confidence Code-backed: Project and workspace types establish organization and Project identity; member settings expose Project participation; benchmark Run presentation identifies the evaluated target boundary. These sources do not establish billing, deployment, or tenancy guarantees. ## Related task pages {% related-card-grid title="Related task pages" %} - [Workspaces and projects](/docs/concepts/workspaces-projects) - [Product map](/docs/getting-oriented/product-map) - [Product boundaries](/docs/introduction/product-boundaries) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: orientation.key-objects title: Key objects and relationships summary: Understand how project foundations, contributions, datasets, evaluations, and improvement artifacts connect. kind: concept product_area: reference status: stable updated: 2026-09-07 canonical: /docs/getting-oriented/key-objects-and-relationships --- # Key objects and relationships Teammately's evidence is trustworthy when a reader can move from project understanding and specialist authority to the exact Case, Benchmark version, Harness version, Run, and Improvement Session involved. This page gives the shared object graph. > Artifact state matters > > An object name is not enough. Preserve project or benchmark scope, version or snapshot identity, provenance, and lifecycle state whenever the object supports evaluation or improvement. ## Definition A **Project Agent Brief** and published **Reference block** give agents project understanding. **Project Input Schema** governs canonical **Case** input and materials. **Dimensions**, **Project Topics**, and Case Construction Patterns define reusable coverage structure. A saved **Harness version** identifies an executable candidate. A benchmark selects Cases into a **Dataset snapshot** and combines them with governed **Policies** and **Rubrics** through a **Benchmark version**. An **Expert Contribution** requests specialist judgment through one or more Tasks and Checkpoints. Its **Contributed artifact** can become a policy, rubric, case, or coverage observation while retaining provenance. A **Run** evaluates a saved Harness Version against a Benchmark Version. Its response, Rubric outcomes, settings, mapping, and metadata form evaluation evidence. An **Improvement Session** pins target evidence through a Goal Contract, creates or receives candidates, records evaluation receipts and safe session narration, and maintains a **Current frontier**. ## Decision checkpoint | Object | Scope | Relationship that must remain visible | | --- | --- | --- | | Project Agent Brief / Reference block | Project | What agents understood and which source generation was available | | Case / Harness version | Project | Which reusable asset and exact candidate state was selected | | Contribution / Checkpoint | Benchmark | Which expert supplied or confirmed the judgment | | Policy / Rubric | Project governance | Which authority, applicability, cases, and provenance support it | | Dataset snapshot / Benchmark version | Benchmark | Which cases and correctness boundary define evidence | | Run | Benchmark version | Which Harness, settings, mapping, and metadata produced results | | Improvement Session / Current frontier | Benchmark version | Which goal and evaluation receipts justify retained candidates | {% artifact-map title="How correctness artifacts connect" %} {% /artifact-map %} ## Static materials and executable worlds Canonical Case content separates `content.input` from optional `content.case_materials`. Static execution support uses case-material references. A `world_instance_ref` represents an executable or queryable environment and follows a separate capability and lifecycle boundary. The rendered case view helps people and adapters inspect canonical content; it does not create another authoring source. ## Provenance across scopes Project assets can be reused across benchmarks, while dataset snapshots, Contributions, Runs, and Improvement Sessions remain benchmark-scoped. Materializing a contributed policy moves its governed owner to project scope without erasing the benchmark Contribution that supplied it. Evaluating a candidate records the saved Harness version rather than whichever Draft is currently open. {% example-demo title="Contribution to frontier" %} An Expert Contribution confirms a source-authority Policy and Rubric from selected Cases. The Cases enter a Dataset snapshot and the standard enters a Benchmark version. A Run evaluates Harness version 8 and exposes three failures. An Improvement Session pins those failures, evaluates versions 9 and 10, and retains version 10 in the Current frontier with canonical evaluation receipts. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [First correctness loop](/docs/operating-manual/first-correctness-loop) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Start an Improvement Session](/docs/improve/start-improvement-session) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Object model](/docs/object-model) - [Project Input Schema](/docs/project-settings/input-schema) - [Product map](/docs/getting-oriented/product-map) {% /related-card-grid %} ## Source confidence Code-backed: active navigation, canonical case contracts, Contribution surfaces, versioned evaluation routes, and Improvement Session contracts support this object graph. --- id: intro.correctness-lifecycle title: The Teammately correctness lifecycle summary: Follow specialist AI work from project foundations through coverage, elicitation, construction, evaluation, and improvement. kind: concept product_area: introduction status: stable updated: 2026-08-22 canonical: /docs/introduction/correctness-lifecycle --- # The Teammately correctness lifecycle The correctness lifecycle describes how a team turns domain knowledge into an improving specialist AI system. It begins with reusable project foundations, narrows into a benchmark workspace, and cycles through coverage, expert contribution, evaluation, and improvement without losing the evidence that explains each change. ## Definition The lifecycle has three scopes: 1. **Project foundation.** Define Project Context and Reference Materials, configure the Project Input Schema, govern policies and rubrics, establish Coverage Facets, and manage reusable Assets. Reusable Review Screens and Comparison Directions are authored under Assets; Contribution-specific agent behavior and asset selection happen in benchmark work. 2. **Benchmark work.** Select a benchmark dataset and snapshot, manage coverage, request expert contributions, and bind the work to a benchmark version. 3. **Evaluation and improvement.** Evaluate saved harness versions, inspect cases and rubric results, compare candidates, and start an Improvement Session from pinned evidence. The public five-capability model runs through these scopes. Coverage Engineering shapes representation. Correctness Elicitation captures judgment. Weave constructs the challenge set. Trialground produces evaluation evidence. Coevolve turns that evidence into bounded candidate work. ## Decision checkpoint | Current condition | Next lifecycle action | Boundary to preserve | | --- | --- | --- | | Agents lack the project purpose or controlling knowledge | Complete Agent Setup | Project Context and Reference Materials remain distinct from governed policies | | Case shape is ambiguous | Configure Project Input Schema | `content.input` and case materials follow one declared architecture | | Coverage exists but correctness is tacit | Request an Expert Contribution | Human responses remain attributable before materialization | | Cases and standards are ready | Create or select a benchmark snapshot and evaluate a saved Harness version | The run stays bound to exact versions and settings | | Evaluation evidence exposes a candidate weakness | Start an Improvement Session | The Goal Contract and starting evidence remain pinned | | Candidate exploration exposes a benchmark gap | Return to coverage, standards, or cases | Do not interpret missing evidence as candidate failure | ## Why ordering matters Running evaluations before the benchmark has deliberate coverage can produce precise but misleading results. Writing rubrics without expert-grounded policies can turn vague preferences into brittle checks. Generating cases without a Project Input Schema can create records that reviewers or harnesses interpret differently. Starting improvement from an aggregate score can hide the cases and standards that actually justify a change. The lifecycle prevents those shortcuts by giving each artifact an owner and scope. Project foundations are reusable across benchmarks. Benchmark datasets and versions define the evidence boundary. Contributions provide human authority. Runs and comparisons expose candidate behavior. Improvement Sessions retain the chronology between a goal, a proposed change, and its evaluation receipt. ## How learning returns to the system The lifecycle closes when new evidence changes an upstream artifact. A failed run may show that a policy is too broad, a rubric is ambiguous, a case lacks required material, or a coverage tuple is absent. An expert can contribute a new policy, rubric, case, or coverage observation. An Improvement Session can test a harness candidate while also reporting missing correctness or coverage. Each return path should name the responsible artifact. Simultaneously changing cases, standards, harness code, and evaluation settings makes the next result difficult to explain. {% example-demo title="Specialist procurement assistant" %} The team configures structured procurement inputs and indexes its buying rules. Coverage Engineering maps product category, contract state, and exception type. An expert contribution resolves when an exception requires escalation and materializes the policy and rubric. Weave prepares cases with conflicting contract material. Trialground evaluates a saved harness version and exposes failures on expired agreements. Coevolve tests a source-date validation change while retaining the original benchmark evidence. {% /example-demo %} ## Where the lifecycle stops Teammately makes correctness work inspectable and reusable. It does not determine the customer's downstream deployment, operational, or governance action. The product supplies governed artifacts, benchmark evidence, and improvement history so the accountable team can make that decision with a clear record. ## Related workflows {% related-card-grid title="Related workflows" %} - [First correctness loop](/docs/operating-manual/first-correctness-loop) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Project Input Schema](/docs/project-settings/input-schema) - [Expert Contributions](/docs/expert-contributions) - [Improvement Sessions](/docs/improve) {% /related-card-grid %} ## Source confidence Doctrine-backed: this lifecycle joins the current public capability model to code-backed product scopes. Linked pages define individual object and state behavior. --- id: assets.overview title: Assets summary: Manage reusable project cases, worlds, project tools, harnesses, weights, comparison directions, and review screens before selecting them for benchmark work. kind: concept product_area: assets status: stable updated: 2026-09-07 canonical: /docs/assets --- # Assets Assets is the project-level pool for cases, worlds, project tools, harnesses, weights, Comparison Directions, and Review Screens. Assets are managed once at project scope and selected for use in a specific benchmark rather than being recreated inside every benchmark workspace. ## Definition The active tabs are **Cases**, **Worlds**, **Project Tools**, **Harnesses**, **Weights**, **Comparison Directions**, and **Review Screens**. Cases provide the canonical situations evaluated or reviewed. Harnesses provide executable candidate implementations with Draft and saved Versions. Comparison Directions guide comparative output variation, and Review Screens provide reusable expert-facing presentation templates. Worlds, Project Tools, and Weights are visible categories whose current pages expose empty states rather than creation or lifecycle controls. Assets is distinct from Benchmark Datasets. The project pool answers what is available to the project. A benchmark dataset answers which cases and snapshot define one benchmark's evidence boundary. ## Decision checkpoint | Need | Asset or workspace | Boundary | | --- | --- | --- | | Create or inspect a reusable situation | Assets → Cases | Case content follows Project Input Schema | | Edit candidate code or prompt logic | Assets → Harnesses | A Draft must be saved as an exact version before evaluation | | Select cases for a benchmark | Benchmark Datasets | Selection and snapshot are benchmark-scoped | | Supply static documents or values to a case | Case materials | Static support is not a World | | Inspect planned environment assets | Worlds | Current product exposes the category but no public lifecycle yet | | Inspect planned callable project assets | Project Tools | Current product exposes the category but no public lifecycle yet | | Inspect planned model-weight assets | Weights | Current product exposes the category but no public lifecycle yet | | Guide comparative output variation | Comparison Directions | Direction guidance is separate from coverage structure and approval | | Configure reusable expert-facing presentation | Review Screens | Presentation is separate from Case content and Contribution objectives | ## Project reuse and benchmark selection Project scope makes assets reusable across multiple benchmarks. That reuse also increases the impact of changes. Editing a case can affect any future benchmark snapshot that selects it. Saving a new Harness version does not silently change Runs that referenced an older version. Benchmark evidence should always identify the exact asset versions or snapshot involved. Worlds, Project Tools, and Weights are visible product categories, but their current pages do not expose durable user actions. Do not infer persistence, activation, execution, or evaluation semantics from the navigation label alone. Their reference pages record this limitation so operators and agents do not invent a workflow. ## Relationship to the five capabilities Weave creates and curates cases and supporting materials. Trialground evaluates saved Harness versions. Coevolve can materialize or evaluate candidate Harness versions during Improvement Sessions. Coverage Engineering and Correctness Elicitation influence which cases and candidates are useful, but ownership remains with the appropriate Asset or governed project surface. {% example-demo title="Shared harness pool" %} A project contains two saved retrieval Harness versions and one draft experiment. Two benchmarks select different case snapshots but can evaluate either saved Harness version. The draft remains editable and cannot be mistaken for the candidate used by an existing Run. Compare can therefore attribute result movement to the saved candidate and benchmark evidence boundary. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Work with cases](/docs/assets/cases) - [Manage Harnesses](/docs/assets/harnesses) - [Understand Project Tools](/docs/assets/project-tools) - [Understand Worlds](/docs/assets/worlds) - [Understand Weights](/docs/assets/weights) - [Manage Comparison Directions](/docs/assets/comparison-directions) - [Manage Review Screens](/docs/assets/review-screens) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Project Input Schema](/docs/project-settings/input-schema) - [Benchmarks](/docs/object-model/benchmarks) - [Product boundaries](/docs/introduction/product-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active Assets layout and navigation establish the project-level pool and tab names. Cases, Harnesses, Comparison Directions, and Review Screens have active surfaces; the current Worlds, Project Tools, and Weights routes expose empty states only. --- id: benchmark-evaluations.overview title: Benchmark Evaluations summary: Run and inspect exact Harness Versions against an immutable Benchmark Version through Dashboard, List, Arena, and Compare. kind: concept product_area: benchmark_evaluations status: stable updated: 2026-09-13 canonical: /docs/benchmark-evaluations --- # Benchmark Evaluations Benchmark Evaluations is the version-scoped workspace for executing and comparing candidate systems. The active top-level tabs are **Dashboard**, **List**, **Arena**, and **Compare**. Every managed Run binds an exact saved Harness Version to the immutable Benchmark Version shown in the route. > Evaluation boundary > > Interpret evidence inside its recorded Benchmark Version, Harness Version, Run or Run Group, evaluator set, and metadata. Run counts belong to launches. Additional launches add evidence without rewriting earlier Runs. ## Surfaces and objects Dashboard summarizes progress, leaderboards, rank progression across Runs, and available resource telemetry. List is segmented into **Runs**, **Evaluation results**, and **Traces / Spans**. The results segment summarizes Case outcomes and Policy or Rubric failures. Arena compares candidate pairs across governed metrics. Compare is a symmetric matrix of Harness Versions across selected evidence rows. A Run Group can collect one standard attempt or repeated attempts. A Run records one candidate execution and its per-Case progress. Evaluation results record the admitted Policy and Rubric outcomes. Costs, tokens, and latency are telemetry only when the provider or execution path captured them. > Traces / Spans capability fence > > The List navigation exposes Traces / Spans, but the current benchmark API does not expose evaluation execution traces. Do not claim that trajectories, spans, private reasoning, or tool traces can be inspected from Benchmark Evaluations today. ## Decision checkpoint | Need | Open | Evidence to preserve | | --- | --- | --- | | Configure and launch managed Runs | Evaluation Settings and New evaluation run | Machine, saved Harness Versions, and per-Harness Run counts | | Start candidate execution | Run modal | Exact Harness and Benchmark Versions | | Inspect status and output summaries | List → Runs or Evaluation results | Run Group, attempt, Case counts, incomplete state | | Compare candidate pairs | Arena | Metric family, pair count, only-A, only-B, shared failures | | Compare many candidates by governed rows | Compare | Harness columns and chosen Case or facet row mode | | Admit external reference outputs | Output mapping | Case mapping, attempt assignment, insert/update report | ## Rankings and repeated sampling Dashboard aggregates compatible observed Runs for each saved Harness Version across launches. Average score weights Runs equally. Supported binary views report passed at least once or passed every time over the observed case outcomes. Counts and missing evidence are shown; unequal counts do not prevent comparison. Historical group metrics retain their recorded meanings. Ranking is a routing signal. A candidate can lead overall while failing required Policy or high-impact Rubric evidence. Use Arena or Compare to locate the disagreement and List to confirm completeness before starting Improve work. ## External outputs Uploaded or API-supplied reference outputs create output-only Runs that can be scored and inspected in List. They are not saved Harness Versions and therefore cannot be optimized in Improve or selected as Harness columns in Compare or Arena. {% example-demo title="Example: repeated evaluation without evidence drift" %} A team launches three Runs of Harness Version 8 and one Run of Version 11 against the same Benchmark Version. Both appear with their evidence counts. A later launch of Version 11 adds two Runs to its aggregate evidence without changing either launch group. The team can inspect individual Runs before deciding whether more evidence is useful. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Configure evaluation execution](/docs/benchmark-evaluations/execution-settings) - [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) - [Use Arena and rankings](/docs/benchmark-evaluations/arena-and-rankings) - [Compare Harness Versions](/docs/benchmark-evaluations/compare) - [Map external outputs](/docs/benchmark-evaluations/output-mapping) {% /related-card-grid %} ## Source confidence Code-backed: the active version-scoped workspace, settings, Run modal, List segments, Dashboard, Arena, and Compare routes define the current evaluation model and capability fences.