# Comparison Directions Generated: 2026-09-13T04:39:33.444Z Source build: local Canonical docs: https://teammately.ai/docs --- id: assets.comparison-directions title: Comparison Directions summary: Create reusable guidance for meaningful candidate-output differences in comparative expert work. kind: reference product_area: assets status: stable updated: 2026-09-07 canonical: /docs/assets/comparison-directions --- # Comparison Directions ## Definition A Comparison Direction is a reusable project Asset that describes how candidate outputs should differ during comparative expert work. It can focus attention on a meaningful contrast such as evidence grounding, uncertainty handling, or response strategy without declaring which candidate is correct. Comparison Directions are project-scoped. A Contribution can select or allow a pool of directions for its comparative component, while the benchmark Contribution still owns the objective, cases, candidates, and expert task. ## Fields, states, or lifecycle rules - A direction has a name or label and a description of the intended contrast. - Users can create, edit, pin, archive, and remove directions from **Assets → Comparison Directions**. - Pinned directions are surfaced when a Contribution request selects comparative output guidance. - The suggestion experience creates draft candidates in a suggestion run. Nothing enters the reusable library until a user accepts it. - A direction can carry a staleness advisory when its source context has changed. Dismissing that advisory records a review decision; it does not approve a Policy, Rubric, Case, or Benchmark. - A direction guides comparative presentation or generation. It does not create a Case, change coverage structure, or replace expert judgment. ## Correct scope Use Dimensions, Project Topics, and Case Construction Patterns for the behavior space a benchmark should represent. Use Comparison Directions for how candidate outputs should be contrasted within a comparative Contribution. Use Correctness Governance for the approved standard that determines how an output is judged. {% example-demo title="Example: source-grounding contrast" %} A project creates one Comparison Direction asking for a response that cites the current source conservatively and another asking for a focused clarification when the source hierarchy is unresolved. A comparative Contribution can use those directions to elicit an expert preference. The direction does not approve either response or create the governing rubric. {% /example-demo %} ## Source confidence Code-backed: the active Assets routes expose the Comparison Directions library, detail controls, suggestion runs, accept or dismiss decisions, pinning, and staleness review. The API keeps legacy compatibility names internally, but this page uses the current product label. ## Related task pages {% related-card-grid title="Related task pages" %} - [Assets](/docs/assets) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Manage benchmark coverage](/docs/coverage-management) {% /related-card-grid %} --- id: assets.overview title: Assets summary: Manage reusable project cases, worlds, project tools, harnesses, weights, comparison directions, and review screens before selecting them for benchmark work. kind: concept product_area: assets status: stable updated: 2026-09-07 canonical: /docs/assets --- # Assets Assets is the project-level pool for cases, worlds, project tools, harnesses, weights, Comparison Directions, and Review Screens. Assets are managed once at project scope and selected for use in a specific benchmark rather than being recreated inside every benchmark workspace. ## Definition The active tabs are **Cases**, **Worlds**, **Project Tools**, **Harnesses**, **Weights**, **Comparison Directions**, and **Review Screens**. Cases provide the canonical situations evaluated or reviewed. Harnesses provide executable candidate implementations with Draft and saved Versions. Comparison Directions guide comparative output variation, and Review Screens provide reusable expert-facing presentation templates. Worlds, Project Tools, and Weights are visible categories whose current pages expose empty states rather than creation or lifecycle controls. Assets is distinct from Benchmark Datasets. The project pool answers what is available to the project. A benchmark dataset answers which cases and snapshot define one benchmark's evidence boundary. ## Decision checkpoint | Need | Asset or workspace | Boundary | | --- | --- | --- | | Create or inspect a reusable situation | Assets → Cases | Case content follows Project Input Schema | | Edit candidate code or prompt logic | Assets → Harnesses | A Draft must be saved as an exact version before evaluation | | Select cases for a benchmark | Benchmark Datasets | Selection and snapshot are benchmark-scoped | | Supply static documents or values to a case | Case materials | Static support is not a World | | Inspect planned environment assets | Worlds | Current product exposes the category but no public lifecycle yet | | Inspect planned callable project assets | Project Tools | Current product exposes the category but no public lifecycle yet | | Inspect planned model-weight assets | Weights | Current product exposes the category but no public lifecycle yet | | Guide comparative output variation | Comparison Directions | Direction guidance is separate from coverage structure and approval | | Configure reusable expert-facing presentation | Review Screens | Presentation is separate from Case content and Contribution objectives | ## Project reuse and benchmark selection Project scope makes assets reusable across multiple benchmarks. That reuse also increases the impact of changes. Editing a case can affect any future benchmark snapshot that selects it. Saving a new Harness version does not silently change Runs that referenced an older version. Benchmark evidence should always identify the exact asset versions or snapshot involved. Worlds, Project Tools, and Weights are visible product categories, but their current pages do not expose durable user actions. Do not infer persistence, activation, execution, or evaluation semantics from the navigation label alone. Their reference pages record this limitation so operators and agents do not invent a workflow. ## Relationship to the five capabilities Weave creates and curates cases and supporting materials. Trialground evaluates saved Harness versions. Coevolve can materialize or evaluate candidate Harness versions during Improvement Sessions. Coverage Engineering and Correctness Elicitation influence which cases and candidates are useful, but ownership remains with the appropriate Asset or governed project surface. {% example-demo title="Shared harness pool" %} A project contains two saved retrieval Harness versions and one draft experiment. Two benchmarks select different case snapshots but can evaluate either saved Harness version. The draft remains editable and cannot be mistaken for the candidate used by an existing Run. Compare can therefore attribute result movement to the saved candidate and benchmark evidence boundary. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Work with cases](/docs/assets/cases) - [Manage Harnesses](/docs/assets/harnesses) - [Understand Project Tools](/docs/assets/project-tools) - [Understand Worlds](/docs/assets/worlds) - [Understand Weights](/docs/assets/weights) - [Manage Comparison Directions](/docs/assets/comparison-directions) - [Manage Review Screens](/docs/assets/review-screens) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Project Input Schema](/docs/project-settings/input-schema) - [Benchmarks](/docs/object-model/benchmarks) - [Product boundaries](/docs/introduction/product-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active Assets layout and navigation establish the project-level pool and tab names. Cases, Harnesses, Comparison Directions, and Review Screens have active surfaces; the current Worlds, Project Tools, and Weights routes expose empty states only. --- id: expert-contributions.overview title: Expert Contributions summary: Coordinate benchmark-scoped expert work, attributable judgment, governed artifacts, and the decisions that move correctness forward. kind: concept product_area: expert_contributions status: stable updated: 2026-09-07 canonical: /docs/expert-contributions --- # Expert Contributions Expert Contributions is the benchmark-scoped workspace for requesting, conducting, and materializing specialist work. It coordinates the expert, objective, selected evidence, task sequence, checkpoints, attributable responses, and contributed artifacts needed to move a benchmark forward. ## Definition The administrator workspace contains **Overview**, **Contributions**, **Contributed Artifacts**, and **Logs & Status**. **Request Contribution** opens the composer for a new contribution. The expert follows a contribution-specific experience that can contain form, chat, interview, and case-review tasks, along with checkpoints and completion states. A Contribution is the unit of requested expert effort. It replaces broad workflow configuration with a bounded statement of what this benchmark needs from this expert now. The work can result in contributed policies, rubrics, cases, or coverage observations without flattening all expert activity into one generic approval record. ## Decision checkpoint | Need | Contribution element | Result to inspect | | --- | --- | --- | | Resolve a specific benchmark question | Contribution statement and scoped objectives | The expert can explain the requested decision | | Ground work in concrete behavior | Selected or designated cases | Case-level responses remain attributable | | Supply supporting knowledge | Attachments and scoped statements | The expert sees the relevant source boundary | | Choose the right interaction | Form, chat, interview, or case review task | Task output matches the kind of judgment needed | | Confirm consequential learning | Checkpoint | Accepted, revised, or unresolved state is explicit | | Reuse the result | Contributed Artifacts | Policies, rubrics, cases, and coverage observations retain provenance | ## Lifecycle and status The durable Contribution statuses are `PREPARING_DIRECTION`, `AWAITING_DIRECTION_ALIGNMENT`, `MATERIALIZING_TASKS`, `READY`, `IN_PROGRESS`, `COMPLETED`, and `CANCELLED`. The interface presents these as planning direction, waiting for alignment, preparing tasks, ready, active, completed, or cancelled. The exact task sequence can vary by Contribution. Realtime updates and durable transitions help the administrator and expert see current progress without inventing completion. A waiting state, checkpoint, or finalization step should be shown as such. Completing the expert experience does not imply that every proposed artifact has been accepted into its project-level owner. ## Contribution evidence Logs & Status exposes operational and engagement records. Contributed Artifacts organizes materialized or contributed cases, policies, rubrics, and new coverage observations. Correctness Governance, Assets, or Coverage Management owns the resulting project or benchmark artifact after materialization. This model improves return on expert effort. Agents prepare focused work from project context, indexed material, benchmark cases, and unresolved questions. The expert supplies the authority; the result can be reused across standards, coverage, evaluation, and improvement. {% example-demo title="Resolve source authority" %} A benchmark contains cases where an operational runbook conflicts with a newer policy page. The operator requests a Contribution from the policy owner, selects the conflicting cases, attaches both sources, and uses case review plus a checkpoint. The expert establishes which source controls, contributes a scoped policy and rubric, and records one coverage observation for an unrepresented exception. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) - [Contribution lifecycle and status](/docs/expert-contributions/lifecycle-and-status) - [Logs & Status](/docs/expert-contributions/logs-and-status) - [Agent Setup](/docs/agent-setup) - [Human Approval Boundaries](/docs/governance/human-approval-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active benchmark workspace, Contribution dashboard, composer, administrator detail, and expert routes support the scope, task, status, and artifact model described here. --- id: concepts.weave title: Weave summary: Construct deliberate challenge sets from coverage structure, canonical cases, variants, multimodal materials, and supported worlds. kind: concept product_area: weave status: stable updated: 2026-09-07 canonical: /docs/concepts/weave --- # Weave Weave is the case-construction capability inside Teammately correctness infrastructure. It turns structured coverage intent into the cases, response variants, supporting materials, and supported environments needed to challenge a specialist AI system. Its goal is deliberate representation, not undirected synthetic volume. ## Definition Coverage Engineering supplies the structure: Dimensions, Project Topics, Case Construction Patterns, coverage tuples, and known gaps. Weave uses that structure to prepare candidate cases that exercise meaningful combinations and boundaries. Cases follow the project's Input Schema, so their primary input and required materials have a consistent, inspectable shape. A case can include text, chat messages, structured data, images, documents, tabular files, presentations, source text, or audio when the Project Input Schema accepts those artifacts. Static support belongs to case materials. An executable or queryable environment is a separate world boundary and should only be used where the product supplies an inspectable world reference and runtime. ## Decision checkpoint | Construction need | Use | Verify before benchmark membership | | --- | --- | --- | | Add a known real example | Assets → Cases or dataset import | Input and materials conform to Project Input Schema | | Cover a missing facet combination | Coverage Management and Case Foundry | The case targets a named tuple or gap | | Generate a meaningful output variation | Case construction plus Comparison Directions | The idea guides variation without pretending the output is already reviewed | | Include supporting files | Canonical case materials | Artifact family and extension are accepted by the project | | Exercise changing state or tools | Supported World | World identity, capabilities, visibility, and lifecycle are explicit | ## Deliberate construction Case count is a weak target. A useful challenge set explains why every important slice exists and which behavior boundary it exercises. Case Construction Patterns provide reusable ways to instantiate a behavior problem. Coverage Management shows where cases already exist and where the benchmark remains thin. Case Review lets operators inspect generated materials before those cases become trusted dataset members. Comparison Directions are reusable Assets selected by comparative Contributions. They guide how candidate responses should vary—for example, changing evidence quality or reasoning approach—without serving as a coverage taxonomy or an automatically accepted collection of variants. ## Canonical case boundary `content.input` is the primary case input. Optional `content.case_materials` contains declared supporting values or artifacts. `record_content.case_view` is a rendered projection for users and execution adapters; it is not a second source of truth. Static runtime delivery uses case-material references. Executable worlds use a world instance reference. This distinction keeps the same case understandable to case authors, experts, harness code, and evaluation services. It also lets the system reject material that is missing, undeclared, or incompatible before a Run produces misleading evidence. {% example-demo title="Conflicting-source case" %} Coverage Management identifies a thin tuple combining policy exception, superseded source, and high customer impact. A Case Construction Pattern describes how to pair a current rule with a plausible older document. Weave creates a chat input and attaches both documents as case materials. Case Review confirms that the conflict is visible and realistic before the case enters the benchmark dataset snapshot. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Configure Project Input Schema](/docs/project-settings/input-schema) - [Manage benchmark coverage](/docs/coverage-management) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Assets and cases](/docs/assets) - [Cases](/docs/object-model/cases) - [Comparison Directions](/docs/assets/comparison-directions) {% /related-card-grid %} ## Source confidence Doctrine-backed: this page defines Weave as the public construction capability. It deliberately limits world claims to the verified runtime boundary and delegates current controls to linked code-backed pages.