# Weave Generated: 2026-09-13T04:34:29.335Z Source build: local Canonical docs: https://teammately.ai/docs --- id: concepts.weave title: Weave summary: Construct deliberate challenge sets from coverage structure, canonical cases, variants, multimodal materials, and supported worlds. kind: concept product_area: weave status: stable updated: 2026-09-07 canonical: /docs/concepts/weave --- # Weave Weave is the case-construction capability inside Teammately correctness infrastructure. It turns structured coverage intent into the cases, response variants, supporting materials, and supported environments needed to challenge a specialist AI system. Its goal is deliberate representation, not undirected synthetic volume. ## Definition Coverage Engineering supplies the structure: Dimensions, Project Topics, Case Construction Patterns, coverage tuples, and known gaps. Weave uses that structure to prepare candidate cases that exercise meaningful combinations and boundaries. Cases follow the project's Input Schema, so their primary input and required materials have a consistent, inspectable shape. A case can include text, chat messages, structured data, images, documents, tabular files, presentations, source text, or audio when the Project Input Schema accepts those artifacts. Static support belongs to case materials. An executable or queryable environment is a separate world boundary and should only be used where the product supplies an inspectable world reference and runtime. ## Decision checkpoint | Construction need | Use | Verify before benchmark membership | | --- | --- | --- | | Add a known real example | Assets → Cases or dataset import | Input and materials conform to Project Input Schema | | Cover a missing facet combination | Coverage Management and Case Foundry | The case targets a named tuple or gap | | Generate a meaningful output variation | Case construction plus Comparison Directions | The idea guides variation without pretending the output is already reviewed | | Include supporting files | Canonical case materials | Artifact family and extension are accepted by the project | | Exercise changing state or tools | Supported World | World identity, capabilities, visibility, and lifecycle are explicit | ## Deliberate construction Case count is a weak target. A useful challenge set explains why every important slice exists and which behavior boundary it exercises. Case Construction Patterns provide reusable ways to instantiate a behavior problem. Coverage Management shows where cases already exist and where the benchmark remains thin. Case Review lets operators inspect generated materials before those cases become trusted dataset members. Comparison Directions are reusable Assets selected by comparative Contributions. They guide how candidate responses should vary—for example, changing evidence quality or reasoning approach—without serving as a coverage taxonomy or an automatically accepted collection of variants. ## Canonical case boundary `content.input` is the primary case input. Optional `content.case_materials` contains declared supporting values or artifacts. `record_content.case_view` is a rendered projection for users and execution adapters; it is not a second source of truth. Static runtime delivery uses case-material references. Executable worlds use a world instance reference. This distinction keeps the same case understandable to case authors, experts, harness code, and evaluation services. It also lets the system reject material that is missing, undeclared, or incompatible before a Run produces misleading evidence. {% example-demo title="Conflicting-source case" %} Coverage Management identifies a thin tuple combining policy exception, superseded source, and high customer impact. A Case Construction Pattern describes how to pair a current rule with a plausible older document. Weave creates a chat input and attaches both documents as case materials. Case Review confirms that the conflict is visible and realistic before the case enters the benchmark dataset snapshot. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Configure Project Input Schema](/docs/project-settings/input-schema) - [Manage benchmark coverage](/docs/coverage-management) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Assets and cases](/docs/assets) - [Cases](/docs/object-model/cases) - [Comparison Directions](/docs/assets/comparison-directions) {% /related-card-grid %} ## Source confidence Doctrine-backed: this page defines Weave as the public construction capability. It deliberately limits world claims to the verified runtime boundary and delegates current controls to linked code-backed pages. --- id: coverage.overview title: Coverage Engineering summary: Design the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work. kind: concept product_area: coverage_engineering status: stable updated: 2026-09-07 canonical: /docs/coverage-engineering --- # Coverage Engineering Coverage Engineering is the capability for designing what a specialist AI benchmark must represent. It turns requirements, project knowledge, existing cases, and observed failures into a structured coverage map that guides dataset selection, case construction, expert contributions, and evaluation interpretation. ## Definition Coverage work has two scopes. At project scope, **Coverage Facets** manages reusable Dimensions, Project Topics, and Case Construction Patterns. At benchmark scope, **Coverage Management** applies those facets to setup, representation, Coverage Stories, Case Review, Case Foundry, and contribution requests. **Benchmark Datasets** owns the selected Cases and snapshots that embody the resulting coverage. Coverage Engineering is therefore broader than adding cases. It explains why a behavior slice matters, how it is represented, which combinations are thin, and what work should close the gap. ## Decision checkpoint | Question | Product surface | Durable result | | --- | --- | --- | | Which axes distinguish important behavior? | Coverage Facets → Dimensions | Reusable coverage axes | | Which domain subjects must be represented? | Coverage Facets → Project Topics | Project topic structure | | How should cases be constructed repeatedly? | Coverage Facets → Case Construction Patterns | Reusable construction guidance | | What should this benchmark cover? | Coverage Management → Get Started | Benchmark-specific coverage guidance | | Where is the selected dataset thin? | Representation and Coverage Stories | Named gaps and sourcing work | | Which exact cases define evidence? | Benchmark Datasets | Selection and snapshot boundary | ## Coverage map and benchmark evidence A coverage map should identify meaningful combinations rather than isolated tags. A source-freshness Dimension may be well populated overall while the combination of superseded source, high customer impact, and exception request remains absent. Coverage Stories make those combinations operational; Case Foundry and expert contribution requests can then target them. Representation is evidence about the dataset, not proof that the benchmark is complete. A large or balanced count can still omit a critical boundary. Product teams should use specialist judgment to decide which gaps materially affect trust. ## Relationship to correctness and construction Correctness Elicitation answers what should count as correct in a represented situation. Weave constructs or imports the cases and materials needed to exercise the situation. If the team cannot judge a coverage slice, request an Expert Contribution. If the judgment is clear but no case exists, use Case Foundry or other case-construction work. If cases exist but are not selected, update Benchmark Datasets. Comparison Directions are not Coverage Facets. They guide comparative response variation and belong to **Assets → Comparison Directions**. Keep benchmark representation in Dimensions, Topics, Patterns, Stories, and dataset snapshots. {% example-demo title="Authority-conflict coverage" %} A project creates source authority and customer impact Dimensions, a Project Topic for eligibility exceptions, and a pattern for pairing current and superseded documents. Coverage Management shows that the high-impact conflict tuple has no selected cases. A Coverage Story justifies the gap, Case Foundry prepares cases, and the accepted cases enter a new dataset snapshot. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Plan benchmark coverage](/docs/coverage-engineering/plan-benchmark-coverage) - [Manage Coverage](/docs/coverage-management) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coverage Dimensions](/docs/object-model/coverage-dimensions) - [Ontology](/docs/object-model/ontology) - [Cases](/docs/assets/cases) {% /related-card-grid %} ## Source confidence Code-backed: current navigation and benchmark coverage routes establish project Coverage Facets, benchmark Coverage Management, setup, representation, Stories, Case Review, and Case Foundry responsibilities. --- id: assets.overview title: Assets summary: Manage reusable project cases, worlds, project tools, harnesses, weights, comparison directions, and review screens before selecting them for benchmark work. kind: concept product_area: assets status: stable updated: 2026-09-07 canonical: /docs/assets --- # Assets Assets is the project-level pool for cases, worlds, project tools, harnesses, weights, Comparison Directions, and Review Screens. Assets are managed once at project scope and selected for use in a specific benchmark rather than being recreated inside every benchmark workspace. ## Definition The active tabs are **Cases**, **Worlds**, **Project Tools**, **Harnesses**, **Weights**, **Comparison Directions**, and **Review Screens**. Cases provide the canonical situations evaluated or reviewed. Harnesses provide executable candidate implementations with Draft and saved Versions. Comparison Directions guide comparative output variation, and Review Screens provide reusable expert-facing presentation templates. Worlds, Project Tools, and Weights are visible categories whose current pages expose empty states rather than creation or lifecycle controls. Assets is distinct from Benchmark Datasets. The project pool answers what is available to the project. A benchmark dataset answers which cases and snapshot define one benchmark's evidence boundary. ## Decision checkpoint | Need | Asset or workspace | Boundary | | --- | --- | --- | | Create or inspect a reusable situation | Assets → Cases | Case content follows Project Input Schema | | Edit candidate code or prompt logic | Assets → Harnesses | A Draft must be saved as an exact version before evaluation | | Select cases for a benchmark | Benchmark Datasets | Selection and snapshot are benchmark-scoped | | Supply static documents or values to a case | Case materials | Static support is not a World | | Inspect planned environment assets | Worlds | Current product exposes the category but no public lifecycle yet | | Inspect planned callable project assets | Project Tools | Current product exposes the category but no public lifecycle yet | | Inspect planned model-weight assets | Weights | Current product exposes the category but no public lifecycle yet | | Guide comparative output variation | Comparison Directions | Direction guidance is separate from coverage structure and approval | | Configure reusable expert-facing presentation | Review Screens | Presentation is separate from Case content and Contribution objectives | ## Project reuse and benchmark selection Project scope makes assets reusable across multiple benchmarks. That reuse also increases the impact of changes. Editing a case can affect any future benchmark snapshot that selects it. Saving a new Harness version does not silently change Runs that referenced an older version. Benchmark evidence should always identify the exact asset versions or snapshot involved. Worlds, Project Tools, and Weights are visible product categories, but their current pages do not expose durable user actions. Do not infer persistence, activation, execution, or evaluation semantics from the navigation label alone. Their reference pages record this limitation so operators and agents do not invent a workflow. ## Relationship to the five capabilities Weave creates and curates cases and supporting materials. Trialground evaluates saved Harness versions. Coevolve can materialize or evaluate candidate Harness versions during Improvement Sessions. Coverage Engineering and Correctness Elicitation influence which cases and candidates are useful, but ownership remains with the appropriate Asset or governed project surface. {% example-demo title="Shared harness pool" %} A project contains two saved retrieval Harness versions and one draft experiment. Two benchmarks select different case snapshots but can evaluate either saved Harness version. The draft remains editable and cannot be mistaken for the candidate used by an existing Run. Compare can therefore attribute result movement to the saved candidate and benchmark evidence boundary. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Work with cases](/docs/assets/cases) - [Manage Harnesses](/docs/assets/harnesses) - [Understand Project Tools](/docs/assets/project-tools) - [Understand Worlds](/docs/assets/worlds) - [Understand Weights](/docs/assets/weights) - [Manage Comparison Directions](/docs/assets/comparison-directions) - [Manage Review Screens](/docs/assets/review-screens) - [Work with Benchmark Datasets](/docs/benchmark-datasets) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Project Input Schema](/docs/project-settings/input-schema) - [Benchmarks](/docs/object-model/benchmarks) - [Product boundaries](/docs/introduction/product-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active Assets layout and navigation establish the project-level pool and tab names. Cases, Harnesses, Comparison Directions, and Review Screens have active surfaces; the current Worlds, Project Tools, and Weights routes expose empty states only. --- id: benchmark-datasets.overview title: Benchmark Datasets summary: Select benchmark Cases, inspect representation, and freeze immutable Snapshots for reproducible evidence. kind: concept product_area: benchmark_datasets status: stable updated: 2026-08-22 canonical: /docs/benchmark-datasets --- # Benchmark Datasets Benchmark Datasets defines the evidence set for one benchmark through **Cases**, **Representation**, and **Snapshots**. The current dataset is editable. It selects reusable project Cases and reflects current facet, policy, rubric, and contributor facts. A Snapshot freezes the exact dataset state needed by a Benchmark Version and its evaluations. These are deliberately different surfaces: editing the current set must not rewrite historical evidence. ## Decision checkpoint | Surface | Use it to | Evidence rule | | --- | --- | --- | | Cases | Inspect and change current benchmark membership | Selection is live until snapshotted | | Representation | Find concentration and absence across governed facets | Read distribution together with distinct Case counts | | Snapshots | Freeze Cases, evaluator links, and representation facts | Snapshot content is read-only | Coverage Management acts on gaps found in the dataset. Assets remains the project-level reusable pool. Benchmark Evaluations runs exact Harness Versions against an immutable Benchmark Version rather than an unspecified “current dataset.” ## Evidence flow Cases usually begin in project Assets or materialize through Case Review or Expert Contributions. Selecting them makes them part of the current benchmark dataset. Representation then summarizes the current assignments and evaluator relationships. Snapshot readiness checks whether that state can be frozen. A Snapshot supplies the immutable dataset facts used by a Benchmark Version. This flow is one-way for historical evidence. Later edits to an Asset, facet assignment, Policy, Rubric, or current membership may improve the next Snapshot, but they do not update a previous Snapshot. Compare candidates within one Benchmark Version unless the analysis explicitly accounts for a moved evidence boundary. ## Before creating evidence Check Case clarity and schema conformance, then inspect Representation for intended behavior and provenance. Confirm approved eligible evaluator links. Resolve Snapshot blockers and preserve the resulting label, version, content hash, creation time, and Case count. A Snapshot can be reproducible while still being incomplete as product coverage. Reproducibility answers which evidence was evaluated; Representation and Coverage Management answer whether that evidence supports the intended product claim. {% example-demo title="Example: editable set versus frozen evidence" %} The current dataset gains four Cases and a corrected Rubric link after an expert Contribution is reconciled. An earlier Run still points to its old Benchmark Version. The operator creates a new Snapshot and Version for the changed set rather than comparing the new candidate against the old Run as though only Harness behavior moved. {% /example-demo %} {% related-card-grid title="Dataset workflows" %} - [Manage benchmark Cases](/docs/benchmark-datasets/cases) - [Inspect Representation](/docs/benchmark-datasets/representation) - [Create and inspect Snapshots](/docs/benchmark-datasets/snapshots) - [Manage coverage](/docs/coverage-management) {% /related-card-grid %} ## Source confidence Code-backed: the active dataset routes establish the editable current set, representation workspace, and immutable Snapshot boundary.