# Teammately Core Context
Generated: 2026-09-13T04:32:38.447Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: concepts.coevolve
title: Coevolve
summary: Explore multiple evidence-backed candidate directions while keeping goals, benchmark receipts, trajectories, and the current frontier connected.
kind: concept
product_area: coevolve
status: stable
updated: 2026-08-22
canonical: /docs/concepts/coevolve
---
# Coevolve
Coevolve is the improvement capability inside Teammately correctness infrastructure. It lets agents and engineers pursue multiple candidate directions from benchmark evidence, evaluate those candidates through the same canonical path, and continue from stronger branches without losing the goal, chronology, or evidence behind the current frontier.
## Definition
Coevolve is realized through **Improve** and durable **Improvement Sessions**. A session starts from selected benchmark evidence and a target Harness. A Goal Contract turns a free-form objective into pinned target identities, measurement bindings, constraints, unresolved items, and permitted intervention boundaries. Once confirmed, that contract governs future work without rewriting earlier evidence.
Sessions can use Work or Evolve behavior. Work supports bounded implementation and review. Evolve supports parallel candidate exploration across epochs and retains a frontier based on current evaluation evidence. Candidate Harness versions become meaningful only when canonical Runs return observable results.
## Decision checkpoint
| Situation | Improvement action | Evidence requirement |
| --- | --- | --- |
| One known candidate change needs implementation | Start a Work session | Confirmed Goal Contract and pinned target evidence |
| Several hypotheses should compete | Start an Evolve session | Explicit authorization, measurement bindings, and comparable evaluation path |
| An external coding worker will implement | Prepare a scoped worker package | Returned Harness version or evaluation request before claiming observable progress |
| A candidate looks stronger | Inspect the current frontier | Canonical evaluation receipts support the retained position |
| Exploration exposes missing correctness or coverage | Return the observation upstream | Identify the policy, rubric, case, or coverage artifact that must change |
## Evidence-backed branching
Candidate exploration is not a sequence of undocumented edits. Each proposal should state the hypothesis and its relationship to the Goal Contract. Evaluation receipts bind candidate identity to benchmark identity and result. Narrated trajectories can explain the work performed, while chronology records durable transitions, agent activity, pauses, resumptions, and terminal state.
The current frontier is not simply the newest candidate. It represents the candidates retained by the session's evidence and goal constraints. A candidate can improve one slice and regress another; the frontier and comparison views keep that tradeoff visible.
## Bridge between experts and engineers
Improvement can reveal that the candidate is not the only incomplete part of the system. A missing coverage tuple, unclear rubric, contradictory policy, or insufficient case material should become an upstream contribution opportunity. This is how Coevolve connects coding agents to domain experts: engineering work is guided by benchmark evidence, and newly discovered correctness questions return to focused expert work.
External workers remain bounded. Teammately may prepare a package for Codex, Claude Code, or another worker, but it records only the work returned through the defined contract. It does not infer private activity or fabricate a working state.
{% example-demo title="Parallel grounding hypotheses" %}
An Evolve session starts from failures involving conflicting policy documents. One candidate changes retrieval filtering, another changes source ranking, and a third changes answer construction. Each saved Harness version is evaluated against the pinned benchmark. The frontier retains the candidates supported by grounding and uncertainty rubrics, while a newly observed source-authority ambiguity becomes an Expert Contribution opportunity.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Start an Improvement Session](/docs/improve/start-improvement-session)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Improve](/docs/improve)
- [Harnesses](/docs/assets/harnesses)
- [Product boundaries](/docs/introduction/product-boundaries)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page defines Coevolve as the public capability. The Improve pages provide code-backed session, contract, candidate, and frontier behavior.
---
id: concepts.correctness-elicitation
title: Correctness Elicitation
summary: Turn tacit specialist judgment into attributable contributions, governed policies, applicability conditions, and binary rubrics.
kind: concept
product_area: correctness_elicitation
status: stable
updated: 2026-09-07
canonical: /docs/concepts/correctness-elicitation
---
# Correctness Elicitation
Correctness Elicitation is the capability for turning specialist judgment into explicit, reusable correctness specifications. It handles the parts of AI behavior that cannot be settled by a generic score: domain preferences, exceptions, conflicts between sources, applicability boundaries, unacceptable failure modes, and the evidence an expert needs before making a decision.
## Definition
Elicitation begins before the expert opens a task. Teammately agents can organize relevant Reference Materials, cases, candidate responses, possible policies, rubric questions, and unresolved conflicts into a focused Contribution. The expert then works through forms, chat, interviews, case review, or checkpoints according to the requested components.
The contribution remains attributable. Accepted learning can be materialized into policies, rubrics, cases, or coverage observations, while drafts and unresolved statements keep their own state. Correctness Governance owns the resulting policies and rubrics; Expert Contributions owns the benchmark-scoped human work that produced them.
## Decision checkpoint
| Need | Elicitation method | Durable destination |
| --- | --- | --- |
| Confirm a known rule across several cases | Focused form or case review | Policy, applicability, or rubric contribution |
| Discover reasoning that is hard to pre-structure | Chat or interview | Attributable transcript, checkpoints, and contributed artifacts |
| Resolve disagreement or ambiguity | Targeted comparison and checkpoint | Explicit unresolved or accepted statement |
| Identify missing benchmark behavior | Cases plus coverage questions | Contributed case or coverage observation |
| Configure reviewer presentation | Assets → Review Screens | Reusable Review Screen |
| Set a contribution's agent behavior | Expert Contribution | Contribution behavior and selected components |
## Why preparation matters
An expert should not have to reconstruct the project, search for the controlling source, or infer why a case was selected. Project Context gives the agent the stable brief. Reference Materials provide indexed project knowledge. The Contribution selects the benchmark evidence, states the objective, and configures agent behavior. Review Screen controls presentation.
This separation preserves authority. Reference material can inform an answer without becoming a policy. Agent-authored draft wording can focus the expert without becoming approved. A completed expert task can contribute evidence without automatically changing every project artifact.
## Relationship to the product
Correctness Elicitation is broader than any single screen. Use **Expert Contributions** to request and track benchmark-scoped expert work. Use **Correctness Governance** to inspect and maintain policies and rubrics after they are materialized. Use **Agent Setup** to configure what agents understand, **Assets → Review Screens** for reusable expert-facing presentation, and the Contribution itself for scoped behavior and components.
The capability also returns learning to Coverage Engineering and Weave. An expert may identify a missing behavior combination, contribute a new case, or explain that existing material is insufficient. Those outputs should update the owning coverage or case artifacts instead of being flattened into a general review note.
{% example-demo title="Exception handling" %}
An agent prepares three cases where a procurement rule might allow an exception, attaches the controlling policy material, and asks a specialist to distinguish approved exceptions from escalation-only situations. The specialist completes a case review and confirms a checkpoint. The accepted contribution materializes a scoped policy and binary rubric, while one unresolved source conflict becomes a coverage observation for follow-up.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Manage policies and rubrics](/docs/correctness-governance)
- [Configure Agent Setup](/docs/agent-setup)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Human Approval Boundaries](/docs/governance/human-approval-boundaries)
- [Policies](/docs/object-model/policies)
- [Rubrics](/docs/object-model/rubrics)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page defines the public capability. The linked product pages are code-backed and define the active contribution, governance, and agent-configuration surfaces.
---
id: concepts.trialground
title: Trialground
summary: Evaluate exact Harness and Benchmark Versions in a managed proving ground with inspectable responses and Rubric evidence.
kind: concept
product_area: trialground
status: stable
updated: 2026-09-07
canonical: /docs/concepts/trialground
---
# Trialground
Trialground is the evaluation capability inside Teammately correctness infrastructure. It gives AI engineers a managed proving ground for running exact candidate implementations against exact Benchmark Versions while preserving the response, Rubric outcomes, settings, and metadata needed to explain the result.
## Definition
A candidate is represented by a saved Harness version rather than an informal description of a prompt or code state. A benchmark version provides the cases and governed correctness boundary. Benchmark Evaluations binds those inputs into Runs, computes rubric-level evidence, and exposes results through Dashboard, List, Compare, Arena, and run-detail views.
Trialground names the capability; **Harnesses** and **Benchmark Evaluations** are the current UI surfaces. The distinction matters when giving instructions: users edit and save a Harness under Assets, then select its saved version for a benchmark evaluation.
## Decision checkpoint
| Question | Inspect | Required boundary |
| --- | --- | --- |
| What code or prompt was evaluated? | Harness and saved version | Draft editor state is not the candidate identity |
| What behavior space was tested? | Benchmark version and dataset snapshot | Dataset membership is fixed for interpretation |
| Which standards judged each case? | Applicable policies and rubrics | Suggested or unrelated standards are excluded |
| Why did a candidate pass or fail? | Case response and applicable Rubric outcomes | Aggregate movement is traced to Case-level evidence |
| Can two candidates be compared? | Compare or Arena | Both candidates expose compatible benchmark and settings boundaries |
## Managed evaluation
Harness Assets provide a multi-file editing surface, Draft and Saved states, coding-agent assistance, secret references, and benchmark activation. Evaluation setup selects the relevant Harness version and preserves run metadata. Managed execution can scale case work while keeping every result attached to the benchmark and candidate identities.
The current Benchmark Evaluations API does not expose execution trajectories, spans, private reasoning, or tool traces. The visible **Traces / Spans** segment is therefore a capability fence, not inspectable Run evidence. Improve's narrated trajectories are separate session chronology and must not be described as evaluation execution traces. Rubric outcomes remain the governed evidence within the evaluation contract.
## Reading evidence
Start with Dashboard to understand overall progress and distribution, then use List for Run and Case-level results. Compare is a symmetric matrix whose columns are saved Harness Versions and whose rows can group Cases, all results, Policies, Rubrics, Dimensions, Topics, Groups, or Patterns. Arena computes pair evidence from admitted results; it is not a subjective preference interview. Run detail is the place to resolve execution state, output mapping, response evidence, and failures.
When a result is surprising, check versions and configuration before changing the candidate. A different benchmark snapshot, Harness version, output mapping, evaluation setting, or run-metadata value can explain movement that otherwise looks like a behavior change.
{% example-demo title="Retrieval candidate comparison" %}
Two saved Harness versions use different source-selection strategies. Both run against the same benchmark version containing current-source, stale-source, and missing-source cases. Compare shows that one candidate improves current-source grounding but regresses when no controlling source exists. The team opens the failed cases and rubric results before deciding whether the candidate belongs in the next Improvement Session.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Harnesses](/docs/assets/harnesses)
- [Benchmarks](/docs/object-model/benchmarks)
- [Run metadata](/docs/benchmark-evaluations/run-metadata)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page defines Trialground as the public capability. Harness and Benchmark Evaluation pages provide code-backed behavior and state details.
---
id: concepts.weave
title: Weave
summary: Construct deliberate challenge sets from coverage structure, canonical cases, variants, multimodal materials, and supported worlds.
kind: concept
product_area: weave
status: stable
updated: 2026-09-07
canonical: /docs/concepts/weave
---
# Weave
Weave is the case-construction capability inside Teammately correctness infrastructure. It turns structured coverage intent into the cases, response variants, supporting materials, and supported environments needed to challenge a specialist AI system. Its goal is deliberate representation, not undirected synthetic volume.
## Definition
Coverage Engineering supplies the structure: Dimensions, Project Topics, Case Construction Patterns, coverage tuples, and known gaps. Weave uses that structure to prepare candidate cases that exercise meaningful combinations and boundaries. Cases follow the project's Input Schema, so their primary input and required materials have a consistent, inspectable shape.
A case can include text, chat messages, structured data, images, documents, tabular files, presentations, source text, or audio when the Project Input Schema accepts those artifacts. Static support belongs to case materials. An executable or queryable environment is a separate world boundary and should only be used where the product supplies an inspectable world reference and runtime.
## Decision checkpoint
| Construction need | Use | Verify before benchmark membership |
| --- | --- | --- |
| Add a known real example | Assets → Cases or dataset import | Input and materials conform to Project Input Schema |
| Cover a missing facet combination | Coverage Management and Case Foundry | The case targets a named tuple or gap |
| Generate a meaningful output variation | Case construction plus Comparison Directions | The idea guides variation without pretending the output is already reviewed |
| Include supporting files | Canonical case materials | Artifact family and extension are accepted by the project |
| Exercise changing state or tools | Supported World | World identity, capabilities, visibility, and lifecycle are explicit |
## Deliberate construction
Case count is a weak target. A useful challenge set explains why every important slice exists and which behavior boundary it exercises. Case Construction Patterns provide reusable ways to instantiate a behavior problem. Coverage Management shows where cases already exist and where the benchmark remains thin. Case Review lets operators inspect generated materials before those cases become trusted dataset members.
Comparison Directions are reusable Assets selected by comparative Contributions. They guide how candidate responses should vary—for example, changing evidence quality or reasoning approach—without serving as a coverage taxonomy or an automatically accepted collection of variants.
## Canonical case boundary
`content.input` is the primary case input. Optional `content.case_materials` contains declared supporting values or artifacts. `record_content.case_view` is a rendered projection for users and execution adapters; it is not a second source of truth. Static runtime delivery uses case-material references. Executable worlds use a world instance reference.
This distinction keeps the same case understandable to case authors, experts, harness code, and evaluation services. It also lets the system reject material that is missing, undeclared, or incompatible before a Run produces misleading evidence.
{% example-demo title="Conflicting-source case" %}
Coverage Management identifies a thin tuple combining policy exception, superseded source, and high customer impact. A Case Construction Pattern describes how to pair a current rule with a plausible older document. Weave creates a chat input and attaches both documents as case materials. Case Review confirms that the conflict is visible and realistic before the case enters the benchmark dataset snapshot.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Configure Project Input Schema](/docs/project-settings/input-schema)
- [Manage benchmark coverage](/docs/coverage-management)
- [Work with Benchmark Datasets](/docs/benchmark-datasets)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Assets and cases](/docs/assets)
- [Cases](/docs/object-model/cases)
- [Comparison Directions](/docs/assets/comparison-directions)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page defines Weave as the public construction capability. It deliberately limits world claims to the verified runtime boundary and delegates current controls to linked code-backed pages.
---
id: coverage.overview
title: Coverage Engineering
summary: Design the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work.
kind: concept
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering
---
# Coverage Engineering
Coverage Engineering is the capability for designing what a specialist AI benchmark must represent. It turns requirements, project knowledge, existing cases, and observed failures into a structured coverage map that guides dataset selection, case construction, expert contributions, and evaluation interpretation.
## Definition
Coverage work has two scopes. At project scope, **Coverage Facets** manages reusable Dimensions, Project Topics, and Case Construction Patterns. At benchmark scope, **Coverage Management** applies those facets to setup, representation, Coverage Stories, Case Review, Case Foundry, and contribution requests. **Benchmark Datasets** owns the selected Cases and snapshots that embody the resulting coverage.
Coverage Engineering is therefore broader than adding cases. It explains why a behavior slice matters, how it is represented, which combinations are thin, and what work should close the gap.
## Decision checkpoint
| Question | Product surface | Durable result |
| --- | --- | --- |
| Which axes distinguish important behavior? | Coverage Facets → Dimensions | Reusable coverage axes |
| Which domain subjects must be represented? | Coverage Facets → Project Topics | Project topic structure |
| How should cases be constructed repeatedly? | Coverage Facets → Case Construction Patterns | Reusable construction guidance |
| What should this benchmark cover? | Coverage Management → Get Started | Benchmark-specific coverage guidance |
| Where is the selected dataset thin? | Representation and Coverage Stories | Named gaps and sourcing work |
| Which exact cases define evidence? | Benchmark Datasets | Selection and snapshot boundary |
## Coverage map and benchmark evidence
A coverage map should identify meaningful combinations rather than isolated tags. A source-freshness Dimension may be well populated overall while the combination of superseded source, high customer impact, and exception request remains absent. Coverage Stories make those combinations operational; Case Foundry and expert contribution requests can then target them.
Representation is evidence about the dataset, not proof that the benchmark is complete. A large or balanced count can still omit a critical boundary. Product teams should use specialist judgment to decide which gaps materially affect trust.
## Relationship to correctness and construction
Correctness Elicitation answers what should count as correct in a represented situation. Weave constructs or imports the cases and materials needed to exercise the situation. If the team cannot judge a coverage slice, request an Expert Contribution. If the judgment is clear but no case exists, use Case Foundry or other case-construction work. If cases exist but are not selected, update Benchmark Datasets.
Comparison Directions are not Coverage Facets. They guide comparative response variation and belong to **Assets → Comparison Directions**. Keep benchmark representation in Dimensions, Topics, Patterns, Stories, and dataset snapshots.
{% example-demo title="Authority-conflict coverage" %}
A project creates source authority and customer impact Dimensions, a Project Topic for eligibility exceptions, and a pattern for pairing current and superseded documents. Coverage Management shows that the high-impact conflict tuple has no selected cases. A Coverage Story justifies the gap, Case Foundry prepares cases, and the accepted cases enter a new dataset snapshot.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Plan benchmark coverage](/docs/coverage-engineering/plan-benchmark-coverage)
- [Manage Coverage](/docs/coverage-management)
- [Work with Benchmark Datasets](/docs/benchmark-datasets)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Dimensions](/docs/object-model/coverage-dimensions)
- [Ontology](/docs/object-model/ontology)
- [Cases](/docs/assets/cases)
{% /related-card-grid %}
## Source confidence
Code-backed: current navigation and benchmark coverage routes establish project Coverage Facets, benchmark Coverage Management, setup, representation, Stories, Case Review, and Case Foundry responsibilities.
---
id: orientation.key-objects
title: Key objects and relationships
summary: Understand how project foundations, contributions, datasets, evaluations, and improvement artifacts connect.
kind: concept
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/getting-oriented/key-objects-and-relationships
---
# Key objects and relationships
Teammately's evidence is trustworthy when a reader can move from project understanding and specialist authority to the exact Case, Benchmark version, Harness version, Run, and Improvement Session involved. This page gives the shared object graph.
> Artifact state matters
>
> An object name is not enough. Preserve project or benchmark scope, version or snapshot identity, provenance, and lifecycle state whenever the object supports evaluation or improvement.
## Definition
A **Project Agent Brief** and published **Reference block** give agents project understanding. **Project Input Schema** governs canonical **Case** input and materials. **Dimensions**, **Project Topics**, and Case Construction Patterns define reusable coverage structure. A saved **Harness version** identifies an executable candidate.
A benchmark selects Cases into a **Dataset snapshot** and combines them with governed **Policies** and **Rubrics** through a **Benchmark version**. An **Expert Contribution** requests specialist judgment through one or more Tasks and Checkpoints. Its **Contributed artifact** can become a policy, rubric, case, or coverage observation while retaining provenance.
A **Run** evaluates a saved Harness Version against a Benchmark Version. Its response, Rubric outcomes, settings, mapping, and metadata form evaluation evidence. An **Improvement Session** pins target evidence through a Goal Contract, creates or receives candidates, records evaluation receipts and safe session narration, and maintains a **Current frontier**.
## Decision checkpoint
| Object | Scope | Relationship that must remain visible |
| --- | --- | --- |
| Project Agent Brief / Reference block | Project | What agents understood and which source generation was available |
| Case / Harness version | Project | Which reusable asset and exact candidate state was selected |
| Contribution / Checkpoint | Benchmark | Which expert supplied or confirmed the judgment |
| Policy / Rubric | Project governance | Which authority, applicability, cases, and provenance support it |
| Dataset snapshot / Benchmark version | Benchmark | Which cases and correctness boundary define evidence |
| Run | Benchmark version | Which Harness, settings, mapping, and metadata produced results |
| Improvement Session / Current frontier | Benchmark version | Which goal and evaluation receipts justify retained candidates |
{% artifact-map title="How correctness artifacts connect" %}
{% /artifact-map %}
## Static materials and executable worlds
Canonical Case content separates `content.input` from optional `content.case_materials`. Static execution support uses case-material references. A `world_instance_ref` represents an executable or queryable environment and follows a separate capability and lifecycle boundary. The rendered case view helps people and adapters inspect canonical content; it does not create another authoring source.
## Provenance across scopes
Project assets can be reused across benchmarks, while dataset snapshots, Contributions, Runs, and Improvement Sessions remain benchmark-scoped. Materializing a contributed policy moves its governed owner to project scope without erasing the benchmark Contribution that supplied it. Evaluating a candidate records the saved Harness version rather than whichever Draft is currently open.
{% example-demo title="Contribution to frontier" %}
An Expert Contribution confirms a source-authority Policy and Rubric from selected Cases. The Cases enter a Dataset snapshot and the standard enters a Benchmark version. A Run evaluates Harness version 8 and exposes three failures. An Improvement Session pins those failures, evaluates versions 9 and 10, and retains version 10 in the Current frontier with canonical evaluation receipts.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Object model](/docs/object-model)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Product map](/docs/getting-oriented/product-map)
{% /related-card-grid %}
## Source confidence
Code-backed: active navigation, canonical case contracts, Contribution surfaces, versioned evaluation routes, and Improvement Session contracts support this object graph.
---
id: orientation.product-map
title: Product map
summary: Navigate Teammately across workspace entry points, project foundations, benchmark workspaces, expert contribution UI, and administration.
kind: concept
product_area: introduction
status: stable
updated: 2026-09-07
canonical: /docs/getting-oriented/product-map
---
# Product map
Teammately separates reusable project foundations from benchmark-scoped work. The Main UI uses the selected project and benchmark to route operators to the right scope. Experts receive a focused Expert contribution UI. Admin Console owns organization-level controls, while AI-assisted background work prepares and coordinates bounded tasks.
> Surface routing
>
> Before changing an artifact, identify its scope. Project foundations can affect several benchmarks; dataset selection, Contributions, Evaluations, and Improvement Sessions belong to a selected benchmark or benchmark version.
## Definition
The **Main UI** begins at Project Home and groups project-level work into Correctness Governance, Coverage Facets, Assets, Agent Setup, and Project Settings. After a benchmark is selected, its workspace exposes Benchmark Overview, Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve.
The **Expert contribution UI** presents one Contribution and its form, chat, interview, case-review, checkpoint, waiting, and completion states. The expert does not need the full project navigation to supply attributable judgment.
The **Admin Console** contains organization administration such as members, groups, roles, domain controls, integrations, and other code-backed administrative surfaces. Public docs keep detailed security, billing, retention, and compliance claims outside the boundary unless separately verified.
**AI-assisted background work** can index reference material, prepare contributions, suggest coverage or standards, construct cases, run evaluations, and coordinate candidate exploration. Its outputs retain the authority of the owning artifact and workflow.
## Decision checkpoint
| Work | Scope | Surface |
| --- | --- | --- |
| Project purpose and knowledge | Project | Agent Setup |
| Policies and rubrics | Project | Correctness Governance |
| Dimensions, Topics, and construction patterns | Project | Coverage Facets |
| Reusable Cases and Harnesses | Project | Assets |
| Input architecture | Project | Project Settings |
| Benchmark-level run fields | Benchmark | Benchmark Evaluations |
| Selected Cases, representation, and snapshots | Benchmark | Benchmark Datasets |
| Coverage setup, Stories, Case Review, and Foundry | Benchmark | Coverage Management |
| Specialist requests and contributed artifacts | Benchmark | Expert Contributions |
| Runs, results, Compare, and Arena | Benchmark version | Benchmark Evaluations |
| Goal Contracts, candidates, and frontier | Benchmark version | Improve |
## How selection affects navigation
Project surfaces require a project. Benchmark surfaces also require a benchmark, and Evaluations or Improve may resolve the current benchmark version. If a destination is unavailable, confirm the current selectors before assuming that the feature or data is missing.
Project folders and search help users move across a larger workspace, but they do not change artifact ownership. Search results and creation actions should preserve the selected project or benchmark scope.
{% surface-map title="Teammately product surfaces" %}
{% /surface-map %}
{% example-demo title="Route a new rubric need" %}
An evaluation exposes inconsistent handling of expired agreements. The operator uses the benchmark workspace to request an Expert Contribution with the failed cases. The expert works in the focused contribution UI. The resulting rubric is reconciled in Correctness Governance at project scope, then included in a later benchmark version and evaluation.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Task index](/docs/operating-manual/task-index)
- [Operating Teammately end to end](/docs/getting-oriented/operating-teammately-end-to-end)
- [Product quickstart](/docs/quickstart)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [User roles](/docs/getting-oriented/user-roles)
- [Product boundaries](/docs/introduction/product-boundaries)
{% /related-card-grid %}
## Source confidence
Code-backed: current navigation and active project, benchmark, Contribution, and administration routes establish the scope and labels described here.
---
id: intro.correctness-infrastructure
title: What is correctness infrastructure?
summary: Learn how five connected capabilities turn specialist judgment into cases, executable standards, evaluation evidence, and improvement.
kind: concept
product_area: introduction
status: stable
updated: 2026-08-22
canonical: /docs/introduction/correctness-infrastructure
---
# What is correctness infrastructure?
Correctness infrastructure is the operating layer that lets a team specify, test, and improve the behavior of specialist AI. It connects the behavior space that matters, the expert judgment that defines acceptable behavior, the cases that challenge a system, the evidence produced by repeatable evaluation, and the engineering work that follows.
{% visual-hero src="/docs-assets/assets/correctness-infrastructure-workbench.png" alt="Workbench connecting coverage design, expert judgment, cases, evaluation evidence, and improvement." %}
The visual is a category anchor. The selectable capability names and current product mappings below are authoritative.
{% /visual-hero %}
## Definition
Teammately expresses correctness infrastructure through five capabilities:
1. **Coverage Engineering** designs what the benchmark must represent.
2. **Correctness Elicitation** captures policies, exceptions, applicability, and expert rubrics.
3. **Weave** constructs cases, variants, multimodal materials, and supported worlds from the coverage design.
4. **Trialground** runs harnesses and weights against benchmark cases and records rubric evidence.
5. **Coevolve** explores stronger candidate branches and missing correctness or coverage.
The sequence is connected rather than strictly linear. An evaluation failure may expose a missing rubric. An expert contribution may reveal a new coverage facet. An Improvement Session may find that the benchmark needs a harder case before a harness change can be trusted.
## Decision checkpoint
| Question | Inspect first | Expected durable result |
| --- | --- | --- |
| Does the benchmark represent the important behavior space? | Coverage Engineering | Dimensions, Project Topics, Case Construction Patterns, and benchmark coverage state |
| What should count as correct for this situation? | Correctness Elicitation | Governed policies, applicability, and binary rubrics |
| Do we have cases that exercise the difficult combinations? | Weave | Reviewable cases, variants, materials, and dataset membership candidates |
| How did this candidate behave against the benchmark? | Trialground | Runs, responses, Rubric results, comparisons, and rankings |
| What change is justified by the evidence? | Coevolve | A bounded Improvement Session with candidates and evaluation receipts |
> Keep the category centered
>
> Evaluation execution is one capability inside correctness infrastructure. The category also includes the expert-derived specification, deliberate coverage, case construction, and evidence-backed improvement that make evaluation meaningful.
## From expert effort to reusable infrastructure
Expert time is most valuable when it resolves ambiguity that agents and engineers cannot settle from existing evidence. Teammately therefore prepares a structured contribution: the relevant cases, reference materials, candidate interpretations, possible policies, rubric questions, and unresolved conflicts. Once an expert responds, the contribution can affect more than the immediate task. It can refine the coverage map, materialize a policy or rubric, qualify a case, or identify the next evaluation.
This creates a higher return on expert effort. The product does not ask specialists to repeatedly label disconnected outputs; it preserves why a judgment was made and where that judgment applies.
## Product realization
The project foundation holds reusable context and assets. Correctness Governance owns policies and rubrics. Coverage Facets owns Dimensions, Project Topics, and Case Construction Patterns. Assets owns cases, harnesses, and other project-level resources, including reusable Comparison Directions and Review Screens. Agent Setup gives agents reusable project understanding through Project Context and Reference Materials. Expert Contributions configure scoped agent behavior and select the presentation and variation assets needed for the work.
A benchmark workspace then creates a scoped operational loop. Benchmark Datasets defines the case set and snapshots. Coverage Management exposes represented and missing behavior. Expert Contributions requests governed human work. Benchmark Evaluations produces versioned evidence. Improve starts from that evidence and coordinates candidate exploration.
{% layer-map title="Five capabilities of correctness infrastructure" %}
{% /layer-map %}
{% example-demo title="Policy exception handling" %}
A support assistant must answer exception requests. Coverage Engineering identifies exception type, customer state, and source freshness as important facets. Correctness Elicitation records the expert rule for when an exception can be stated or must be escalated. Weave prepares ordinary, conflicting-source, and boundary cases. Trialground shows that a candidate handles ordinary cases but invents approval on ambiguous ones. Coevolve tests a harness change and keeps the resulting candidate tied to the exact failed cases and rubric evidence.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [The correctness lifecycle](/docs/introduction/correctness-lifecycle)
- [The correctness loop](/docs/product-loop)
- [Product quickstart](/docs/quickstart)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Product map](/docs/getting-oriented/product-map)
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [Object model](/docs/object-model)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page defines the current public category and capability model. Product-surface pages provide code-backed operational detail.
---
id: intro.correctness-lifecycle
title: The Teammately correctness lifecycle
summary: Follow specialist AI work from project foundations through coverage, elicitation, construction, evaluation, and improvement.
kind: concept
product_area: introduction
status: stable
updated: 2026-08-22
canonical: /docs/introduction/correctness-lifecycle
---
# The Teammately correctness lifecycle
The correctness lifecycle describes how a team turns domain knowledge into an improving specialist AI system. It begins with reusable project foundations, narrows into a benchmark workspace, and cycles through coverage, expert contribution, evaluation, and improvement without losing the evidence that explains each change.
## Definition
The lifecycle has three scopes:
1. **Project foundation.** Define Project Context and Reference Materials, configure the Project Input Schema, govern policies and rubrics, establish Coverage Facets, and manage reusable Assets. Reusable Review Screens and Comparison Directions are authored under Assets; Contribution-specific agent behavior and asset selection happen in benchmark work.
2. **Benchmark work.** Select a benchmark dataset and snapshot, manage coverage, request expert contributions, and bind the work to a benchmark version.
3. **Evaluation and improvement.** Evaluate saved harness versions, inspect cases and rubric results, compare candidates, and start an Improvement Session from pinned evidence.
The public five-capability model runs through these scopes. Coverage Engineering shapes representation. Correctness Elicitation captures judgment. Weave constructs the challenge set. Trialground produces evaluation evidence. Coevolve turns that evidence into bounded candidate work.
## Decision checkpoint
| Current condition | Next lifecycle action | Boundary to preserve |
| --- | --- | --- |
| Agents lack the project purpose or controlling knowledge | Complete Agent Setup | Project Context and Reference Materials remain distinct from governed policies |
| Case shape is ambiguous | Configure Project Input Schema | `content.input` and case materials follow one declared architecture |
| Coverage exists but correctness is tacit | Request an Expert Contribution | Human responses remain attributable before materialization |
| Cases and standards are ready | Create or select a benchmark snapshot and evaluate a saved Harness version | The run stays bound to exact versions and settings |
| Evaluation evidence exposes a candidate weakness | Start an Improvement Session | The Goal Contract and starting evidence remain pinned |
| Candidate exploration exposes a benchmark gap | Return to coverage, standards, or cases | Do not interpret missing evidence as candidate failure |
## Why ordering matters
Running evaluations before the benchmark has deliberate coverage can produce precise but misleading results. Writing rubrics without expert-grounded policies can turn vague preferences into brittle checks. Generating cases without a Project Input Schema can create records that reviewers or harnesses interpret differently. Starting improvement from an aggregate score can hide the cases and standards that actually justify a change.
The lifecycle prevents those shortcuts by giving each artifact an owner and scope. Project foundations are reusable across benchmarks. Benchmark datasets and versions define the evidence boundary. Contributions provide human authority. Runs and comparisons expose candidate behavior. Improvement Sessions retain the chronology between a goal, a proposed change, and its evaluation receipt.
## How learning returns to the system
The lifecycle closes when new evidence changes an upstream artifact. A failed run may show that a policy is too broad, a rubric is ambiguous, a case lacks required material, or a coverage tuple is absent. An expert can contribute a new policy, rubric, case, or coverage observation. An Improvement Session can test a harness candidate while also reporting missing correctness or coverage.
Each return path should name the responsible artifact. Simultaneously changing cases, standards, harness code, and evaluation settings makes the next result difficult to explain.
{% example-demo title="Specialist procurement assistant" %}
The team configures structured procurement inputs and indexes its buying rules. Coverage Engineering maps product category, contract state, and exception type. An expert contribution resolves when an exception requires escalation and materializes the policy and rubric. Weave prepares cases with conflicting contract material. Trialground evaluates a saved harness version and exposes failures on expired agreements. Coevolve tests a source-date validation change while retaining the original benchmark evidence.
{% /example-demo %}
## Where the lifecycle stops
Teammately makes correctness work inspectable and reusable. It does not determine the customer's downstream deployment, operational, or governance action. The product supplies governed artifacts, benchmark evidence, and improvement history so the accountable team can make that decision with a clear record.
## Related workflows
{% related-card-grid title="Related workflows" %}
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Project Input Schema](/docs/project-settings/input-schema)
- [Expert Contributions](/docs/expert-contributions)
- [Improvement Sessions](/docs/improve)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this lifecycle joins the current public capability model to code-backed product scopes. Linked pages define individual object and state behavior.
---
id: intro.product-boundaries
title: Product boundaries
summary: Understand what Teammately owns across correctness specification, benchmark development, evaluation, and improvement.
kind: concept
product_area: introduction
status: stable
updated: 2026-09-07
canonical: /docs/introduction/product-boundaries
---
# Product boundaries
Teammately owns the correctness system that connects specialist judgment to deliberate benchmark coverage, executable standards, constructed cases, evaluation evidence, and improvement history. This page distinguishes that system from adjacent inputs and downstream responsibilities.
> Adjacent systems are inputs
>
> Logs, traces, source repositories, model endpoints, coding environments, and external evaluation results can supply material or receive work. Their presence does not change the Teammately ownership boundary: Teammately governs the connected correctness artifacts and the evidence produced from them.
## Definition
The product boundary follows artifacts and authority. Teammately can index project knowledge, prepare an expert contribution, materialize an accepted policy or rubric, construct a case, execute an evaluation through a managed harness, and coordinate an Improvement Session. It preserves which inputs, versions, settings, and human decisions produced the resulting evidence.
Customer teams own the AI system outside that evidence graph and the action taken afterward. Teammately can prepare a scoped package for an external coding worker, but it does not claim private work performed outside the product. It can show benchmark evidence, but it does not turn that evidence into an automatic downstream decision.
## Decision checkpoint
| Area | Teammately owns | Boundary |
| --- | --- | --- |
| Project knowledge and agent context | Materials, Indexed Reference, Project Context, and Contribution-scoped agent behavior | Reference material is not automatically a governed policy or rubric |
| Expert work | Contribution scope, tasks, checkpoints, attributable responses, and contributed artifacts | Agent preparation does not substitute for the expert's judgment |
| Cases and worlds | Canonical case input, case materials, generated artifacts, and verified world references | Static materials and executable environments remain distinct |
| Evaluation | Benchmark Versions, saved Harness Versions, settings, Runs, responses, Rubric results, and comparisons | A score alone does not explain correctness; execution traces are not currently exposed |
| Improvement | Goal Contracts, candidates, evaluation receipts, frontiers, and chronology | External worker activity is represented only when returned through the defined contract |
| Downstream action | Inspectable correctness evidence and review context | The customer decides what operational action follows |
## Human and agent authority
AI agents scale preparation and exploration. They can organize source material, propose coverage structure, draft possible standards, generate cases, evaluate candidates, and suggest improvement directions. The owning surface determines when an artifact becomes durable or governed. An agent proposal does not silently acquire expert authority.
Expert Contributions make this boundary explicit. The product can prepare focused questions and relevant evidence, while the domain specialist supplies the judgment. Correctness Governance records policies and rubrics as governed project assets. Improvement Sessions can branch candidate hypotheses, but retained candidates require observable evaluation evidence.
## Data and execution boundary
Project Input Schema controls the accepted shape of case input and materials. Static context remains part of case content or case-material references. An executable or queryable environment uses a world reference and follows a separate runtime boundary. Public docs describe the behavior visible through stable product surfaces; they do not promote internal storage or service structures into customer-facing contracts.
Similarly, the presence of Harness Assets and managed Runs does not imply that Teammately owns a customer's model registry, production telemetry, or deployment system. A harness is the executable candidate boundary used by a benchmark evaluation.
{% example-demo title="External coding worker" %}
An Improvement Session starts from failed grounding Cases and a confirmed Goal Contract. Teammately prepares a scoped package for a coding worker with the pinned target and evidence. The worker's private activity is outside the product boundary. A returned Harness Version and canonical evaluation request become observable candidates; their Rubric results enter the session record, while Improve may add a safe narrated trajectory of observable session activity.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [The correctness lifecycle](/docs/introduction/correctness-lifecycle)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
- [Use Reference Materials](/docs/agent-setup/reference-materials)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Human Approval Boundaries](/docs/governance/human-approval-boundaries)
- [What AI Features Can and Cannot Do](/docs/governance/what-ai-features-can-and-cannot-do)
- [Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page states product ownership and authority boundaries. Exact UI and execution behavior is delegated to linked code-backed pages.
---
id: intro.what-is-teammately
title: What is Teammately?
summary: Understand Teammately as correctness infrastructure for building trustworthy specialist AI with expert judgment and AI agents.
kind: concept
product_area: introduction
status: stable
updated: 2026-08-22
canonical: /docs/introduction/what-is-teammately
---
# What is Teammately?
Teammately is correctness infrastructure for teams building specialist AI. It turns in-house experts' judgment into an operating system for designing benchmark coverage, making correctness explicit, constructing challenging cases, evaluating candidate behavior, and deciding what to improve next. AI agents prepare and connect the work so scarce expert attention is spent on consequential judgment rather than manual organization.
> Category boundary
>
> Teammately centers the definition and development of trustworthy AI behavior. Logs, traces, model endpoints, coding environments, and external data can enter the workflow, but the product's durable value is the connected correctness system built from expert judgment, cases, standards, evaluation evidence, and improvement history.
## Definition
The system has five connected capabilities. [Coverage Engineering](/docs/coverage-engineering) defines the behavior space a benchmark must represent. [Correctness Elicitation](/docs/concepts/correctness-elicitation) turns tacit preferences, exceptions, and disagreements into policies, applicability conditions, and binary rubrics. [Weave](/docs/concepts/weave) constructs cases, response variants, case materials, and—where supported—worlds from that structure. [Trialground](/docs/concepts/trialground) evaluates Harnesses and weights against benchmark Cases and preserves responses and Rubric results. [Coevolve](/docs/concepts/coevolve) explores candidate improvements and keeps every retained direction tied to current benchmark evidence.
These capability names explain how the system works. Procedures use the labels visible in the product, such as Correctness Governance, Agent Setup, Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve.
## Decision checkpoint
| If the team needs... | Capability | Product surfaces to open |
| --- | --- | --- |
| A deliberate map of important behavior | Coverage Engineering | Coverage Facets and Coverage Management |
| Reusable standards grounded in specialist judgment | Correctness Elicitation | Correctness Governance and Expert Contributions |
| Challenging cases and supporting materials | Weave | Assets, Benchmark Datasets, Case Construction Patterns, and Case Foundry |
| Repeatable evidence about candidate behavior | Trialground | Harnesses and Benchmark Evaluations |
| Evidence-backed candidate improvement | Coevolve | Improve and Improvement Sessions |
## Why teams use it
A benchmark score cannot define correctness on its own. Specialist systems depend on domain rules, exceptions, source authority, interaction patterns, and consequences that generic criteria do not capture. Teammately gives experts and AI engineers a shared artifact graph: an expert contribution can inform a policy, a policy can produce a rubric, a coverage gap can motivate a case, a case can expose a harness weakness, and an evaluation can become the starting evidence for an Improvement Session.
This reuse is the practical meaning of scaling expert judgment. Teammately prepares coverage structure, candidate cases, possible standards, and unresolved questions before asking an expert. The expert's response remains attributable and can be materialized into governed artifacts instead of disappearing into meeting notes.
## Product scope
Project-level foundations hold reusable knowledge and assets: Correctness Governance, Coverage Facets, Assets, Agent Setup, and Project Settings. Benchmark workspaces bind those foundations to a concrete evaluation program through Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve.
Teammately preserves correctness evidence and makes the next engineering question inspectable. Customer teams remain responsible for downstream product, governance, deployment, and operational choices.
> Human ownership
>
> AI agents can prepare, draft, classify, generate, evaluate, and propose. A suggestion is not a governed policy, accepted expert contribution, benchmark membership decision, or retained candidate merely because an agent produced it. Use the state shown by the owning product surface.
{% example-demo title="Grounded enterprise search" %}
Coverage Engineering identifies conflicting-current-source questions as an important behavior slice. Correctness Elicitation records the expert rule that material claims must cite the controlling source or state uncertainty. Weave creates cases with current and superseded documents. Trialground evaluates a retrieval harness and exposes unsupported blends of the two sources. Coevolve starts from those failures, tests a source-selection change, and retains only candidates supported by evaluation evidence.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Product quickstart](/docs/quickstart)
- [The correctness loop](/docs/product-loop)
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Product map](/docs/getting-oriented/product-map)
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [Product boundaries](/docs/introduction/product-boundaries)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page follows the current public top-page story and the approved product-to-UI mapping. Linked code-backed pages define exact routes, states, and controls.
---
id: product-loop
title: The Teammately correctness loop
summary: See how coverage, elicitation, case construction, evaluation, and improvement reinforce one another.
kind: concept
product_area: introduction
status: stable
updated: 2026-08-22
canonical: /docs/product-loop
---
# The Teammately correctness loop
The correctness loop is how a team repeatedly turns domain knowledge into stronger AI behavior. It follows the five public capabilities while preserving a trace from every result back to the project context, expert contribution, case, policy, rubric, benchmark version, Harness version, and evaluation setting that made the result meaningful.
## Definition
1. **Design coverage.** Establish Dimensions, Project Topics, and Case Construction Patterns, then decide which combinations the benchmark must represent.
2. **Elicit correctness.** Use focused expert contributions to resolve policies, exceptions, applicability, disagreements, and binary rubric language.
3. **Construct the challenge set.** Create or import canonical cases, attach required materials, generate difficult variants, and curate benchmark dataset membership.
4. **Evaluate behavior.** Run an exact saved Harness Version against an exact Benchmark Version and inspect responses, Case-level Rubric evidence, comparisons, and rankings.
5. **Improve from evidence.** Start an Improvement Session with a bounded Goal Contract, explore candidates, evaluate them through the canonical path, and retain a current frontier.
6. **Return new learning.** Update coverage, correctness, cases, or the candidate according to what the evidence actually showed.
## Decision checkpoint
| Evidence says... | Responsible part of the loop | Change first |
| --- | --- | --- |
| Important behavior has no cases | Coverage Engineering or Weave | Coverage facet, construction pattern, or case set |
| Experts cannot apply the standard consistently | Correctness Elicitation | Policy scope, applicability, or rubric wording |
| A case cannot be interpreted or executed reliably | Weave and Project Input Schema | Input shape, case material, or world boundary |
| One saved candidate fails applicable rubrics | Trialground | Harness candidate or its runtime configuration |
| Several candidate branches improve different slices | Coevolve | Goal constraints, next experiment, or retained frontier |
| Result movement cannot be explained | Benchmark version and evaluation boundary | Versions, settings, mapping, or run metadata before any product change |
## How expert effort compounds
The loop should ask an expert only after agents have prepared the relevant structure and evidence. A Contribution can include selected Cases, source attachments, scoped statements, draft Policies, Rubric questions, or coverage uncertainty. Completed expert work can materialize as an attributable contributed Policy, Rubric, Case, or coverage observation through the owning workflow.
That same judgment can guide future case construction, determine which rubrics apply during evaluation, and identify missing correctness during improvement. Reuse across the loop is more valuable than maximizing the number of disconnected review actions.
## How product scope changes through the loop
Project foundations are reusable. Project Context, Reference Materials, policies, rubrics, Coverage Facets, Cases, and Harnesses do not belong to only one benchmark. A benchmark workspace selects and versions the relevant subset, manages coverage, coordinates contributions, evaluates candidates, and records improvement.
This scope distinction prevents accidental drift. Editing a project-level policy may affect several benchmarks. Changing dataset membership should create a new benchmark evidence boundary. Saving a Harness draft is different from selecting an exact saved Harness version for a Run.
## Before and after
| Before | Loop work | After |
| --- | --- | --- |
| Domain knowledge is distributed across people and files | Agent Setup and Correctness Elicitation organize it | Project context and governed correctness artifacts are inspectable |
| Examples are convenient rather than deliberate | Coverage Engineering and Weave shape the challenge set | Dataset representation and missing coverage are explicit |
| Candidate behavior is discussed from anecdotes | Trialground runs a versioned evaluation | Case-level rubric evidence and comparisons are available |
| Improvement is a sequence of untracked edits | Coevolve starts from pinned evidence | Candidate branches, receipts, chronology, and current frontier remain connected |
{% example-demo title="Changing a retrieval harness" %}
An evaluation shows failures only when current and superseded documents appear together. The team first confirms that the coverage slice and grounding rubric are valid. An Improvement Session pins those cases and the failing Harness version, then tests source-date filtering and citation-selection candidates. A stronger candidate becomes part of the current frontier only after a canonical evaluation produces the expected rubric evidence. If the work uncovers an unseen source-conflict pattern, that observation returns to Coverage Management.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Product quickstart](/docs/quickstart)
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Product map](/docs/getting-oriented/product-map)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Expert Contributions](/docs/expert-contributions)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page explains the approved operating loop. Linked product pages are the authority for exact controls and lifecycle states.
---
id: quickstart.product
title: Product quickstart
summary: Configure one project foundation, one benchmark slice, one expert contribution, one evaluation, and one evidence-backed improvement.
kind: quickstart
product_area: introduction
status: stable
updated: 2026-09-07
canonical: /docs/quickstart
---
# Product quickstart
Run one narrow correctness loop. The goal is not a large benchmark; it is a traceable chain from project context and deliberate coverage to expert-grounded standards, a versioned evaluation, and one justified next change.
## When to use it
Use this path for a new project or for an existing AI system whose correctness work is scattered across documents, examples, and informal expert feedback. Choose one behavior slice with a clear specialist owner.
## Decision checkpoint
| Starting point | First action | Ready to continue when... |
| --- | --- | --- |
| Agents do not understand the product or domain | Complete Agent Setup | Project Context and controlling Reference Materials are inspectable |
| Cases arrive in inconsistent shapes | Configure Project Input Schema | One input architecture and any required case materials are declared |
| Important behavior is not represented deliberately | Define Coverage Facets | Dimensions, Project Topics, and Case Construction Patterns name the slice |
| Correctness depends on tacit judgment | Request an Expert Contribution | The expert's scope, selected evidence, and required decisions are explicit |
| Cases and standards are ready | Create a benchmark snapshot and evaluate a saved Harness version | Exact cases, rubrics, candidate, and settings are bound to the Run |
## Prerequisites
- A Teammately project for the specialist AI behavior.
- An accountable project operator and at least one domain expert.
- A small number of representative examples or enough Reference Materials to construct them.
- A candidate system that can be represented by a saved Harness version before evaluation.
## Before and after
| Before | Action | After | Stop if... |
| --- | --- | --- | --- |
| Domain context is implicit | Write the Project Agent Brief and connect Reference Materials | Agents have explicit project understanding | Controlling sources are missing or contradictory without an owner |
| Inputs and supporting artifacts vary | Save Project Input Schema | Cases share one canonical content contract | Existing cases cannot satisfy the proposed schema |
| Expert knowledge is tacit | Request and complete a focused contribution | Policies, rubrics, cases, or coverage observations can be materialized | The request asks for a label without the evidence needed to explain it |
| Candidate behavior is anecdotal | Evaluate a saved Harness version | Results are traceable to cases and applicable rubrics | Dataset snapshot or candidate version is ambiguous |
| A weakness is confirmed | Start an Improvement Session from evidence | Candidate work follows a bounded Goal Contract | The requested outcome has no pinned measurement binding |
## Steps
1. Open or create the project and write the Project Agent Brief in **Agent Setup → Project Context**.
2. Add controlling knowledge through **Agent Setup → Reference Materials → Materials**, then inspect the published blocks in **Indexed Reference**.
3. Configure **Project Settings → Input Schema**. Select plain text, chat, or structured input and declare required case materials and accepted artifact families.
4. Create the smallest useful set of Coverage Facets: a Dimension, relevant Project Topics, and a Case Construction Pattern for the chosen behavior slice.
5. Add or construct cases in Assets, then select the intended cases in **Benchmark Datasets**. Confirm Representation and create or choose the appropriate snapshot.
6. In **Expert Contributions**, request one focused contribution. Select the expert, state the objective, attach or select the relevant cases, and include only the contribution components needed to resolve the question.
7. Inspect the completed contribution and materialize accepted policies, rubrics, cases, or coverage observations through their owning surfaces.
8. Save an exact Harness version. In **Benchmark Evaluations**, configure and run it against the selected benchmark version.
9. Inspect Dashboard and List results before using Compare or Arena. Trace important movement to case-level rubric evidence and run metadata.
10. If a candidate change is justified, open **Improve**, start from the relevant evidence, prepare and confirm the Goal Contract, and evaluate candidate work through the canonical Run path.
## Object and state changes
This path can create or update Project Context, Reference Materials items and indexed blocks, Project Input Schema, Coverage Facets, Cases, benchmark dataset membership and snapshots, Contributions, contributed artifacts, policies, rubrics, Harness drafts and saved versions, Runs, evaluation results, and Improvement Sessions. Each object keeps its own authority boundary; completing one step does not automatically approve or materialize every downstream artifact.
## Success criteria
- Another operator can identify the project context and source material used by agents.
- The case set conforms to the Project Input Schema and represents a named coverage slice.
- Expert judgment is attributable to a completed Contribution and its accepted artifacts.
- The evaluation binds an exact benchmark version to an exact saved Harness version.
- Any improvement work starts from pinned evidence and records its Goal Contract, candidate results, and current frontier.
## Common failure modes
- Treating Reference Materials as approved policies.
- Asking experts broad questions without selected cases or a concrete contribution objective.
- Evaluating an unsaved Harness draft or an unclear benchmark snapshot.
- Reading only an aggregate score and skipping failed case/rubric pairs.
- Starting improvement before the target and measurement evidence are resolved.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Agent Setup](/docs/agent-setup)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Expert Contributions](/docs/expert-contributions)
- [Benchmark Evaluations](/docs/benchmark-evaluations)
- [Improve](/docs/improve)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Dataset upload](/docs/troubleshooting/dataset-upload)
- [Unclear cases](/docs/troubleshooting/unclear-cases)
- [Benchmark results changed unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this quickstart connects the current public story to code-backed product surfaces. Follow the linked pages for exact states and controls.