# Glossary
Generated: 2026-09-13T04:34:25.023Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: reference.glossary
title: Glossary
summary: Definitions for current Teammately capabilities, product surfaces, artifacts, and lifecycle states.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/reference/glossary
---
# Glossary
Use these definitions when choosing a product surface, writing an operating instruction, or interpreting evaluation evidence. Capitalization identifies current product capabilities, surfaces, and named artifacts where it matters.
## Definition
The glossary fixes the meaning of current capability, surface, artifact, and lifecycle terms. Capability names describe what Teammately enables; product-surface names describe where people operate; artifact names identify the state that must remain traceable.
### Capabilities
- **Correctness Elicitation:** Turns specialist judgment into attributable standards and cases through structured dialogue, tasks, and checkpoints.
- **Coverage Engineering:** Represents the behavior space that a benchmark must cover through facets, stories, patterns, and concrete Cases.
- **Weave:** Builds case-grounded agent behavior from project knowledge, policies, Rubrics, tools, materials, and worlds.
- **Trialground:** Evaluates saved Harness versions against versioned benchmark boundaries and exposes inspectable evidence.
- **Coevolve:** Improves agents and benchmark understanding together by feeding contribution, coverage, and evaluation evidence into governed iteration.
## Fields, states, or lifecycle rules
### Project foundations
- **Project Agent Brief:** The published project-understanding document in Project Context.
- **Reference Materials:** The Agent Setup surface for connecting and indexing source-backed project knowledge.
- **Reference block:** An indexed unit of project knowledge with source and generation identity.
- **Comparison Direction:** Project Asset that guides meaningful candidate-output differences in comparative expert work; it is not a coverage facet or approved standard.
- **Review Screen:** Project Asset that configures the context and inputs presented to an expert.
- **Policy:** A governed statement of required or prohibited behavior, with applicability and provenance.
- **Rubric:** An executable judgment criterion, commonly binary, that tests applicable behavior against a Policy.
- **Coverage Facet:** Reusable project structure for representing behavior space, including Dimensions, Project Topics, and Case Construction Patterns.
- **Case:** A reusable behavior situation with canonical input, optional case materials, metadata, and an optional executable world reference.
- **Harness:** The agent configuration being built and evaluated. A mutable Draft can be saved as an immutable Harness version.
- **Project Input Schema:** The project-level architecture governing canonical Case input and material fields.
- **Benchmark Run Metadata:** Benchmark-level descriptive context requested when an Evaluation Run is created; it is not a project-level template or version identity.
### Benchmark workspace
- **Benchmark:** A named workspace for a particular correctness boundary and its datasets, coverage, contributions, evaluations, and improvement work.
- **Dataset snapshot:** A reproducible selection and representation of benchmark Cases.
- **Coverage Story:** Benchmark-scoped intent connecting a behavior risk or need to concrete coverage work.
- **Expert Contribution:** A benchmark-scoped request for specialist judgment, containing one or more Tasks and optional Checkpoints.
- **Task:** A bounded unit of work inside a Contribution, such as form input, chat, interview, or Case Review.
- **Checkpoint:** An explicit confirmation boundary inside a Contribution.
- **Contributed artifact:** A policy, Rubric, Case, or coverage observation produced by an expert while retaining Contribution provenance.
- **Benchmark version:** The fixed dataset and correctness boundary used for reproducible evaluation.
- **Evaluation Run:** One execution of a saved Harness Version against a Benchmark Version, with response, Rubric outcomes, settings, mapping, and metadata. Execution trajectories are not currently exposed.
- **Compare:** A symmetric Benchmark Evaluations matrix whose columns are saved Harness Versions and whose rows group governed evidence within one Benchmark Version.
- **Arena:** A comparative evaluation surface inside Benchmark Evaluations.
- **Improvement Session:** A benchmark-scoped process that explores candidate Harness changes against an explicit goal and pinned evidence.
- **Goal Contract:** The Improvement Session definition of target evidence, success criteria, and constraints.
- **Evaluation receipt:** Canonical evidence that a particular candidate was evaluated under a particular boundary.
- **Current frontier:** The set of retained candidates that currently represent the session's best supported tradeoffs.
## Decision checkpoint
| If you mean... | Use... | Do not substitute... |
| --- | --- | --- |
| Reusable source-backed project knowledge | Reference Materials / Reference block | An untracked attachment or the latest file without generation identity |
| Human specialist work for a benchmark | Expert Contribution | A generic approval queue |
| The agent state actually evaluated | Saved Harness version | The mutable Harness Draft |
| A fixed evaluation boundary | Benchmark version | A Run or a score |
| One execution and its evidence | Evaluation Run | The Benchmark itself |
| Goal-directed candidate exploration | Improvement Session | An unversioned list of suggestions |
{% example-demo title="Distinguish benchmark and run" %}
Benchmark Version 4 fixes the selected Cases and governed standards. Harness Version 9 is the candidate. The Evaluation Run is the one execution of Harness Version 9 against Benchmark Version 4. Compare can place Harness Version 9 beside other saved Harness Versions in a symmetric evidence matrix, while an Improvement Session can use failed Case evidence as a pinned target for new candidates.
{% /example-demo %}
## Source confidence
Docs-backed and code-aligned: the product doctrine defines the five capabilities, and current navigation establishes the product-surface names. Use the linked code-backed pages when an exact field, state transition, or route behavior matters.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Product map](/docs/getting-oriented/product-map)
- [Product loop](/docs/product-loop)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Object model](/docs/object-model)
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [IDs and identity](/docs/reference/ids)
{% /related-card-grid %}
---
id: product-loop
title: The Teammately correctness loop
summary: See how coverage, elicitation, case construction, evaluation, and improvement reinforce one another.
kind: concept
product_area: introduction
status: stable
updated: 2026-08-22
canonical: /docs/product-loop
---
# The Teammately correctness loop
The correctness loop is how a team repeatedly turns domain knowledge into stronger AI behavior. It follows the five public capabilities while preserving a trace from every result back to the project context, expert contribution, case, policy, rubric, benchmark version, Harness version, and evaluation setting that made the result meaningful.
## Definition
1. **Design coverage.** Establish Dimensions, Project Topics, and Case Construction Patterns, then decide which combinations the benchmark must represent.
2. **Elicit correctness.** Use focused expert contributions to resolve policies, exceptions, applicability, disagreements, and binary rubric language.
3. **Construct the challenge set.** Create or import canonical cases, attach required materials, generate difficult variants, and curate benchmark dataset membership.
4. **Evaluate behavior.** Run an exact saved Harness Version against an exact Benchmark Version and inspect responses, Case-level Rubric evidence, comparisons, and rankings.
5. **Improve from evidence.** Start an Improvement Session with a bounded Goal Contract, explore candidates, evaluate them through the canonical path, and retain a current frontier.
6. **Return new learning.** Update coverage, correctness, cases, or the candidate according to what the evidence actually showed.
## Decision checkpoint
| Evidence says... | Responsible part of the loop | Change first |
| --- | --- | --- |
| Important behavior has no cases | Coverage Engineering or Weave | Coverage facet, construction pattern, or case set |
| Experts cannot apply the standard consistently | Correctness Elicitation | Policy scope, applicability, or rubric wording |
| A case cannot be interpreted or executed reliably | Weave and Project Input Schema | Input shape, case material, or world boundary |
| One saved candidate fails applicable rubrics | Trialground | Harness candidate or its runtime configuration |
| Several candidate branches improve different slices | Coevolve | Goal constraints, next experiment, or retained frontier |
| Result movement cannot be explained | Benchmark version and evaluation boundary | Versions, settings, mapping, or run metadata before any product change |
## How expert effort compounds
The loop should ask an expert only after agents have prepared the relevant structure and evidence. A Contribution can include selected Cases, source attachments, scoped statements, draft Policies, Rubric questions, or coverage uncertainty. Completed expert work can materialize as an attributable contributed Policy, Rubric, Case, or coverage observation through the owning workflow.
That same judgment can guide future case construction, determine which rubrics apply during evaluation, and identify missing correctness during improvement. Reuse across the loop is more valuable than maximizing the number of disconnected review actions.
## How product scope changes through the loop
Project foundations are reusable. Project Context, Reference Materials, policies, rubrics, Coverage Facets, Cases, and Harnesses do not belong to only one benchmark. A benchmark workspace selects and versions the relevant subset, manages coverage, coordinates contributions, evaluates candidates, and records improvement.
This scope distinction prevents accidental drift. Editing a project-level policy may affect several benchmarks. Changing dataset membership should create a new benchmark evidence boundary. Saving a Harness draft is different from selecting an exact saved Harness version for a Run.
## Before and after
| Before | Loop work | After |
| --- | --- | --- |
| Domain knowledge is distributed across people and files | Agent Setup and Correctness Elicitation organize it | Project context and governed correctness artifacts are inspectable |
| Examples are convenient rather than deliberate | Coverage Engineering and Weave shape the challenge set | Dataset representation and missing coverage are explicit |
| Candidate behavior is discussed from anecdotes | Trialground runs a versioned evaluation | Case-level rubric evidence and comparisons are available |
| Improvement is a sequence of untracked edits | Coevolve starts from pinned evidence | Candidate branches, receipts, chronology, and current frontier remain connected |
{% example-demo title="Changing a retrieval harness" %}
An evaluation shows failures only when current and superseded documents appear together. The team first confirms that the coverage slice and grounding rubric are valid. An Improvement Session pins those cases and the failing Harness version, then tests source-date filtering and citation-selection candidates. A stronger candidate becomes part of the current frontier only after a canonical evaluation produces the expected rubric evidence. If the work uncovers an unseen source-conflict pattern, that observation returns to Coverage Management.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Product quickstart](/docs/quickstart)
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Product map](/docs/getting-oriented/product-map)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Expert Contributions](/docs/expert-contributions)
{% /related-card-grid %}
## Source confidence
Doctrine-backed: this page explains the approved operating loop. Linked product pages are the authority for exact controls and lifecycle states.
---
id: orientation.product-map
title: Product map
summary: Navigate Teammately across workspace entry points, project foundations, benchmark workspaces, expert contribution UI, and administration.
kind: concept
product_area: introduction
status: stable
updated: 2026-09-07
canonical: /docs/getting-oriented/product-map
---
# Product map
Teammately separates reusable project foundations from benchmark-scoped work. The Main UI uses the selected project and benchmark to route operators to the right scope. Experts receive a focused Expert contribution UI. Admin Console owns organization-level controls, while AI-assisted background work prepares and coordinates bounded tasks.
> Surface routing
>
> Before changing an artifact, identify its scope. Project foundations can affect several benchmarks; dataset selection, Contributions, Evaluations, and Improvement Sessions belong to a selected benchmark or benchmark version.
## Definition
The **Main UI** begins at Project Home and groups project-level work into Correctness Governance, Coverage Facets, Assets, Agent Setup, and Project Settings. After a benchmark is selected, its workspace exposes Benchmark Overview, Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve.
The **Expert contribution UI** presents one Contribution and its form, chat, interview, case-review, checkpoint, waiting, and completion states. The expert does not need the full project navigation to supply attributable judgment.
The **Admin Console** contains organization administration such as members, groups, roles, domain controls, integrations, and other code-backed administrative surfaces. Public docs keep detailed security, billing, retention, and compliance claims outside the boundary unless separately verified.
**AI-assisted background work** can index reference material, prepare contributions, suggest coverage or standards, construct cases, run evaluations, and coordinate candidate exploration. Its outputs retain the authority of the owning artifact and workflow.
## Decision checkpoint
| Work | Scope | Surface |
| --- | --- | --- |
| Project purpose and knowledge | Project | Agent Setup |
| Policies and rubrics | Project | Correctness Governance |
| Dimensions, Topics, and construction patterns | Project | Coverage Facets |
| Reusable Cases and Harnesses | Project | Assets |
| Input architecture | Project | Project Settings |
| Benchmark-level run fields | Benchmark | Benchmark Evaluations |
| Selected Cases, representation, and snapshots | Benchmark | Benchmark Datasets |
| Coverage setup, Stories, Case Review, and Foundry | Benchmark | Coverage Management |
| Specialist requests and contributed artifacts | Benchmark | Expert Contributions |
| Runs, results, Compare, and Arena | Benchmark version | Benchmark Evaluations |
| Goal Contracts, candidates, and frontier | Benchmark version | Improve |
## How selection affects navigation
Project surfaces require a project. Benchmark surfaces also require a benchmark, and Evaluations or Improve may resolve the current benchmark version. If a destination is unavailable, confirm the current selectors before assuming that the feature or data is missing.
Project folders and search help users move across a larger workspace, but they do not change artifact ownership. Search results and creation actions should preserve the selected project or benchmark scope.
{% surface-map title="Teammately product surfaces" %}
{% /surface-map %}
{% example-demo title="Route a new rubric need" %}
An evaluation exposes inconsistent handling of expired agreements. The operator uses the benchmark workspace to request an Expert Contribution with the failed cases. The expert works in the focused contribution UI. The resulting rubric is reconciled in Correctness Governance at project scope, then included in a later benchmark version and evaluation.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Task index](/docs/operating-manual/task-index)
- [Operating Teammately end to end](/docs/getting-oriented/operating-teammately-end-to-end)
- [Product quickstart](/docs/quickstart)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [User roles](/docs/getting-oriented/user-roles)
- [Product boundaries](/docs/introduction/product-boundaries)
{% /related-card-grid %}
## Source confidence
Code-backed: current navigation and active project, benchmark, Contribution, and administration routes establish the scope and labels described here.
---
id: orientation.key-objects
title: Key objects and relationships
summary: Understand how project foundations, contributions, datasets, evaluations, and improvement artifacts connect.
kind: concept
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/getting-oriented/key-objects-and-relationships
---
# Key objects and relationships
Teammately's evidence is trustworthy when a reader can move from project understanding and specialist authority to the exact Case, Benchmark version, Harness version, Run, and Improvement Session involved. This page gives the shared object graph.
> Artifact state matters
>
> An object name is not enough. Preserve project or benchmark scope, version or snapshot identity, provenance, and lifecycle state whenever the object supports evaluation or improvement.
## Definition
A **Project Agent Brief** and published **Reference block** give agents project understanding. **Project Input Schema** governs canonical **Case** input and materials. **Dimensions**, **Project Topics**, and Case Construction Patterns define reusable coverage structure. A saved **Harness version** identifies an executable candidate.
A benchmark selects Cases into a **Dataset snapshot** and combines them with governed **Policies** and **Rubrics** through a **Benchmark version**. An **Expert Contribution** requests specialist judgment through one or more Tasks and Checkpoints. Its **Contributed artifact** can become a policy, rubric, case, or coverage observation while retaining provenance.
A **Run** evaluates a saved Harness Version against a Benchmark Version. Its response, Rubric outcomes, settings, mapping, and metadata form evaluation evidence. An **Improvement Session** pins target evidence through a Goal Contract, creates or receives candidates, records evaluation receipts and safe session narration, and maintains a **Current frontier**.
## Decision checkpoint
| Object | Scope | Relationship that must remain visible |
| --- | --- | --- |
| Project Agent Brief / Reference block | Project | What agents understood and which source generation was available |
| Case / Harness version | Project | Which reusable asset and exact candidate state was selected |
| Contribution / Checkpoint | Benchmark | Which expert supplied or confirmed the judgment |
| Policy / Rubric | Project governance | Which authority, applicability, cases, and provenance support it |
| Dataset snapshot / Benchmark version | Benchmark | Which cases and correctness boundary define evidence |
| Run | Benchmark version | Which Harness, settings, mapping, and metadata produced results |
| Improvement Session / Current frontier | Benchmark version | Which goal and evaluation receipts justify retained candidates |
{% artifact-map title="How correctness artifacts connect" %}
{% /artifact-map %}
## Static materials and executable worlds
Canonical Case content separates `content.input` from optional `content.case_materials`. Static execution support uses case-material references. A `world_instance_ref` represents an executable or queryable environment and follows a separate capability and lifecycle boundary. The rendered case view helps people and adapters inspect canonical content; it does not create another authoring source.
## Provenance across scopes
Project assets can be reused across benchmarks, while dataset snapshots, Contributions, Runs, and Improvement Sessions remain benchmark-scoped. Materializing a contributed policy moves its governed owner to project scope without erasing the benchmark Contribution that supplied it. Evaluating a candidate records the saved Harness version rather than whichever Draft is currently open.
{% example-demo title="Contribution to frontier" %}
An Expert Contribution confirms a source-authority Policy and Rubric from selected Cases. The Cases enter a Dataset snapshot and the standard enters a Benchmark version. A Run evaluates Harness version 8 and exposes three failures. An Improvement Session pins those failures, evaluates versions 9 and 10, and retains version 10 in the Current frontier with canonical evaluation receipts.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Object model](/docs/object-model)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Product map](/docs/getting-oriented/product-map)
{% /related-card-grid %}
## Source confidence
Code-backed: active navigation, canonical case contracts, Contribution surfaces, versioned evaluation routes, and Improvement Session contracts support this object graph.
---
id: object-model.overview
title: Object model
summary: Understand how project foundations, benchmark artifacts, contributions, evaluations, and improvement sessions connect.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/object-model
---
# Object model
Use this reference when a workflow depends on artifact ownership, identity, version history, or provenance. Teammately keeps reusable project foundations separate from benchmark-scoped evidence so a later reader can reconstruct what the agent knew, what was evaluated, who supplied judgment, and why a candidate was retained.
> Preserve scope and version
>
> An artifact name alone is not sufficient evidence. Preserve its project or benchmark scope, version or snapshot identity, lifecycle state, and provenance whenever it affects evaluation or improvement.
## Definition
A **Project** owns reusable foundations: the Project Agent Brief, Reference blocks, policies, rubrics, coverage facets, Cases, Harnesses, Review Screen configuration, Comparison Directions, and Project Input Schema. These objects can support more than one benchmark.
A **Benchmark** owns a particular correctness boundary. Its workspace selects Cases into Dataset snapshots, organizes benchmark-scoped coverage work, requests Expert Contributions, and creates Benchmark versions. A Benchmark version is the fixed boundary against which a saved Harness version is evaluated.
An **Evaluation Run** records the response, Rubric outcomes, mapping, settings, and metadata produced by one saved Harness Version against one Benchmark Version. It does not currently expose execution trajectories. An **Improvement Session** pins target evidence through a Goal Contract, evaluates candidates, retains canonical receipts and safe narrated session activity, and updates a Current frontier. The frontier is evidence-backed session state, not an unnamed replacement for the saved Harness.
## Decision checkpoint
| Question | Object boundary to preserve | Common mistake |
| --- | --- | --- |
| What did the agent understand? | Project Agent Brief and Reference block generation | Treating the latest source file as the knowledge used by an earlier run |
| What behavior was evaluated? | Case, Dataset snapshot, Benchmark version, and saved Harness version | Describing only a prompt and score |
| What standard applied? | Policy, applicability, Rubric, and provenance | Letting a Rubric stand in for the Policy it checks |
| Who supplied judgment? | Contribution, Task, Checkpoint, expert, and Contributed artifact | Treating an unattributed comment as governed truth |
| Why was a candidate retained? | Improvement Session, Goal Contract, evaluation receipt, and Current frontier | Treating an unsaved draft as the evaluated candidate |
## Fields, states, or lifecycle rules
### Project foundations
- **Project Agent Brief:** Published project understanding used to orient agents.
- **Reference block:** Indexed knowledge with source and generation identity.
- **Policy and Rubric:** Governed meaning and executable judgment criteria, with applicability and provenance.
- **Coverage Facet:** Reusable Dimensions, Project Topics, and Case Construction Patterns that describe important behavior space.
- **Case:** Canonical `content.input`, optional `content.case_materials`, metadata, and optional executable world reference.
- **Harness:** An agent configuration with mutable Draft state and immutable saved versions used for evaluation.
- **Project Input Schema:** The project-level architecture that governs canonical Case input and materials.
### Benchmark artifacts
- **Dataset snapshot:** A reproducible selection and representation of benchmark Cases.
- **Coverage Story:** Benchmark-scoped intent that connects coverage structure to concrete case work.
- **Expert Contribution:** A benchmark-scoped request containing Tasks, context, statuses, and optional Checkpoints.
- **Contributed artifact:** A policy, Rubric, Case, or coverage observation supplied through a Contribution with attributable provenance.
- **Benchmark version:** The fixed evaluation boundary used by Runs and Improvement Sessions.
- **Evaluation Run:** One execution with candidate, benchmark, response, Rubric outcomes, settings, mapping, and metadata identity.
- **Improvement Session:** A goal-directed candidate exploration process with pinned evidence, receipts, trajectories, and frontier state.
{% artifact-map title="How correctness artifacts connect" %}
{% /artifact-map %}
## Static materials and executable worlds
Case materials and executable worlds are intentionally different. `content.case_materials` holds canonical material data or references that can accompany the input. A `world_instance_ref` identifies an executable or queryable environment with a separate capability and lifecycle boundary. Rendering either for inspection does not create another authoring source.
{% example-demo title="From contribution to frontier" %}
A benchmark Contribution asks a procurement specialist to resolve three ambiguous Cases. The expert confirms a Policy and binary Rubric at a Checkpoint. The artifacts are reconciled into Correctness Governance while retaining their Contribution provenance. The Cases enter a Dataset snapshot and the standard enters Benchmark version 6. An Evaluation Run tests Harness version 11 and exposes two failures. An Improvement Session pins those failures, evaluates versions 12 and 13, and retains version 13 in the Current frontier with canonical evaluation receipts.
{% /example-demo %}
## Source confidence
Code-backed: current navigation establishes project and benchmark scopes; canonical case-content code establishes the input and materials boundary; the active Improve session contract establishes Goal Contract, candidate, receipt, and frontier concepts. This page describes public object semantics, not a public API or export schema.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [Cases](/docs/assets/cases)
- [Harnesses](/docs/assets/harnesses)
- [Glossary](/docs/reference/glossary)
{% /related-card-grid %}