Teammately Docs
Docs menu

reference

Object model

Understand how project foundations, benchmark artifacts, contributions, evaluations, and improvement sessions connect.

Object model

Use this reference when a workflow depends on artifact ownership, identity, version history, or provenance. Teammately keeps reusable project foundations separate from benchmark-scoped evidence so a later reader can reconstruct what the agent knew, what was evaluated, who supplied judgment, and why a candidate was retained.

Definition

A Project owns reusable foundations: the Project Agent Brief, Reference blocks, policies, rubrics, coverage facets, Cases, Harnesses, Review Screen configuration, Comparison Directions, and Project Input Schema. These objects can support more than one benchmark.

A Benchmark owns a particular correctness boundary. Its workspace selects Cases into Dataset snapshots, organizes benchmark-scoped coverage work, requests Expert Contributions, and creates Benchmark versions. A Benchmark version is the fixed boundary against which a saved Harness version is evaluated.

An Evaluation Run records the response, Rubric outcomes, mapping, settings, and metadata produced by one saved Harness Version against one Benchmark Version. It does not currently expose execution trajectories. An Improvement Session pins target evidence through a Goal Contract, evaluates candidates, retains canonical receipts and safe narrated session activity, and updates a Current frontier. The frontier is evidence-backed session state, not an unnamed replacement for the saved Harness.

Decision checkpoint

QuestionObject boundary to preserveCommon mistake
What did the agent understand?Project Agent Brief and Reference block generationTreating the latest source file as the knowledge used by an earlier run
What behavior was evaluated?Case, Dataset snapshot, Benchmark version, and saved Harness versionDescribing only a prompt and score
What standard applied?Policy, applicability, Rubric, and provenanceLetting a Rubric stand in for the Policy it checks
Who supplied judgment?Contribution, Task, Checkpoint, expert, and Contributed artifactTreating an unattributed comment as governed truth
Why was a candidate retained?Improvement Session, Goal Contract, evaluation receipt, and Current frontierTreating an unsaved draft as the evaluated candidate

Fields, states, or lifecycle rules

Project foundations

  • Project Agent Brief: Published project understanding used to orient agents.
  • Reference block: Indexed knowledge with source and generation identity.
  • Policy and Rubric: Governed meaning and executable judgment criteria, with applicability and provenance.
  • Coverage Facet: Reusable Dimensions, Project Topics, and Case Construction Patterns that describe important behavior space.
  • Case: Canonical content.input, optional content.case_materials, metadata, and optional executable world reference.
  • Harness: An agent configuration with mutable Draft state and immutable saved versions used for evaluation.
  • Project Input Schema: The project-level architecture that governs canonical Case input and materials.

Benchmark artifacts

  • Dataset snapshot: A reproducible selection and representation of benchmark Cases.
  • Coverage Story: Benchmark-scoped intent that connects coverage structure to concrete case work.
  • Expert Contribution: A benchmark-scoped request containing Tasks, context, statuses, and optional Checkpoints.
  • Contributed artifact: A policy, Rubric, Case, or coverage observation supplied through a Contribution with attributable provenance.
  • Benchmark version: The fixed evaluation boundary used by Runs and Improvement Sessions.
  • Evaluation Run: One execution with candidate, benchmark, response, Rubric outcomes, settings, mapping, and metadata identity.
  • Improvement Session: A goal-directed candidate exploration process with pinned evidence, receipts, trajectories, and frontier state.

Project foundations

Project Agent BriefStable project context for Teammately agents.
Reference blockPublished project knowledge available to agents.
CaseCanonical input and materials available to benchmarks.
Harness versionExact executable candidate for evaluation.
DimensionCoverage axis for slicing behavior.
Project TopicDomain subject represented in coverage.

Correctness and contribution

ContributionBenchmark-scoped request for specialist judgment.
TaskForm, chat, interview, or case-review work.
CheckpointExplicit confirmation of consequential learning.
Contributed artifactAttributable policy, rubric, case, or coverage observation.
PolicyGoverned rule for expected behavior and applicability.
RubricObservable criterion used in evaluation.

Benchmark and improvement

Dataset snapshotExact selected-case boundary for a benchmark.
Benchmark versionVersioned correctness and dataset boundary.
RunOne managed evaluation of a saved candidate.
Evaluation receiptObservable result tied to candidate and benchmark.
Improvement SessionDurable goal, work, candidates, and chronology.
Current frontierCandidates retained by current evidence and constraints.

Static materials and executable worlds

Case materials and executable worlds are intentionally different. content.case_materials holds canonical material data or references that can accompany the input. A world_instance_ref identifies an executable or queryable environment with a separate capability and lifecycle boundary. Rendering either for inspection does not create another authoring source.

Worked example

From contribution to frontier

A benchmark Contribution asks a procurement specialist to resolve three ambiguous Cases. The expert confirms a Policy and binary Rubric at a Checkpoint. The artifacts are reconciled into Correctness Governance while retaining their Contribution provenance. The Cases enter a Dataset snapshot and the standard enters Benchmark version 6. An Evaluation Run tests Harness version 11 and exposes two failures. An Improvement Session pins those failures, evaluates versions 12 and 13, and retains version 13 in the Current frontier with canonical evaluation receipts.

Source confidence

Code-backed: current navigation establishes project and benchmark scopes; canonical case-content code establishes the input and materials boundary; the active Improve session contract establishes Goal Contract, candidate, receipt, and frontier concepts. This page describes public object semantics, not a public API or export schema.

Related task pages

Task / Operating ManualFirst correctness loopComplete one traceable path from project context and benchmark coverage to expert judgment, evaluation evidence, and improvement.Task / Expert ContributionsRequest an Expert ContributionCreate a focused benchmark contribution with an accountable expert, clear objectives, selected cases, attachments, and appropriate task components.Task / Benchmark EvaluationsRun a Benchmark EvaluationLaunch exact active Harness Versions against an immutable Benchmark Version as standard or repeated Run Groups.Task / ImproveStart an Improvement SessionStart from benchmark evidence, prepare a measurable Goal Contract, and choose bounded Work or Evolve behavior.

Related reference pages

Concept / Reference LibraryKey objects and relationshipsUnderstand how project foundations, contributions, datasets, evaluations, and improvement artifacts connect.Reference / AssetsCasesUnderstand canonical project cases, their input and materials, and how they become members of benchmark datasets.Reference / AssetsHarnessesBuild versioned Python Harness bundles, validate and debug Drafts, publish exact Versions, and activate them for benchmark evaluations.Reference / Reference LibraryGlossaryDefinitions for current Teammately capabilities, product surfaces, artifacts, and lifecycle states.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Reference Library Key objects and relationships Understand how project foundations, contributions, datasets, evaluations, and improvement artifacts connect. Learning Manual The Teammately correctness lifecycle Follow specialist AI work from project foundations through coverage, elicitation, construction, evaluation, and improvement. Assets Assets Manage reusable project cases, worlds, project tools, harnesses, weights, comparison directions, and review screens before selecting them for benchmark work. Benchmark Evaluations Benchmark Evaluations Run and inspect exact Harness Versions against an immutable Benchmark Version through Dashboard, List, Arena, and Compare.
AI context