Skip to content

task

Operating Teammately End to End

Coordinate current project configuration, input coverage, expert decisions, benchmark execution, improvement, and training workflows.

Operating Teammately End to End

Operate the loop by its object boundaries. A Project holds reusable understanding and standards; a Benchmark selects the population; recorded execution and attributable judgments explain each result.

Decision checkpoint

PhaseOwnerReady to continue when
Project understandingProject SettingsBrief and indexed source context are usable
Input coverageCoverage DesignRequirements identify meaningful situations
Specialist decisionsExpert Contributions and governanceStandards have explicit attribution and state
Fixed populationBenchmark DatasetsSnapshot and evaluation criteria are ready
Candidate measurementBenchmark EvaluationsExact Run and Trial outcomes are interpretable
Candidate changeHarness improvementGoal and permitted scope are explicit
Training dataWeights trainingDataset, splits and package are inspectable

Prerequisites

Use the intended workspace, Project and Benchmark. Identify who owns specialist judgment and who may manage the relevant settings, membership and candidate work.

Before and after

BeforeAfter
Unstructured sources and examplesCanonical inputs and reusable project understanding
Unclear correctnessAttributable standards and applicable Rubrics
Informal candidate claimsFixed measurement population and recorded outcomes
Disconnected follow-upAn explicit expert, coverage, engineering or training operation

Steps

  1. Maintain Project Context, Reference Materials and the configured Lemon instructions.
  2. Set the project input/material contract and scoring Regime.
  3. Import or create Cases and inspect their actual content.
  4. Use Canvas to define coverage requirements and inspect matching, unknown and remaining counts.
  5. Request the relevant expert work, inspect its plan and preserve separate artifact decisions.
  6. Review selected membership and create a Snapshot/Benchmark Version.
  7. In Benchmark Evaluations → Runs, choose managed execution or external submission. Managed execution uses a saved Harness Version, machine, and Trial count. For execution in your own environment, obtain the exact Case manifest, execute the candidate, and import its responses or recorded failures. Inspect evaluation completeness in Results and Dashboard.
  8. Use Lemon Goal for hosted candidate work or External Agents for the customer's repository.
  9. Use Inbox to decide coverage adoption. Use Fieldloop for captured application inquiries and Weights training for its separate data/job/evaluation path.

Object and state changes

Each phase changes its owning object. Current settings and membership can evolve while accepted Runs retain their recorded inputs and criteria.

Success criteria

A teammate can explain the result from its actual Case, standard, candidate and recorded operation without reconstructing facts from today's mutable settings.

Common failure modes

  • Treating connected sources as approved standards.
  • Treating a report or generated Case as automatic dataset adoption.
  • Replacing incomplete results with zero.
  • Assuming repeated launch always starts new execution.
  • Treating training completion or copied tasks as measured candidate improvement.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue