Operating Teammately End to End
Operate the loop by its object boundaries. A Project holds reusable understanding and standards; a Benchmark selects the population; recorded execution and attributable judgments explain each result.
Decision checkpoint
| Phase | Owner | Ready to continue when |
|---|---|---|
| Project understanding | Project Settings | Brief and indexed source context are usable |
| Input coverage | Coverage Design | Requirements identify meaningful situations |
| Specialist decisions | Expert Contributions and governance | Standards have explicit attribution and state |
| Fixed population | Benchmark Datasets | Snapshot and evaluation criteria are ready |
| Candidate measurement | Benchmark Evaluations | Exact Run and Trial outcomes are interpretable |
| Candidate change | Harness improvement | Goal and permitted scope are explicit |
| Training data | Weights training | Dataset, splits and package are inspectable |
Prerequisites
Use the intended workspace, Project and Benchmark. Identify who owns specialist judgment and who may manage the relevant settings, membership and candidate work.
Before and after
| Before | After |
|---|---|
| Unstructured sources and examples | Canonical inputs and reusable project understanding |
| Unclear correctness | Attributable standards and applicable Rubrics |
| Informal candidate claims | Fixed measurement population and recorded outcomes |
| Disconnected follow-up | An explicit expert, coverage, engineering or training operation |
Steps
- Maintain Project Context, Reference Materials and the configured Lemon instructions.
- Set the project input/material contract and scoring Regime.
- Import or create Cases and inspect their actual content.
- Use Canvas to define coverage requirements and inspect matching, unknown and remaining counts.
- Request the relevant expert work, inspect its plan and preserve separate artifact decisions.
- Review selected membership and create a Snapshot/Benchmark Version.
- In Benchmark Evaluations → Runs, choose managed execution or external submission. Managed execution uses a saved Harness Version, machine, and Trial count. For execution in your own environment, obtain the exact Case manifest, execute the candidate, and import its responses or recorded failures. Inspect evaluation completeness in Results and Dashboard.
- Use Lemon Goal for hosted candidate work or External Agents for the customer's repository.
- Use Inbox to decide coverage adoption. Use Fieldloop for captured application inquiries and Weights training for its separate data/job/evaluation path.
Object and state changes
Each phase changes its owning object. Current settings and membership can evolve while accepted Runs retain their recorded inputs and criteria.
Success criteria
A teammate can explain the result from its actual Case, standard, candidate and recorded operation without reconstructing facts from today's mutable settings.
Common failure modes
- Treating connected sources as approved standards.
- Treating a report or generated Case as automatic dataset adoption.
- Replacing incomplete results with zero.
- Assuming repeated launch always starts new execution.
- Treating training completion or copied tasks as measured candidate improvement.
