Teammately Fall 2026 Edition
As AI becomes more capable and autonomous, our ability to judge and direct it has to grow alongside it.
For an enterprise, that ability can become a lasting advantage. Its experts understand the context in which a decision makes sense, the exceptions that change it, and the taste that distinguishes acceptable work from excellent work. Making that expertise part of AI development gives the organization something it can continue building as models and applications change.
We have evolved Teammately into AI Alignment Infrastructure around this conviction. Expert contributions stay connected to the standards teams adopt, the benchmark coverage they design, and the experiments and teaching material they use to develop AI. Our aim is to give each human judgment more impact, and give enterprises the capability to align and improve AI themselves.
For a long time, we’ve explored how the power of AI can augment our human capability to build a better model aligned with humans.
We also partnered with a few enterprises to find together what achieves the best expert-in-the-loop experience in UI and mechanisms, as well as how to build the fairest datasets to challenge the model despite constant change in user demand and model growth, and how to turn the benchmark into model improvements.
Today, we bring this work together in Socrates, our Fall 2026 Edition.
At its center is Lemon, our AI agent for alignment. Lemon prepares context and controlled alternatives, adapts its inquiry as experts respond, investigates conflicts and failures, and constructs cases and teaching examples. Experts establish the judgment. Accountable owners decide which standards to adopt. Engineers develop and assess the resulting changes.
More impact per human judgment
Consider a replacement for an out-of-stock tailored jacket. It may match the size and color while changing the drape and silhouette of the outfit. An experienced merchant considers the look the customer wants and the pieces they intend to wear with it. Several alternatives may be valid, while one better preserves that intent. This is an illustration of enterprise taste, not a customer result.
A preference records a choice. The reason behind it can help a team understand how to handle another situation.
Lemon Elicit prepares contributions around what needs to be learned. It can present controlled alternatives that make a trade-off concrete, ask an expert to correct an answer, or bring recorded actions into review when the final response alone cannot establish whether the work was acceptable. As the expert responds, Lemon follows the distinctions that need clarification.
The inquiry may uncover a requirement, a preference among valid alternatives, an exception, or a fact the original example left unstated. Lemon develops these findings into proposed Policies, Rubrics and coverage observations. Experts review whether the interpretation preserves what they meant.
Read how Lemon follows up on the expert’s distinction between furniture assembly and masonry work.
Standards that preserve where judgment changes
Enterprise expertise becomes useful across teams when its conditions remain clear. A local preference should not silently become a universal rule; a requirement should not be applied outside the situations it governs.
Policies organize intended behavior and its scope. Rubrics make requirements observable in responses or recorded actions. Applicability determines which criteria are relevant. The contribution behind a standard remains available, so another team can examine whether the judgment holds in its context.
Differences between experts, or even between judgments from the same expert, can expose an unstated assumption or missing condition. Lemon can investigate that conflict and prepare a focused follow-up. The work can improve both the standard and the understanding shared by the people responsible for it.
Take control, open a Policy and inspect its linked Rubrics and conditions.
Decide what the evidence needs to cover
The same judgment helps determine what a team needs to test. Covering every category once is not enough if routine situations dominate the benchmark or meaningful combinations never appear.
Coverage Design lets teams specify the conditions, combinations and proportions that matter to a particular question. Lemon proposes Coverage Stories that explain a representation need: a difficult combination, an ambiguity, or a decision boundary worth examining. Teams inspect and refine that direction, including what each combination should exercise and how many Cases it needs.
Lemon Weave constructs Cases, controlled alternatives and supporting material around that design. A Case has a reason to exist beyond increasing the dataset size. Review determines what enters the benchmark.
A representative assessment and a deliberate challenge set may draw on some of the same Cases while supporting different conclusions. Coverage makes that difference a decision the team can inspect.
Take control to explore Facets, Representation and Tuples beside the Cases preview. Check classification status before interpreting coverage.
Put the distinction to work
Trialground runs saved Python harnesses in managed disposable sandboxes. A harness brings together the code, instructions and configuration through which a model uses tools and task context. Trialground prepares declared dependencies and executes selected benchmark Cases. Teams can inspect retained outputs, execution status and recorded trajectories, then assess the behavior against the relevant Rubrics. Trajectory capture depends on the supported instrumentation in the harness.
The candidate, selected Cases and assessment basis stay attached to the run. Teams can repeat trials and examine the evidence behind a comparison rather than relying on a single successful attempt.
The next step depends on what the evidence reveals. An engineer can use Lemon Code to investigate a harness and inspect a proposed change. Lemon Goal investigates failures, develops competing harness approaches and tests candidates through Trialground against a measured objective. Protected behavior, permitted changes and a search budget bound that work. Engineers examine the evidence and decide what to integrate.
The same expert distinctions can guide post-training: further training that changes an existing model’s weights using teaching examples. Lemon Weave constructs target responses around policy boundaries and exceptions, or preferred and rejected responses to the same input, with the preference grounded in expert Rubrics.
With Weights Training, teams review those teaching examples and prepare a dataset for post-training. They inspect coverage and dataset splits, then prepare an exact export package or configure a supported provider job. Examples from the same construction family stay within one split, and assessment stays separate from the material used to teach the model. A completed training job is a starting point for assessing the resulting model, not evidence of improvement on its own.
These are distinct ways of developing AI against a basis the team understands, not one automatic pipeline from feedback to deployment.
Take control and open Results to compare Policy scores. The Runs list also contains entries awaiting results; inspect their status separately.
An in-house capability that continues to grow
Fieldloop brings selected interactions and results from actual use into coverage design and expert inquiry. Teams can submit them through an API or SDK, or bring them in from observability and online evaluation platforms such as LangSmith, Braintrust or Arize.
Our ambition is for enterprise expertise to remain useful as AI systems, models and the responsibilities entrusted to them change. Experts contribute the distinctions that matter. Engineers use them in what they test and what they teach. Lemon takes on more of the work connecting the two.
That is the direction we are building with Socrates.
What do we stand for?
We always want the alignment capabilities to live in-house, as we believe the best moat of an enterprise can come from its own in-house domain experts, not some outside experts whom everyone else could literally access if properly designed.
We’d like to empower AI engineers and product managers to lead in shaping the internal assets of expertise and tastes for their AI product at scale, leveraging the power of other agents, without relying on external human coordination vendors, as we believe it should be their fundamental capability.
We’d like to honestly assert that building reliable AI models to elicit tastes and files into rubrics needs another cycle of serious research and iterations, and that starting from this might waste the precious time of your experts.
We help you with these agent foundations, so your moat capabilities won’t be lost and will remain independently yours forever.
