Teammately
Docs

reference

Scenarios

Define reusable mechanisms for constructing cases and steer how benchmarks use or avoid them.

Scenarios

Definition

A Scenario describes a reusable mechanism for building cases. It answers “how should this situation be constructed?” while Dimensions describe differentiating values and Topics describe subject matter.

Examples include conflicting authorities, missing prerequisite supporting information, ambiguous user intent, multi-step state change, or a plausible but superseded source. A good Scenario is portable across Topics rather than tied to one case's wording.

Fields, states, or lifecycle rules

Each Scenario has a name, description, origin, usage counts, examples, and benchmark statistics. Origins currently distinguish manual, AI-generated, expert-input, and imported Scenarios. Usage can show Case Pool cases, benchmark cases, requirements, and benchmark steering.

Scenario suggestions can be grounded in Topics, source material, existing cases and Scenarios, the Project Agent Brief, Dimensions and ontology, or expert input. Generated candidates include the proposed definition, why they were suggested, and source references. Accept or reject each candidate explicitly.

Benchmark steering

Canvas settings can use one of three modes:

  • System choose: allow construction to select appropriate Scenarios from the project library.
  • Prefer: encourage named Scenarios when building Coverage Stories and cases.
  • Avoid: prevent named Scenarios from shaping this benchmark's intended case supply.

Coverage Story tuples can also reference specific Scenarios. The tuple reference is the concrete construction instruction for that part of the story; the project Scenario remains reusable and independently editable.

Design rules

  • Name the mechanism, not the domain example. “Conflicting authorities” travels better than “Conflicting HR policies.”
  • Explain what makes a case instantiate the Scenario.
  • Keep Topics out of the Scenario definition unless they are only examples.
  • Do not encode an expected answer or rubric verdict as a scenario.
  • Inspect examples and usage before deleting or materially changing a Scenario.
  • Treat AI and expert suggestions as reviewable proposals. Origin is provenance, not approval.

Worked example

Example: portable scenario

01

Start

Behavior input

The project defines Plausible superseded authority
The project defines Plausible superseded authority: construct a case where an older source appears credible but a newer source controls. Coverage Stories reuse the Scenario for procurement, support, and compliance Topics while applying different source-condition and impact ontology values.
Reference / Coverage EngineeringTopicsMaintain source-grounded subject areas, editable groups, and Atlas relationships used to organize project and benchmark coverage.Task / Coverage DesignCoverage Design SettingsSet benchmark intent, facet handling, and coverage requirements while inheriting materials from Project Input Schema.Task / Coverage DesignCoverage StoriesDescribe meaningful input situations and testable facet combinations used in Coverage Design and Case preparation.Task / Coverage DesignReview Prepared CasesInspect generated inputs and materials against their captured coverage purpose before relying on benchmark membership.

Found something unclear?

Report outdated, unsupported, or confusing docs so we can fix the source page.

Report a docs issue

Continue learning

Related docs

Coverage Engineering Coverage Engineering Design the behavior space a benchmark must represent and connect reusable project facets to benchmark coverage work. Coverage Engineering Dimensions and Ontology Define reusable behavior axes and their allowed values, then inspect how cases and benchmarks cover them.