Write a scenario

A scenario is the flow Kaloko walks: a YAML file with the screens, the way between them and what each screen has to meet. Task flows usually sit next to the task (docs/tasks/TASK-123/qa/scenario.yml), long-lived flows such as the checkout in qa/flows/.

Let the agent write it #

With the skill installed, ask your agent instead of typing YAML:

Read the task in docs/tasks/TASK-123, write its acceptance plan and a Kaloko scenario, and validate it.

The agent writes two files: ACCEPTANCE.md for people (goal, steps, criteria AC1, AC2…) and the scenario for the runner. When the intent changes, it updates the plan first and then the scenario.

The parts of the file #

schema: qawalk.scenario.v1
id: TASK-123-checkout
title: Checkout
kind: task                # task | flow
product: shop             # optional, with module:
locales: [en, cs]
viewports:
  desktop: [1280, 800]
  mobile: [390, 844]
environments: [local, staging]
nodes:
  - id: cart
    kind: screen
    title: Cart
    path: /cart
    purpose: The visitor sees what they are buying and can go on to pay.
    execution:
      strategy: playwright   # playwright | agent | manual
      script: steps/cart.mjs
    criteria:
      - id: AC1
        text: The page has exactly one main heading.
        check: { type: deterministic, assert: count, selector: h1, min: 1, max: 1 }
edges:
  - { from: cart, to: payment, label: continue }

• Step ids (cart) stay stable: links, comments, approvals and docs pictures refer to them.
• purpose is one sentence about what the screen is for. The evaluator reads it.
• execution says who moves to the screen: a Playwright script, an agent that captures with kaloko capture, or a person.
• Other node kinds: decision for a branch, email for a message from the mailbox, stack for many pages of one template.

Every key is in the scenario file reference. To add checks, see Add criteria and check packs.

Check it #

npx kaloko validate qa/flows/checkout.yml
npx kaloko validate qa/flows/checkout.yml --coverage

validate checks the schema and the references between steps. --coverage compares the criteria with the acceptance plan and lists what the plan asks for and the scenario does not check.

Start from public pages #

For a marketing site or documentation, start from a draft:

npx kaloko draft https://example.com/ https://example.com/pricing

Kaloko fetches the pages and writes a read-only production scenario and its plan, with the page title, H1 and description as the purpose and the seo, a11y, perf and console packs. It also adds a step for an address that does not exist, so the 404 page is checked too. Then sharpen each purpose and add what the page really has to meet.