Scenario file
A scenario is a YAML file (usually in qa/flows/) that describes one flow: its steps, the arrows between them and what each step must meet. Its format is qawalk.scenario.v1; kaloko validate <file> checks a file against it. How to write one step by step: Write a scenario.
schema: qawalk.scenario.v1
id: WEB-12-checkout
title: Checkout
locales: [en, cs]
viewports: { desktop: [1280, 800], mobile: iphone-15 }
environments: [local, staging]
nodes:
- id: cart
kind: screen
title: { en: Cart, cs: Košík }
path: /cart
purpose: The shopper sees what they are buying and can continue.
execution: { strategy: playwright, script: steps/cart.mjs, instructions: Add one product and open the cart. }
criteria:
- { id: AC1, text: Shows the total, check: { type: deterministic, assert: exists, selector: .cart-total } }
- id: payment
kind: screen
title: Payment
path: /checkout/payment
packs: [a11y]
edges:
- { from: cart, to: payment }Top-level keys
schema, id, title, nodes and edges are required.
| Key | What it holds |
|---|---|
schema | always qawalk.scenario.v1 |
id | the scenario's id (lowercase letters, digits, dashes); a ticket in it (WEB-12-…) links the issue |
title | a name for people |
kind | task (default) or flow |
archived | true hides a retired scenario from the list; its runs stay reachable |
source | path to the acceptance plan (ACCEPTANCE.md), used by kaloko validate --coverage |
version | a number you raise when the flow changes |
locales | languages to walk, e.g. [en, cs] |
viewports | names with a size ([1280, 800]), a device preset (iphone-15, ipad-landscape), or a list of preset names |
browsers | chromium (default), webkit, firefox |
color_schemes | light, dark |
reduced_motion | no-preference, reduce |
forced_colors | none, active (Windows high contrast) |
display_modes | browser, standalone (an installed web app) |
devices | app scenarios: the emulators, simulators or phones to walk on |
environments | environments of the config the scenario may run on |
readonly | true for a scenario that only reads; required on a read-only environment |
platform | web (default), android, ios, electron, desktop, windows |
tags | free labels |
nodes | the steps (below) |
edges | arrows between steps (below) |
lanes | labels of the rows on the canvas, index = lane number |
issue | the ticket: number, repo (owner/name), key (Jira or Linear, ENG-42), url, title |
sessions | people in the flow, each with title, optional mailbox, account, blocked_on |
project | the Kaloko project for its runs; overrides the config |
product, module | which product and module of products: the flow shows, for the changelog and docs |
capture | capture quality for every step (below) |
Step keys (nodes)
Every step needs id, kind and title.
| Key | What it holds |
|---|---|
id | the step id, used in links, --steps and step references (kaloko:<scenario>/<step>) |
kind | screen, email, external (a third-party screen), decision, stack (many pages of one template) |
title | text, or one per language ({ en: Cart, cs: Košík }) |
path | the address relative to base_url, or one per language; a deep link in app scenarios; {random} makes an address that cannot exist |
purpose | one sentence about what the step is for; the evaluator reads it |
lane | the row on the canvas |
readonly | this step only reads |
execution | strategy (playwright, script, agent, manual), script (path to the step script), instructions (required), requires (e.g. browser) |
mail | e-mail steps: subject and to |
criteria | what the step must meet (below) |
session | which person from sessions does this step |
affects | file globs that shape the step, for kaloko walk --affected, or always |
packs | ready-made sets of criteria: seo, usability, a11y, perf, console, errors, app, design |
capture | capture quality for this step only |
record | true records this step even when the walk does not; false never |
ignore | regions left out of pixel comparisons: { selector } or { rect: [x, y, w, h], viewport }, optionally limited by viewports, locales, with a note |
items | stack steps: where the pages come from (urls, file, sitemap with match/exclude, crawl, script) |
fields | stack steps: up to 12 columns of the stack index (id, label, from) |
Criteria
Each criterion has an id (AC1, AC12b), a text for people, an optional highlight (a CSS selector to outline on the screenshot) and exactly one check: deterministic, semantic or manual. Every check type and option is listed in Criteria, checks and packs.
Edges
| Key | What it holds |
|---|---|
from, to | step ids |
label | text on the arrow, e.g. the choice at a decision |
kind | main (default) or alt for an alternative path |
Capture
| Key | What it holds |
|---|---|
format | webp (default) or png (lossless, for print) |
scale | 1 or 2 (device pixels of the screenshot) |
quality | WebP quality from 0.3 to 1 |
mask | CSS selectors covered by a solid box on the image |
annotate | numbered marks on the elements of criteria with highlight |
full_page | the whole page (default) or, with false, the first screen |
record | true, step, flow, failed or off; see kaloko walk --record |
trace | keep a trace of actions, requests and console per step (default true) |
A step's capture wins over the scenario's, and the scenario's over the config's.