Scenario file

A scenario is a YAML file (usually in qa/flows/) that describes one flow: its steps, the arrows between them and what each step must meet. Its format is qawalk.scenario.v1; kaloko validate <file> checks a file against it. How to write one step by step: Write a scenario.

schema: qawalk.scenario.v1
id: WEB-12-checkout
title: Checkout
locales: [en, cs]
viewports: { desktop: [1280, 800], mobile: iphone-15 }
environments: [local, staging]
nodes:
  - id: cart
    kind: screen
    title: { en: Cart, cs: Košík }
    path: /cart
    purpose: The shopper sees what they are buying and can continue.
    execution: { strategy: playwright, script: steps/cart.mjs, instructions: Add one product and open the cart. }
    criteria:
      - { id: AC1, text: Shows the total, check: { type: deterministic, assert: exists, selector: .cart-total } }
  - id: payment
    kind: screen
    title: Payment
    path: /checkout/payment
    packs: [a11y]
edges:
  - { from: cart, to: payment }

Top-level keys

schema, id, title, nodes and edges are required.

KeyWhat it holds
schemaalways qawalk.scenario.v1
idthe scenario's id (lowercase letters, digits, dashes); a ticket in it (WEB-12-…) links the issue
titlea name for people
kindtask (default) or flow
archivedtrue hides a retired scenario from the list; its runs stay reachable
sourcepath to the acceptance plan (ACCEPTANCE.md), used by kaloko validate --coverage
versiona number you raise when the flow changes
localeslanguages to walk, e.g. [en, cs]
viewportsnames with a size ([1280, 800]), a device preset (iphone-15, ipad-landscape), or a list of preset names
browserschromium (default), webkit, firefox
color_schemeslight, dark
reduced_motionno-preference, reduce
forced_colorsnone, active (Windows high contrast)
display_modesbrowser, standalone (an installed web app)
devicesapp scenarios: the emulators, simulators or phones to walk on
token_modesdesign environments: which modes of the token set to render ([brand-a, brand-b]), or false for the environment's default_token_mode only; not set: every mode
environmentsenvironments of the config the scenario may run on
readonlytrue for a scenario that only reads; required on a read-only environment
platformweb (default), android, ios, electron, desktop, windows
healthacceptance: [seo]: packs whose checks count for acceptance in this scenario; without it pack checks are advisory health (Check the health of your app). site: true (or { paths, ai_crawlers, sitemaps, environments }) adds the site checks once per run: robots.txt, sitemaps, llms.txt, redirects (Criteria, checks and packs). design_packs: [console]: code-only packs (seo, aeo, perf, console, errors) that still run on design environments, where they are off otherwise
trafficSearch Console and GA4 numbers for this scenario's pages; replaces the config's traffic (Configuration file)
tagsfree labels
refsrequirement references the scenario verifies, as your tracker or QMS writes them ([URS-12, FS-3.4]); they reach every criterion in the trace matrix (quality evidence)
nodesthe steps (below)
edgesarrows between steps (below)
laneslabels of the rows on the canvas, index = lane number (a link ending in #lane=<number> opens that row)
issuethe ticket: number, repo (owner/name), key (Jira or Linear, ENG-42), url, title
sessionspeople in the flow, each with title, optional mailbox, account, blocked_on
projectthe Kaloko project for its runs; overrides the config. When the config asks every scenario for its own project (a comment, or a product with projects:), kaloko validate and kaloko start warn when it is missing
product, modulewhich product and module of products: the flow shows, for the changelog and docs
capturecapture quality for every step (below)
piiredact or pseudonymize: personal data hidden in every capture, found automatically (how-to)

Step keys (nodes)

Every step needs id, kind and title.

KeyWhat it holds
idthe step id, used in links, --steps and step references (kaloko:<scenario>/<step>)
kindscreen, email, external (a third-party screen), decision, stack (many pages of one template)
titletext, or one per language ({ en: Cart, cs: Košík })
paththe address relative to base_url, or one per language; a deep link in app scenarios; {random} makes an address that cannot exist
purposeone sentence about what the step is for; the evaluator reads it
refsrequirement references of the step; they reach its criteria in the trace matrix
lanethe row on the canvas
readonlythis step only reads
executionstrategy (playwright, script, agent, manual), script (path to the step script), instructions (required), requires (e.g. browser)
maile-mail steps: subject and to
criteriawhat the step must meet (below)
sessionwhich person from sessions does this step
affectsfile globs that shape the step, for kaloko walk --affected, or always
packsready-made sets of health checks: seo, usability, a11y, perf, console, errors, app, aeo, locale, and design, which counts for acceptance
expect_statusthe HTTP status the step should answer with, e.g. 404 (or a list). A status of 400 or more marks an error page: the errors pack checks HC601 (soft 404) and HC605 (the explanation) only on error pages, that is with expect_status or on a 4xx/5xx answer
capturecapture quality for this step only
recordtrue records this step even when the walk does not; false never
ignoreregions left out of pixel comparisons: { selector } or { rect: [x, y, w, h], viewport }, optionally limited by viewports, locales, with a note
itemsstack steps: where the pages come from (urls, file, sitemap (gzipped too; sitemap: robots or robots: true reads the sitemaps of /robots.txt for the walked locale) with match/exclude/limit, crawl, script). match and exclude are a regex or a map by locale ({ default: '^/eshop/', fi: '^/fi/tuotteet/' }) and apply before limit for every source; rotate: week or rotate: run takes limit pages from a different place each week or run, so a big list is covered in turns; variants: hreflang (with source_locale) takes the list in one locale and walks each page's hreflang alternate in the others, so a page and its translations are one page of the stack and a missing translation is reported (HC1005), in kaloko walk and hosted runs alike; params go to script, which also gets node (id, title, lane, lane_title, session); a script or a JSON list may give each page edit_url (the page in the CMS) and content_ref (its record), and resolve: steps/resolve.mjs (default export (item, ctx) => ({ edit_url, content_ref })) adds them to the walked pages, for the stack index and kaloko export findings; on a design environment the variants are folders of the step, named by urls or found as subfolders with an index.html
fieldsstack steps: up to 12 columns of the stack index (id, label, from: title, h1, description, canonical, lang, words, http, jsonld_types, a meta tag such as og:title or meta:robots, a CSS selector, selector@attribute, count:selector)
detailsnamed parts of a screen step for guides: { id, name, selector }, optionally context (section, card, page or a CSS selector), padding and states; shown in docs with kaloko:<scenario>/<step>#<id> (Publish living docs)
statesother states the step is captured in, e.g. [empty, error, loading]; any name works (below)

Criteria

Each criterion has an id (AC1, AC12b), a text for people, an optional highlight (a CSS selector to outline on the screenshot), optional refs (requirement IDs it verifies, free text) and exactly one check: deterministic, semantic or manual. Every check type and option is listed in Criteria, checks and packs.

environments: [production] limits a criterion to some environments, states: [empty] to some step states. On the other ones it shows as not applicable and does not make the step partial.

Step states

A cart is not one screen: it can be empty, fail to load or still be loading. states: on a step captures each of them next to the default one:

  - id: cart
    kind: screen
    title: Cart
    path: /cart
    states: [empty, error, loading]
    execution: { strategy: playwright, script: steps/cart.mjs, instructions: Open the cart. }
    criteria:
      - { id: AC1, text: Shows the total, check: { type: deterministic, assert: exists, selector: .cart-total } }
      - { id: AC2, text: Says the cart is empty, states: [empty], check: { type: semantic, question: Does the page say the cart is empty? } }

Edges

KeyWhat it holds
from, tostep ids
labeltext on the arrow, e.g. the choice at a decision
kindmain (default) or alt for an alternative path

Capture

KeyWhat it holds
formatwebp (default) or png (lossless, for print)
scale1 or 2 (device pixels of the screenshot)
qualityWebP quality from 0.3 to 1
maskpersonal data hidden in every output of the step: CSS selectors, auto (detection), or both; the scenario's and the step's add up (how-to)
annotatenumbered marks on the elements of criteria with highlight
full_pagethe whole page (default, stack pages too) or, with false, the first screen: several times smaller on long pages
screenshotthumb keeps only the small thumbnail (640 px wide, as tall as the captured page) as the step's picture: for stacks of hundreds of pages whose checks read the page, not the picture, the smallest together with full_page: false; default full
recordtrue, step, flow, failed or off; see kaloko walk --record
tracekeep a trace of actions, requests and console per step (default true)

A step's capture wins over the scenario's, and the scenario's over the config's; masks are the exception and add up.

Kaloko · latest · 2026-11-21