Add criteria and check packs
Each step lists its criteria: what the screen has to meet. Kaloko knows three kinds, and a good scenario uses the cheapest one that can answer the question.
Countable: a deterministic check
Anything you can count or read from the page is checked in the page itself, with the evidence attached:
criteria:
- id: AC1
text: The cart shows two items.
check: { type: deterministic, assert: count, selector: ".cart-item", min: 2, max: 2 }
- id: AC2
text: Paying ends on the thank-you page.
check: { type: deterministic, assert: url_matches, pattern: "/thanks$" }
- id: AC3
text: The Pay button uses the brand colour.
check: { type: deterministic, assert: style_equals, selector: "button.pay", property: background-color, token: color.brand.primary }
Other checks read text, attributes, the HTTP status, overflow on a phone or a value the page exposes (js_equals). The full list is in checks and packs. A step script never has to leave a marker for a criterion to find.
A question: a semantic check
What needs judgement goes to the evaluator as a question about the page:
- id: AC4
text: The primary call to action is clear.
check: { type: semantic, question: Does the page show one clear primary call to action?, scope: main }
The evaluator reads a reduced outline of the page and answers with a score. A clear yes passes, a clear no fails, and anything in between goes to a person. Add vision: true to the check when the look is the point (layout, imagery, "looks like the brand"): the screenshot then gets a second look from a vision model. Keep it to the steps that need it.
A person decides: a manual check
- id: AC5
text: Legal approved the wording of the consent.
check: { type: manual }
A manual criterion stays "not evaluated" until a reviewer decides it on the canvas.
Check packs
A pack adds a fixed set of checks to a step, so common checks are not written by hand. They are health checks: they run on every walk and show apart on the canvas, but they do not decide the step's verdict (the design pack is the exception). See Check the health of your app.
nodes:
- id: home
packs: [seo, a11y, perf, console]
| Pack | What it checks |
|---|---|
seo | one H1, title and description, canonical, hreflang, Open Graph, indexing, html lang |
usability | purpose, one primary action, link texts, unlabelled fields |
a11y | alt texts, names, labels, headings, contrast, landmarks, the Tab walk and visible focus |
perf | LCP, CLS, TTFB and page weight measured in the walk |
console | whether console errors touch what the step is for |
errors | error pages: a real 4xx or 5xx status, a way back, no stack trace |
app | app screens: labels on controls, touch target sizes, cut-off text, crashes |
design | an implementation against the accepted design: pixels, structure, tokens, contrast |
To change a limit, write your own criterion with the check's number as an AC id (for example AC401 for the LCP check HC401); yours wins and, being your own, it counts for acceptance. kaloko health adopt <step>:<check> writes such a criterion for you, with from: HC401 so it is clear where it came from. How verdicts come together is explained in how verdicts are decided.
Kaloko · latest · 2026-11-21