Criteria, checks and packs
A criterion is one thing a step must meet. It has an id (AC1), a text for people and exactly one check. How to add them to a scenario: Add criteria and packs. How the results turn into a verdict: How verdicts are decided.
Kinds of check
type | When to use it | Options |
|---|---|---|
deterministic | anything countable or exact: presence, counts, text, the URL, styles, values in the page | assert and its options (below) |
semantic | judgement: is the copy clear, is there one primary action | question (English, yes = met), true and false (what counts as yes and no), scope (CSS selector sent to the evaluator), about: console (read the console errors instead of the page), vision: true (also show the screenshot) |
manual | neither works; a reviewer decides on the canvas | note |
Numbers, dates and counts belong in deterministic checks, never in a semantic question. Ask one thing per question.
Deterministic checks (assert)
Common options: selector (CSS), pattern (a regex) with flags (i, m, s), value, min, max, all (every match must pass, not just one).
Page content
assert | Passes when |
|---|---|
exists | selector matches at least one element |
absent | selector matches nothing |
count | the number of matches is between min and max |
text_matches | the element's visible text matches pattern |
text_absent | no visible text matches pattern |
text_equals | the whole text equals value (whitespace collapsed) |
attr_equals | attribute attr equals value |
attr_matches | attribute attr matches pattern |
no_overflow | the page does not scroll sideways |
no_console_errors | the console has no errors |
Address, metadata and links
assert | Passes when |
|---|---|
title_matches | the page title matches pattern |
meta_matches | the content of the selector meta tag matches pattern |
link_matches | the href of the selector link matches pattern |
url_matches | the URL after the step matches pattern; part picks href, path, query, hash, host or origin |
url_equals | the URL equals value (a path or a whole URL); ignore_query drops ?… and #… |
http_status | the document's status is between min and max (default 200–299), or equals value |
link_home | the page links back to the home page |
links_ok | every same-origin link answers below 400 (max links per page, pattern skips paths, follow outside read-only environments) |
hreflang_pairs | at least min languages, each version answers 200 and links back |
Structured data and answer engines
assert | Passes when |
|---|---|
jsonld_present | the page has a <script type="application/ld+json">; with types: [Product, BreadcrumbList] its structured data contains each of them (graphs, lists and nested values such as offers count, and so does microdata) |
jsonld_valid | every JSON-LD block parses as JSON; a page without JSON-LD passes. A failure names the block and the parse error |
aeo_faq_schema | a page that asks and answers (two or more details > summary, dl > dt, or h2–h4 ending with "?") has FAQPage or QAPage structured data. A page without questions is not evaluated |
aeo_answer_first | the first paragraph after the H1 is between min and max characters (default 40–320): a short answer before the detail |
llms_txt | /llms.txt of the site answers 200 with text, not an HTML page. One request per site and walk |
nojs_parity | the page's raw HTML, fetched without JavaScript, holds what the rendered page shows: the title and canonical match, the meta description, H1 and every JSON-LD type are there, and the text of the body is at least ratio of the rendered one (default 0.3; judged when the rendered page has 200 characters or more). user_agent: fetches as a crawler: gptbot, oai-searchbot, claudebot, perplexitybot, googlebot, bingbot or a whole user agent string |
Without JavaScript (nojs_parity). Many AI crawlers read the HTML the server sends and never run scripts. The check makes one more GET per page, read in Node without a browser (at most 500,000 characters, 15 s), and compares it with the page the walk captured. Titles and descriptions are compared as people read them (–, », no-break spaces and the like decoded). The text counts the whole body on both sides, hidden parts too, so a part that Next.js streams into a hidden element counts as it does after rendering. With user_agent: it comes like that crawler: without cookies, with the bot's user agent and the walk's Accept-Language, and only after robots.txt allows it. A page robots.txt closes to the bot is not fetched and shows "blocked by robots.txt for GPTBot" as not evaluated; so does every page when robots.txt answers 5xx, because crawlers then fetch nothing. When the site turns the request away (401, 403, 429 or 503, a challenge page, a redirect to another origin such as a sign-in), the check is not evaluated either and says who was turned away: the bot's user agent (the browser's gets the page), the missing cookies of a sign-in or cookie gate, or everyone. That is not a rendering problem. Without user_agent: it uses the walker's browser user agent and session and does not read robots.txt. The evidence lists what is missing without JavaScript, for example GPTBot without JavaScript: no H1 (rendered: "Photo books"); JSON-LD Product only after JavaScript; body text 0 of 1840 characters (0 %), at least 30 % expected. A title in another language says so, with both lang values. A page longer than the limit is cut, and its JSON-LD is then not judged. Each page and user agent is fetched once per walk (the walk keeps a small summary, not the page) and robots.txt once per site and bot. Compare landing steps: the raw HTML is the page as it loads, not after the step's clicks.
Every capture also records structured_data: the number of JSON-LD blocks, the invalid ones, the schema.org types and the recognized ones (Product, Offer, AggregateOffer, BreadcrumbList, FAQPage, QAPage, Article, BlogPosting, NewsArticle, Organization, WebSite). A stack step's items.json counts pages per type, pages without structured data and pages with invalid JSON-LD (structured_data at the top of the index and in the step's stack).
Language versions (the locale pack uses these)
They compare the page with the locale it was walked in (cs, de, en_US). A locale that is not a language tag (an ISO 639-1 language such as cs; cz is a country) makes them not applicable.
assert | Passes when |
|---|---|
locale_lang | html lang names the language of the walked locale (de-DE for de); a different region is mentioned, not failed |
locale_hreflang | the page's own hreflang names it with its language, every code is valid (en-GB, not en_UK), and where addresses carry a language (/de/…) the code agrees. A page without hreflang is not evaluated |
locale_text | the page text reads as the walked language. Kaloko counts common words of 26 languages (most of Europe and Turkish); pages under min words (default 40), other languages and close pairs (Danish, Norwegian, Swedish; Croatian, Slovenian) are not judged |
locale_prices | prices with decimals use the locale's separator (12,50 € in de, $12.50 in en). Pages without such prices are not evaluated |
Styles and values in the page
assert | Passes when |
|---|---|
style_equals | the computed CSS property equals value or a design token; op compares numbers (>=), tolerance allows px rounding |
style_matches | the computed property matches pattern |
js_equals | one read-only JavaScript expression in the page returns value (or matches pattern, or compares with op) |
Accessibility (the a11y pack uses these)
assert | Passes when |
|---|---|
a11y_images_alt | images have alternative text |
a11y_names | links and buttons have an accessible name |
a11y_labels | form fields have labels |
a11y_headings | heading levels are not skipped |
a11y_contrast | text meets WCAG AA contrast |
a11y_main | the page has a main landmark |
a11y_unique_ids | element ids are unique |
a11y_focus_order | Tab order follows the page and reaches every control |
a11y_focus_visible | focus is visible with at least 3 : 1 contrast |
a11y_keyboard_trap | Tab never gets stuck |
a11y_skip_link | a skip link or the main content within three Tab stops |
a11y_ax_names | controls have a name in the browser's accessibility tree |
a11y_landmarks | landmarks are complete and repeated ones labelled |
a11y_aria | ARIA roles and attributes are valid |
Performance, apps and design
assert | Passes when |
|---|---|
perf_lcp, perf_cls, perf_ttfb, perf_weight | largest contentful paint, layout shift, time to first byte or page weight stays within max |
app_labels | every app control has an accessible label |
app_touch_targets | touch targets are at least 48 dp (Android) or 44 pt (iOS) |
app_no_truncation | no text is cut off with an ellipsis |
app_no_crash | the app did not crash or stop responding |
design_pixels | the screen differs from the design reference in at most ratio of pixels (default 0.03) |
design_structure | headings, landmarks, actions and fields match the design reference |
uses_tokens | colours, fonts, sizes, spacing and radii come from the token set |
Packs
packs: [seo, a11y] on a step adds a fixed set of checks. They are health checks: advisory, shown apart from acceptance and never part of the step's verdict. The design pack always counts for acceptance; other packs do when the scenario lists them under health: { acceptance: [seo] }. Every check of every pack, with why it matters and its severity, is in Health checks.
On a design environment (kind: design) the steps are your own HTML served from a folder, so the packs that only code can satisfy (seo, aeo, perf, console, errors) are off there. Keep one on with health: { design_packs: [console] } in the scenario. a11y, usability and locale run on designs too.
Health checks have HC ids, acceptance criteria AC ids. The number is the check's own, so HC401 (LCP as a health check) and AC401 are the same check in its two roles: a pack in acceptance adds AC401, and adopting HC401 gives the step the criterion AC401 with from: HC401. A criterion of your own that takes a check's place replaces it and counts for acceptance, which is how you change a limit (AC401 with max: 4000) or adopt one check (kaloko health adopt <step>:<check>). Runs and scenario files from before 7 Oct 2026 call health checks AC101…; Kaloko reads those as HC101… and accepts the old ids in commands.
| Pack | Ids | What it adds |
|---|---|---|
seo | HC101–HC113, HC121–HC127 | one H1, title 10–70 and description 50–160 characters, canonical, hreflang for every language, Open Graph, not noindex, viewport meta, html lang, no overflow, no console errors, JSON-LD present (HC112) and valid (HC113); on a stack step also the checks across its pages (below) |
usability | HC201–HC204 | the purpose is clear on the first screen, one primary action, labelled fields, link texts that say what they do |
a11y | HC301–HC314 | every accessibility check above; the walk also presses Tab through the page, without clicking or typing |
perf | HC401–HC404 | LCP within 2.5 s, CLS below 0.1, TTFB within 800 ms, page weight within 3 MB (measured in the walk's browser) |
console | HC501 | the evaluator reads the console errors and says whether they touch what the step accepts |
errors | HC601–HC606 | for error pages: a real 4xx/5xx status, a link home, navigation or search, no stack trace, a plain message with a way forward, no console errors. HC601 and HC605 apply only to error pages (a step with expect_status: 404 or a page answering 4xx/5xx); elsewhere they are not applicable |
app | HC701–HC705 | labels, touch targets, no truncated text, no crash, a clear purpose and next action |
design | AC801–AC803 | the implementation against the design reference: pixels, structure, tokens |
aeo | HC901–HC905 | for answer engines (AI search, assistants): FAQPage or QAPage where the page asks and answers, an answer-first paragraph of 40–320 characters after the H1, /llms.txt at the site root, a meta description, and the page without JavaScript as GPTBot gets it (HC905, nojs_parity) |
locale | HC1001–HC1004 (HC1005 on stacks with items.variants, below) | for multilingual sites: html lang, the page's own hreflang, the language of the text and the decimal separator of prices match the walked locale. Only in scenarios that list their locales |
The a11y pack covers part of WCAG 2.2 AA. It does not replace an audit with a screen reader.
The aeo pack holds cheap signals that can be read from the page, not a ranking in any AI assistant. Hosted runs check its meta description (HC904) and show the other aeo checks, and HC112 and HC113, as "not evaluated"; kaloko walk and the scheduled GitHub workflow evaluate them all. HC905 needs a live page: kaloko recheck leaves it as the walk found it. Ids HC9xx belong to aeo, HC10xx to locale; hosted runs show the locale checks as "not evaluated" too.
Across the pages of a stack (seo pack)
A stack step with the seo pack also compares its pages with each other once they are walked. Nothing is fetched again: the checks read the title, description, H1, canonical, robots meta, hreflang and HTTP status of each capture.
| Id | Fails when |
|---|---|
| HC121 | two or more pages of one language share a title. Pages that answer an error, pages that name another canonical, and one address reached twice are left out |
| HC122 | … share a meta description |
| HC123 | … share an H1 |
| HC124 | a page names another address as its canonical (pages pointing to one address form a cluster), or a canonical points to a walked page that answers an error or says noindex. On staging or a preview whose canonicals name the production host, the canonical counts as the page itself when only the host differs |
| HC125 | a page with hreflang does not name itself, or a language version the stack walked answers an error or does not link back. Alternates outside the stack are counted, not fetched (hreflang_pairs fetches them); a missing x-default is mentioned, not failed |
| HC126 | stacks read from a sitemap (items.sitemap, items.robots): a page answers other than 200, redirects or says noindex |
| HC127 | with traffic data only (traffic in the config or the scenario): a page with at least 100 impressions has a CTR below half the site's median CTR at similar positions (1–3, 4–10, 11–20). Revise its title and description (Configuration file) |
| HC1005 | stacks with items.variants: hreflang, with or without a pack: a page of the source locale has no version of its own in a walked locale (no hreflang for it, only x-default, the hreflang points back to the page itself, the page cannot be read, or the translation answers 4xx/5xx: error 404), or several pages lead to one version. The walk reads the hreflang when it looks for the language versions |
They are health findings of the stack step, counted in pages. steps/<step>/items.json keeps them under cross_page with the groups and their addresses (at most 20 groups of 10), and every affected page lists the ids in cross_fails. A check across pages cannot be adopted as a criterion of one page. Orphan pages and click depth are not checked yet.
Site checks (health.site)
Some things belong to the whole site, not to a page. With health: { site: true } in the scenario, the walk checks each site it walks once per run (one per origin of the environment's base URLs) and writes site.json into the run:
| Id | Fails when |
|---|---|
| HC1101 | /robots.txt does not answer 200, is an HTML page, or has lines crawlers ignore (a typo such as Disalow, a rule before any User-agent) |
| HC1102 | robots.txt blocks / or the start path of a walked language for every crawler, Googlebot or Bingbot. An error on the environment named production, a warning elsewhere |
| HC1103 | robots.txt blocks GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended or CCBot from the site or a walked start path. Checked on production only |
| HC1104 | a sitemap robots.txt lists (or /sitemap.xml when it lists none) does not answer, is not a urlset or sitemap index, is cut off, has relative addresses or pages on a host other than the sitemap's or the site's (robots.txt may point to a sitemap on another host), more than 50,000 addresses, or lastmod values that are not dates or lie in the future. Sitemap indexes are followed two levels deep, at most 50 files |
| HC1105 | /llms.txt does not answer 200 with text |
| HC1106 | http:// does not redirect straight to https:// on the same host (a temporary 302 or 307 is mentioned). Not evaluated on a non-default port |
| HC1107 | the other host name (example.com for www.example.com and back) answers itself or its first redirect goes elsewhere. Subdomains have no other form and are not evaluated; a non-default port is kept |
The checks make their own requests, which is why they are off unless the scenario asks. They send the environment's gate (headers, HTTP Basic) to the site's own origin only and follow redirects themselves, read at most 10 MB of an answer and stop after two minutes per site. An answer of 401 (a sign-in gate) makes a check not applicable, not a finding. Environments that need a client certificate (mTLS) are not checked. Options instead of true:
health:
site:
paths: [/, /cs/, /eshop/] # paths robots.txt must leave open (default: / and each walked language's start path)
ai_crawlers: false # blocking AI crawlers is the site's decision: skip HC1103
sitemaps: 20 # most sitemap files to read per site (default and most 50)
environments: [production] # walks on other environments skip the site checks (default: every environment)
kaloko health shows a line per site and its findings under the pack site, as site:<host>:HC1104; the run manifest keeps the results as site. Site checks cannot be adopted as criteria.
Severity of a health finding
A failed check is an error when people or search engines are shut out: HC107 (noindex), HC108 (viewport), HC110 (overflow), HC111, HC501 and HC606 (console errors), HC301–HC303, HC305, HC310 and HC312 (names, labels, contrast, keyboard trap), HC601 and HC604 (soft 404, stack trace), HC701 and HC704 (app labels, crash), HC1102 (robots.txt blocks the walked start pages, on production). Every other failed check is a warning, and an answer the evaluator was unsure about is unsure.