Docs

Kaloko

Download
Markdown with pictures (.zip)Docusaurus folder (.zip)VitePress folder (.zip)MkDocs folder (.zip)Offline HTML (.zip)PDFChangelog as JSON
Pages

Check the health of your app

A client asks for a new checkout. The run comes back red because the order page has no meta description, which nobody asked for. Kaloko keeps such checks apart: the checks of packs (SEO, accessibility, performance, usability, console, error pages, app screens) make up the health of the app. They run on every walk, but they never decide whether a step or a run passes. Acceptance stays with the criteria from the brief.

AcceptanceHealth
The questiondoes the work do what was asked?is the app in good shape?
Where the criteria come fromthe brief, the ticket, peoplepacks, automatically
Part of the verdictyesno, advisory only
Mainly forthe person who approvesthe agent before review; the team over time

Turn it on

Health comes from the packs of a step, as before:

nodes:
  - id: home
    packs: [seo, a11y, perf, console]

The design pack is the exception: it checks the work against the design your team accepted, so it always counts for acceptance. Health checks lists every pack and what each of its checks looks at.

When the scenario is walked on a design environment, the packs that only code can satisfy (seo, aeo, perf, console, errors) are off: a designer's HTML has no real server, meta tags or error statuses. To keep one on, list it under health: { design_packs: [...] }. The scenario's page in the app shows which packs are on or off.

Read it

After a walk, kaloko walk and kaloko evaluate print one line with the counts. The details:

npx kaloko health                 # the current local run
npx kaloko health --remote        # the shared run on kaloko.app
npx kaloko health --json          # for scripts and agents

Each finding has a severity. Error means people or search engines are shut out: a page excluded from indexing, no viewport on phones, console errors, a button without a name, a keyboard trap. Warning is everything else, such as a title that is too long or a slow page. Unsure means the evaluator could not tell. The list also says what is new or fixed since the previous run of the scenario, and how to fix each check.

On the canvas, Health has its own tab. In the run panel, Health sits beside Summary: the counts, the change since the previous run and the findings by pack or by step; a click opens the step on its Health tab, beside Criteria, which holds the acceptance criteria only. Health checks have their own ids (HC103), so they never look like acceptance criteria (AC1, AC2…). Run lists show the counts as a small badge, and the project's Health page draws the trend over time.

--fail-on error makes kaloko health exit with 1 when there are errors. Use it only where the team wants health to stop a pipeline; nothing else in Kaloko fails because of health.

Make one check count for acceptance

Sometimes a health check is part of the brief after all ("every product page has a meta description of at most 155 characters"). Adopt it, as it is or in your own words:

npx kaloko health adopt product:HC103 --text "Every product page has a meta description of at most 155 characters"

The check is written into the scenario file as the criterion AC103 of that step, with from: HC103, and counts for its verdict from the next walk. The step's health no longer lists HC103. Older runs that still call the check AC103 accept that id too. On the canvas, Adopt as criterion on a finding does the same for people with the Developer / Tester role; the agent writes it into the scenario with kaloko health pull.

To keep a whole pack in acceptance, as it was before health existed, name it in the scenario:

health:
  acceptance: [seo]

Hear when it gets worse

Hosted runs and runs shared from CI compare their health with the previous run of the scenario. When there is a new error, Kaloko tells the scenario's followers in the inbox and by mail, posts to Slack and Teams channels that take "Health got worse", and sends the webhook event health.worse. Each schedule can alert on new errors (the default), on new warnings too, or not at all: Settings → Integrations → Hosted runs.

On this pageOn this page