Run Kaloko in CI and comment on pull requests
In CI, Kaloko runs the same commands as on your laptop. The usual setup walks what a pull request changed on its preview deployment (Vercel, Netlify, Cloudflare Pages or any review app) and puts the canvas link into a comment on the pull request.
1. Write the workflow
npx kaloko ci github # writes .github/workflows/kaloko.yml
npx kaloko ci gitlab # writes kaloko.gitlab-ci.yml; include it from .gitlab-ci.yml
npx kaloko ci github --print # show it, write nothing
--scenario a.yml,b.yml limits it to some scenarios. The command also adds a preview environment to kaloko.config.yml with base_url: ${KALOKO_PREVIEW_URL}. A scenario that lists environments: has to include preview, or kaloko start refuses it; the command tells you which ones.
2. Add the token
Create a token with the tester role in Settings → API tokens and add it to the repository's CI secrets as KALOKO_TOKEN. On GitLab, add KALOKO_GITLAB_TOKEN too: a project access token with the api scope, so Kaloko can write the note on the merge request. Commit the workflow.
What the workflow does
On every successful preview deployment of a pull request:
- It installs the project and the browser (
npx kaloko browser install --with-deps), plus WebKit and Firefox when a scenario asks for them. npx kaloko ci preview-url --wait 600 --github-envfinds the preview address, the pull request and its base branch and passes them on asKALOKO_PREVIEW_URL,KALOKO_PRandKALOKO_BASE.- For each scenario,
kaloko affected --fail-if-noneskips it when the change does not reach any of its steps. Otherwise it runs:
npx kaloko start --scenario qa/flows/checkout.yml --env preview
npx kaloko walk --ephemeral --affected
npx kaloko share --pr
share --pr adds one comment per scenario to the pull request and updates it on later pushes. Uploads go by content, so screenshots that did not change are not sent again.
The Kaloko check on the pull request
With the Kaloko GitHub App (an admin installs it in Settings → Integrations → GitHub, Team and up), every pull request in a connected repository gets one check named Kaloko on its newest commit. It turns green when the runs shared for that commit pass, or when the change reaches no scenario. It waits while a scenario the change reaches has no run yet, and turns red when one fails. The runs can come from this workflow, from a coding agent in the cloud or from a laptop: what counts is the commit. A run of uncommitted changes does not count.
To block merging until it is green, require the status check Kaloko in the repository's branch protection. Kaloko never changes branch protection itself.
- Modes (Settings, per repository): off, manual (only after the label
kaloko:runor/kaloko walk), automatic (the default) and strict (every scenario, a review required, a time limit). - One pull request: the labels
kaloko:skip,kaloko:all,kaloko:review, the comment/kaloko skip <reason>(this commit only) or the buttons on the check. Only people with the access your admin chose may skip, never apps or bots, and the check names who skipped it and why. - In the repository: the
github:section ofkaloko.config.ymlon the base branch can make the check stricter, never looser. npx kaloko ci plantells the check exactly which scenarios a commit reaches. The generated workflow sends it before walking.- No check appears? Settings → Integrations → GitHub lists the repositories and says why, for example a repository that is not in the app's installation yet.
Good to know
walk --affectedneeds the git history: check out withfetch-depth: 0on GitHub orGIT_DEPTH: "0"on GitLab.--trigger pull_requestruns on every push instead of the deployment event and waits for the preview.kaloko ci preview-urlprints only the address, so it works in any CI:export KALOKO_PREVIEW_URL="$(npx kaloko ci preview-url)".- A preview behind Vercel's deployment protection needs a
VERCEL_AUTOMATION_BYPASS_SECRETsecret and theaccess.headersblock of the preview environment, which the generated config has commented out. - Runs shared from CI also compare their health (the advisory checks of packs) with the previous run of the scenario and alert when there is a new error.
npx kaloko health --fail-on errorstops the job on health errors, if the team wants that. - No comment on the pull request? Usually there was no preview deployment, or the token is missing. The job log says which.
When release-please, changesets or semantic-release publishes your releases, npx kaloko ci github --release adds a second workflow that records each release on Kaloko. All options are in the CLI reference.
Kaloko · latest · 2026-11-21