Kaloko help

How to set up Kaloko, walk your app with an agent, review what it built and keep a changelog and docs from the result. The pictures come from Kaloko's own accepted runs, so they show the screens as they are today; these pages are themselves living docs published with Kaloko.

Tutorials #

• Your first run in ten minutes: install, sign in, walk a flow, share it and review it
• Design a flow with your agent: live HTML steps, versions and the design reference

Review and approve #

• Review a run: go through the screens of a run on the canvas
• Approve or return a run: say yes to the steps that are right and send back the rest
• Review only what changed: review mode, its keys and approvals carried over
• Read what's new: the changelog with before and after pictures
• Hand over a signed acceptance record: proof of who accepted what, and how to check it

Walk and check #

• Write a scenario: the flow, its steps and what each step must show
• Add criteria and check packs: automatic checks, questions for the evaluator, packs
• Walk in more browsers and on devices: WebKit, Firefox, phones, tablets, dark mode
• Walk Android, iOS and desktop apps: emulators, simulators, Electron, macOS and Windows
• Record a walkthrough video: recordings and traces of each step, one video of the run
• Run Kaloko in CI and comment on pull requests: walk each preview deployment
• Compare with a baseline and ignore what always changes: diffs without clocks and carousels
• Check pages on a schedule (hosted runs): Kaloko's browsers open your pages every day or week

Team and access #

• Invite people and guests: members, guests on single projects, readers from your domain
• Roles and API tokens: who may do what, tokens for agents and CI
• Set up company sign-in: sign-in domains, two-step verification, SAML, OIDC and SCIM
• Connect Jira, Linear, Slack and Teams: issues from steps and channels per project

Changelog and docs #

• Publish a changelog with pictures: draft the entry, record the release, share it
• Publish living docs and catch stale sections: guides whose pictures keep up with the app
• Export docs and the changelog: Markdown, Docusaurus, VitePress, MkDocs, HTML, PDF or JSON

Reference #

• CLI commands: every command of kaloko
• Scenario file: every key of a scenario
• Criteria, checks and packs: every check type and what each pack adds
• Configuration file: kaloko.config.yml
• MCP tools: what agents can do over MCP

Explanation #

• How Kaloko maps to your applications: organizations, products, projects, modules, environments and versions
• How verdicts are decided: checks, the evaluator and people
• Security and your data: what leaves your machine and who sees it

Your first run in ten minutes

By the end of this tutorial one flow of your app is on the canvas at kaloko.app, shared, with every screen in every language and screen size your scenario names. You need Node 20 or newer, Chrome, and an app running locally.

Install the CLI #

In the root of your project:

npm install --save-dev kaloko
npx kaloko init --agent claude --org <your-org>

Use --agent codex for Codex, --agent all for both, or leave it out to drive the CLI yourself. init writes kaloko.config.yml and copies the skill to your agent. It also reads the project: the framework, the dev server's command and URL, the locales and the main routes. From those it drafts a first scenario, qa/flows/main-pages.yml, so the first walk works without editing.

If you have no organization yet, create one first; you become its admin.

Sign this machine in #

npx kaloko login
npx kaloko doctor

login opens the browser, you approve the code and choose what the token may do. It is saved to ~/.config/kaloko/.env. doctor checks Node, Chrome, the config, the environments and the token, and says what to fix next to every ✕.

Walk the flow #

Start your dev server, then:

npx kaloko start --scenario qa/flows/main-pages.yml --env local
npx kaloko walk
npx kaloko preview

start creates a run, walk opens every step in the browser and captures it, and runs the checks as it goes. It ends with the verdicts: what failed and what a person should look at. preview prints the path of the local canvas, which opens from disk.

With the skill installed you can skip the typing and ask the agent instead: "Walk the main pages with Kaloko on local and show me what fails." It runs the same commands.

Share it #

npx kaloko share

The run goes to kaloko.app and the command prints its link. On a branch with a pull request, share --pr also puts the link in a comment on the pull request.

Look at it and decide #

Open the link. The run is a map of its steps; click one to see the screenshot, the criteria and what was found.

Approve the steps that are right, return the rest with a note, and accept the run when you are happy with it. Your agent reads the notes with kaloko feedback.

Where next #

• Write a scenario for the flow you care about most.
• Review a run shows the canvas in more detail.
• Run it in CI on every pull request.
• The CLI reference lists every command.

Design a flow with your agent

In this tutorial you design the screens of one flow as live HTML with your own agent, let the team comment on them, publish a version and make it the design that every implementation is checked against. You need a project where kaloko init has run and a scenario for the flow, for example qa/flows/checkout.yml.

Add a design environment #

A design environment is a folder the CLI serves while it walks. Add it to kaloko.config.yml:

environments:
  design: { kind: design, serve: design }

The scenario stays the one development and QA use: same step ids, same criteria. A design is checked against the acceptance criteria too, before anyone builds it.

Let the agent draw the steps #

Ask your agent, with whatever design skill your team uses:

Design the checkout flow into design/ with our brand skill: one folder per step of qa/flows/checkout.yml, shared CSS in design/_shared, tokens in design/tokens.json. Then run kaloko walk and kaloko sync.

Each step becomes design/<step>/index.html: plain HTML, CSS and JS that open from disk. Links between steps are relative links such as ../payment/index.html.

Walk, lint and sync #

npx kaloko start --scenario qa/flows/checkout.yml --env design
npx kaloko design lint
npx kaloko walk
npx kaloko sync

design lint checks the folder against the portable HTML rules: relative URLs, no CDNs, no storage, predictable content. walk renders only the steps that changed, and sync saves them into the shared draft and takes the steps others changed.

Comment together #

The team opens the draft on the canvas, clicks through the live steps and pins comments to elements. Everyone sees who else is looking. Your agent reads the comments with kaloko feedback, changes what they ask for and syncs again.

Publish a version #

npx kaloko share --name "Checkout v2"

This publishes a version of the whole flow, such as "v2 · 3 changed, 17 unchanged". Unchanged steps are stored once. share --notes-preview shows the notes on what changed before you publish.

Make it the reference #

Once a person has accepted the version, make it the design reference (Team plan and up):

npx kaloko reference --design --version 2

Turn on packs: [design] for the implementation steps of the scenario. On Business and up, every implementation run is then checked against the reference: pixels, structure, tokens and contrast, and kaloko drift names the token to use where the code differs.

Coming from Figma #

If the design already lives in Figma, kaloko design import figma <file-url> --scenario qa/flows/checkout.yml turns its frames into design steps and its variables into tokens. A folder of frames exported from Figma works without a token.

Review a run

A run is one pass of an agent through a flow: every screen it reached, in every language and screen size it walked, with the checks it made. You review it in the browser; there is nothing to install.

Open the run #

1. Open the link from the pull request, the issue or the e-mail. The run opens on the canvas: every step of the flow as a map, with arrows for the way between them.

   

2. If you would rather work through a list, press I. The same run becomes a list of steps with their results, and you can search it and filter by status.

   

Look at a step #

1. Click a step. Its detail shows the screenshot, the criteria with what was found, and the comments people left on it. ← and → move to the previous and next step.

   

2. Something looks wrong? Write a comment on the step. All open comments of the run sit in the comments panel, so nothing gets lost between steps.

   

Walk it like a user #

Press R to replay the run: one screen at a time, in the order a person would see them. It is the quickest way to show the flow to someone who has never seen the canvas.

When you know what you think, approve or return the run.

Approve or return a run

Your decision is what turns "the agent says it is done" into "it is done". You decide step by step, then for the whole run.

Decide each step #

1. Open a step and read its criteria. A green criterion was checked by Kaloko; one that needs a person says so, and that judgement is yours.

   

2. Under the comments, choose Approve when the step is right. When it is not, write what should change and choose Reject: "the total jumps when the price loads", not "fix this". The agent reads the note and works from it.

Decide the run #

1. When every step is approved, choose Accept run. The run becomes the picture of this version: the changelog and the docs show its screens from now on.
2. If something has to change first, choose Return run. Whoever shared it gets your notes; when they share the next run, the comment in the pull request points to it.

   

Next time is shorter #

In the next run, steps whose screens did not change keep your approval. You only look at what is new or different, and the canvas says which approvals were carried over and from which run.

Agents can comment and suggest, but only a person accepts a run.

Read what's new

Every release of a product has an entry in its changelog, written for the people who use it and illustrated with the screens it changed. The pictures come from runs someone accepted, so they show what was approved.

Find it #

1. Open Products in the main menu and choose Changelog at the product, or open the link someone sent you. Each product has its own changelog; its newest release is on top.
2. The versions at the top jump to their release. Everything below a version is what changed in it, grouped as Added, Changed, Fixed and Removed.

Compare before and after #

1. A change to a screen shows the screen before and after on top of each other. Drag the slider to move the split.
2. Side by side puts the two pictures next to each other (on a phone, one above the other).
3. Language and Screen switch every picture to another language or screen size, when the run captured them.
4. Open the flow opens the whole flow on the canvas, if you may see the run.

Keep up #

• E-mail me what's new sends you the entry of every new release, with its pictures, in your language. The link at the bottom of each mail stops them.
• Download gives you the changelog as JSON, Markdown, offline HTML or PDF, for release notes or a client report.
• A public changelog also has a feed for feed readers.

Review only what changed

Review mode shows one step at a time, starting with the steps that need a person: failed and returned steps, criteria the AI was unsure about, steps that changed since the last accepted run and steps with open comments. Steps that did not change keep the approval someone gave them before.

Open review mode #

1. Open the review link from the pull request or the e-mail (it ends in #review), or open the run and press Q.

   

2. The step you are deciding sits next to the accepted run, so you see what moved.

Decide with the keyboard #

When every step is decided, accept or return the whole run.

Look closer #

In the large view you can compare the two runs side by side, as a difference, with Swipe (drag a divider across both screenshots) or with Onion skin (this run faded over the other). To point at a problem, draw a box, an arrow or a line on the screenshot while you write the comment, or paste a picture.

Approvals carried over #

A carried approval names who gave it and in which run. If you think the step needs another look, revoke it and the step waits again. A carried approval never accepts the run for you.

Hand over a signed acceptance record

When a client or an auditor asks who accepted a release, give them the acceptance record of the run. It lists who approved what and when, the results of the criteria and a fingerprint of every screenshot, and Kaloko signs it. Records are part of the Business and Enterprise plans.

Download it #

1. Open the accepted run and its Acceptance record panel.

   

2. Choose Download PDF for people, or JSON for a system that keeps records. The PDF carries the same JSON inside it.

A run that is not accepted has no record. Accept it first, or ask the person who has to.

Check one you received #

Anyone can check a record without an account. Developers run kaloko signoff --verify record.pdf; it checks the signature against Kaloko's public key and says whether anything was changed. If a single character of the record was edited, the check fails.

Connect Jira, Linear, Slack and Teams

For organization admins on the Team plan and up. Once connected, people file a returned step as an issue straight from the canvas, and each project tells its own chat channel what happened.

Jira or Linear #

1. Open Settings → Integrations and connect Jira Cloud (sign in with Atlassian, or paste an API token) or Linear (sign in, or paste an API key).
2. Map each Kaloko project to a Jira project and issue type, or to a Linear team. A default row covers projects you do not list.
3. For Jira, add the webhook Kaloko shows to your Jira site, so a finished issue closes its comment in Kaloko at once. Without it, Kaloko checks every 30 minutes.

People then choose New issue on a step. The issue gets the screenshot, the criterion and the reviewers' notes, with a link back.

Slack or Teams channels #

1. In Settings → Integrations, add a channel for a project: Add to Slack for Slack, or the address of a Teams Workflows flow for Microsoft Teams.
2. Tick the events the channel wants: new run, returned, accepted, comment, mention, release recorded.
3. Set quiet hours if the channel should not hear from Kaloko at night; messages wait until they end.

A restricted project only reaches channels that name it. The organization-wide Slack channel keeps working.

Write a scenario

A scenario is the flow Kaloko walks: a YAML file with the screens, the way between them and what each screen has to meet. Task flows usually sit next to the task (docs/tasks/TASK-123/qa/scenario.yml), long-lived flows such as the checkout in qa/flows/.

Let the agent write it #

With the skill installed, ask your agent instead of typing YAML:

Read the task in docs/tasks/TASK-123, write its acceptance plan and a Kaloko scenario, and validate it.

The agent writes two files: ACCEPTANCE.md for people (goal, steps, criteria AC1, AC2…) and the scenario for the runner. When the intent changes, it updates the plan first and then the scenario.

The parts of the file #

schema: qawalk.scenario.v1
id: TASK-123-checkout
title: Checkout
kind: task                # task | flow
product: shop             # optional, with module:
locales: [en, cs]
viewports:
  desktop: [1280, 800]
  mobile: [390, 844]
environments: [local, staging]
nodes:
  - id: cart
    kind: screen
    title: Cart
    path: /cart
    purpose: The visitor sees what they are buying and can go on to pay.
    execution:
      strategy: playwright   # playwright | agent | manual
      script: steps/cart.mjs
    criteria:
      - id: AC1
        text: The page has exactly one main heading.
        check: { type: deterministic, assert: count, selector: h1, min: 1, max: 1 }
edges:
  - { from: cart, to: payment, label: continue }

• Step ids (cart) stay stable: links, comments, approvals and docs pictures refer to them.
• purpose is one sentence about what the screen is for. The evaluator reads it.
• execution says who moves to the screen: a Playwright script, an agent that captures with kaloko capture, or a person.
• Other node kinds: decision for a branch, email for a message from the mailbox, stack for many pages of one template.

Every key is in the scenario file reference. To add checks, see Add criteria and check packs.

Check it #

npx kaloko validate qa/flows/checkout.yml
npx kaloko validate qa/flows/checkout.yml --coverage

validate checks the schema and the references between steps. --coverage compares the criteria with the acceptance plan and lists what the plan asks for and the scenario does not check.

Start from public pages #

For a marketing site or documentation, start from a draft:

npx kaloko draft https://example.com/ https://example.com/pricing

Kaloko fetches the pages and writes a read-only production scenario and its plan, with the page title, H1 and description as the purpose and the seo, a11y, perf and console packs. It also adds a step for an address that does not exist, so the 404 page is checked too. Then sharpen each purpose and add what the page really has to meet.

Add criteria and check packs

Each step lists its criteria: what the screen has to meet. Kaloko knows three kinds, and a good scenario uses the cheapest one that can answer the question.

Countable: a deterministic check #

Anything you can count or read from the page is checked in the page itself, with the evidence attached:

criteria:
  - id: AC1
    text: The cart shows two items.
    check: { type: deterministic, assert: count, selector: ".cart-item", min: 2, max: 2 }
  - id: AC2
    text: Paying ends on the thank-you page.
    check: { type: deterministic, assert: url_matches, pattern: "/thanks$" }
  - id: AC3
    text: The Pay button uses the brand colour.
    check: { type: deterministic, assert: style_equals, selector: "button.pay", property: background-color, token: color.brand.primary }

Other checks read text, attributes, the HTTP status, overflow on a phone or a value the page exposes (js_equals). The full list is in checks and packs. A step script never has to leave a marker for a criterion to find.

A question: a semantic check #

What needs judgement goes to the evaluator as a question about the page:

  - id: AC4
    text: The primary call to action is clear.
    check: { type: semantic, question: Does the page show one clear primary call to action?, scope: main }

The evaluator reads a reduced outline of the page and answers with a score. A clear yes passes, a clear no fails, and anything in between goes to a person. Add vision: true to the check when the look is the point (layout, imagery, "looks like the brand"): the screenshot then gets a second look from a vision model. Keep it to the steps that need it.

A person decides: a manual check #

  - id: AC5
    text: Legal approved the wording of the consent.
    check: { type: manual }

A manual criterion stays "not evaluated" until a reviewer decides it on the canvas.

Check packs #

A pack adds a fixed set of criteria to a step, so common checks are not written by hand:

nodes:
  - id: home
    packs: [seo, a11y, perf, console]

To change a limit, write your own criterion with the same id as the pack's (for example AC401 for LCP); yours wins. How verdicts come together is explained in how verdicts are decided.

Walk in more browsers and on devices

By default a run captures every step in Chromium, in each locale and viewport of the scenario. You can add browser engines, device presets, dark mode and a few more settings. Every combination becomes its own capture, so add only what the task is about: a Safari bug needs WebKit on the affected steps, not every setting on every step.

In the scenario #

viewports: [iphone-se, iphone-15, pixel-8, ipad-landscape, laptop]
browsers: [chromium, webkit, firefox]
color_schemes: [light, dark]
reduced_motion: [no-preference, reduce]   # optional
forced_colors: [none, active]             # optional, Windows high contrast
display_modes: [browser, standalone]      # optional, the page as an installed web app

Defaults for the whole project go under walk: in kaloko.config.yml. The command line wins over the scenario, and the scenario over the config.

For one run #

npx kaloko start --scenario qa/flows/checkout.yml --env staging \
  --browsers chromium,webkit --color-schemes light,dark --viewports iphone-se,pixel-8,desktop-hd
npx kaloko walk                      # one pass per engine and device
npx kaloko walk --browsers webkit    # only the WebKit captures of the run

Device presets #

npx kaloko devices lists the presets: phones (iphone-se, iphone-15, pixel-8, galaxy-s24…), foldables (fold-folded, fold-open), tablets (ipad-portrait, ipad-landscape…), computers (laptop, desktop-hd, desktop-wide) and tv-1080p. Add -landscape to turn a phone, foldable or tablet sideways (iphone-15-landscape). A preset sets the size, pixel ratio, touch and the device's user agent. It catches layout bugs; a real device catches the rest.

Install the engines #

WebKit (Safari's engine) and Firefox are downloaded once per machine or CI job:

npx kaloko browser install --browsers webkit,firefox

npx kaloko doctor lists the engines your scenarios need and says how to install a missing one. On Linux CI add --with-deps for the system libraries.

What to expect #

• A few checks run only in Chromium: names from the accessibility tree, visible focus and layout shifts, and on WebKit also the Tab walk. On other engines they say "not evaluated" with the reason and do not make the step partial.
• Contrast is checked per colour scheme, so a dark-mode contrast bug fails on the dark capture.
• A run refuses more than 400 combinations.

On the canvas the browser, colour scheme and other settings sit next to the language switcher. The step detail of a device preset draws the device frame, and Beside: shows the same step in another browser or scheme next to it.

Walk Android, iOS and desktop apps

Kaloko walks native apps the way it walks web pages: every step becomes a screenshot with its UI tree, checked criteria and a place on the canvas. The scenario says which platform it is, and the config says which app to open.

1. Name the app per environment #

# kaloko.config.yml
environments:
  staging:
    app:
      android: { package: com.acme.shop.staging, apk: build/app-staging.apk, reset: true }
      ios: { bundle: com.acme.shop, app: build/Shop.app, device: iPhone 17 }
  desktop:
    app:
      electron: { main: dist/main.js }
      # desktop: { name: Notes }                         # a macOS app by its window
      # windows: { name: contoso, path: dist/Contoso.exe }

2. Set the platform in the scenario #

platform: android        # android | ios | electron | desktop | windows
devices: [pixel-8-api-34, pixel-7-api-33]
color_schemes: [light, dark]
nodes:
  - id: product
    kind: screen
    path: shop://product/42     # a deep link
    packs: [app]

Steps are reached by deep links or by a step script (app.tap(…), app.type(…)). Every locale starts the app fresh in that language, and reset: true clears its data first.

What each platform can do #

The app pack checks labels on controls, touch target sizes, cut-off text and crashes.

Screens from other tools #

Maestro, Appium, XCUITest, Detox or a person can take the screenshot, and Kaloko lays it out with its UI dump:

npx kaloko capture --step checkout --image shot.png --tree hierarchy.xml --device "Pixel 8" --density 2.625

--tree reads uiautomator or Appium XML, idb ui describe-all --json and Maestro's hierarchy JSON. Without a tree, --ocr reads the text from the image on macOS. A whole folder of screenshots or a test report goes in with npx kaloko import.

Check the machine first #

npx kaloko doctor

It says which app tools are missing on this machine (adb, simctl, idb, ffmpeg) and how to install them. On macOS, kaloko doctor --fix asks for the Screen Recording and Accessibility permissions desktop captures need.

Record a walkthrough video

A screenshot shows where a step ended. A recording shows how it got there: the spinner that never stopped, the menu that opened twice. Record when the screenshot cannot show what a reviewer has to judge: animations, loading states, drag and drop, a flaky step, or a demo for a client.

Record a walk #

npx kaloko walk --record           # each step, from the previous screen to its capture
npx kaloko walk --record flow      # the whole walk per language, a chapter per step
npx kaloko walk --record failed    # only steps whose script or checks fail
npx kaloko walk --record --record-quality low --steps checkout

To record by default, set it in the scenario or in kaloko.config.yml:

capture:
  record: { mode: failed, quality: low, max_seconds: 45 }
nodes:
  - id: checkout
    record: true        # this step always; record: false never

Quality is low, medium (the default) or high. Recordings are much larger than screenshots, so do not record every walk.

Traces come with every web step #

Each walked web step keeps a trace: clicks and typing, network requests with status and timing, and console messages. It is small and on by default; --no-trace turns it off.

npx kaloko video list                    # what was recorded and traced, with sizes
npx kaloko video trace --step checkout   # the step's trace as a timeline in the terminal

On the canvas the step detail shows Video next to the screenshot, and the Technical tab shows the trace. Click a row and the video jumps there. Replay plays the flow recording chapter by chapter.

Make one video of the run #

npx kaloko video export --format mp4     # or webm, gif
npx kaloko video export --locale cs --steps cart,payment --lang cs --out checkout.mp4

The export puts the recordings in flow order with a title card before each step (its title and verdict). It is the file to send to a client or attach to a release note.

Tools you need #

Web and Electron recordings need nothing but Chrome. With ffmpeg installed you also get MP4 for Safari, app recordings (Android, the iOS Simulator, macOS) and the walkthrough export. npx kaloko video tools says what this machine has. Walks in WebKit and Firefox are neither recorded nor traced.

Sharing #

Recording works on every plan. kaloko share uploads recordings with the run on Business and Enterprise; on Free and Team they stay in the local preview, and the screenshots and traces are shared. Shared recordings are kept for 30 days (Enterprise sets its own period in Settings → Storage). A recording over 25 MB stays local: lower the quality or max_seconds.

Run Kaloko in CI and comment on pull requests

In CI, Kaloko runs the same commands as on your laptop. The usual setup walks what a pull request changed on its preview deployment (Vercel, Netlify, Cloudflare Pages or any review app) and puts the canvas link into a comment on the pull request.

1. Write the workflow #

npx kaloko ci github     # writes .github/workflows/kaloko.yml
npx kaloko ci gitlab     # writes kaloko.gitlab-ci.yml; include it from .gitlab-ci.yml
npx kaloko ci github --print   # show it, write nothing

--scenario a.yml,b.yml limits it to some scenarios. The command also adds a preview environment to kaloko.config.yml with base_url: ${KALOKO_PREVIEW_URL}. A scenario that lists environments: has to include preview, or kaloko start refuses it; the command tells you which ones.

2. Add the token #

Create a token with the tester role in Settings → API tokens and add it to the repository's CI secrets as KALOKO_TOKEN. On GitLab, add KALOKO_GITLAB_TOKEN too: a project access token with the api scope, so Kaloko can write the note on the merge request. Commit the workflow.

What the workflow does #

On every successful preview deployment of a pull request:

1. It installs the project and the browser (npx kaloko browser install --with-deps), plus WebKit and Firefox when a scenario asks for them.
2. npx kaloko ci preview-url --wait 600 --github-env finds the preview address, the pull request and its base branch and passes them on as KALOKO_PREVIEW_URL, KALOKO_PR and KALOKO_BASE.
3. For each scenario, kaloko affected --check skips it when the change does not reach any of its steps. Otherwise it runs:

npx kaloko start --scenario qa/flows/checkout.yml --env preview
npx kaloko walk --ephemeral --affected
npx kaloko share --pr

share --pr adds one comment per scenario to the pull request and updates it on later pushes. Uploads go by content, so screenshots that did not change are not sent again.

Good to know #

• walk --affected needs the git history: check out with fetch-depth: 0 on GitHub or GIT_DEPTH: "0" on GitLab.
• --trigger pull_request runs on every push instead of the deployment event and waits for the preview.
• kaloko ci preview-url prints only the address, so it works in any CI: export KALOKO_PREVIEW_URL="$(npx kaloko ci preview-url)".
• A preview behind Vercel's deployment protection needs a VERCEL_AUTOMATION_BYPASS_SECRET secret and the access.headers block of the preview environment, which the generated config has commented out.
• No comment on the pull request? Usually there was no preview deployment, or the token is missing. The job log says which.

When release-please, changesets or semantic-release publishes your releases, npx kaloko ci github --release adds a second workflow that records each release on Kaloko. All options are in the CLI reference.

Compare with a baseline and ignore what always changes

Before a release you want one answer: what looks different from the version the team accepted, and is it on purpose. A good run of a scenario becomes its baseline, and every later run is compared with it.

Set the baseline #

On the canvas, open a run you are happy with, choose Accept run and then Set as baseline. Each scenario has one baseline, and it moves only when someone sets a new one. From the terminal:

npx kaloko reference --scenario checkout --baseline --run <run-id>
npx kaloko reference --scenario checkout                     # show the current one

Compare a run #

On the canvas, choose Compare and the baseline (or the previous run, or any other run of the scenario). Each step gets a badge with the share of pixels that changed, and criteria whose verdict or evidence changed are marked.

In the step detail you can look at the change four ways: Side by side, Diff, Swipe (drag a line across the two screenshots) and Onion skin (fade one into the other). G shows one step in every language and screen size at once.

The same comparison as text, for an agent or the release notes:

npx kaloko compare                       # the current run against the baseline, else the previous run
npx kaloko compare --run <id> --baseline <id> --json

Ignore what always changes #

Clocks, ads, carousels and "3 minutes ago" change on every run and drown the real differences. Leave them out of the pixel comparison with ignore: on the step:

nodes:
  - id: home
    ignore:
      - { selector: ".ticker", note: live prices }
      - { rect: [0, 620, 1280, 180], viewport: desktop }

You do not have to find them by hand:

npx kaloko walk --stability 3      # three screenshots of each capture a moment apart
npx kaloko stability               # what moved, compared with earlier runs of the scenario
npx kaloko stability --apply       # writes the suggestions into ignore:

Look at the screenshots before you apply them: a region that changed because the page really changed is not noise.

Reviewers can mark noise too. In the step detail on kaloko.app, Mark a changing region and draw a rectangle on the screenshot. It shows dashed on every run of the flow, and the canvas diff skips it at once. To put those regions into the scenario, run:

npx kaloko stability --pull

Walk the affected steps again afterwards: a capture records the regions it ignored.

Check pages on a schedule (hosted runs)

The marketing site changes several times a week, and none of it goes through your pull requests. A hosted run opens the pages of a read-only scenario on Kaloko's own browsers every day or week, takes full-page screenshots in every language and screen size, checks what can be read from the page and shares the run under your name. Nobody has to start the CLI. Hosted runs are part of the Business plan (2,000 loaded pages a month) and Enterprise (10,000).

1. Prepare a read-only scenario #

The scenario's steps open pages. To start from the pages themselves:

npx kaloko draft https://example.com/ https://example.com/pricing

2. Schedule it #

npx kaloko schedule add --scenario qa/flows/public.yml --env production --every day --at 5
npx kaloko schedule add --scenario qa/flows/public.yml --env production --every week --weekday mon

--at is the hour in UTC. --every manual creates a schedule that runs only when you start it.

Pages behind a gate work too. HTTP Basic credentials of the environment are sent once and kept encrypted. For signed-in pages, a person saves a session with npx kaloko auth save --env production --account member, and the schedule uses it with --account member.

3. Manage it #

npx kaloko schedule list            # next and last run, pages used this month
npx kaloko schedule run <id>        # run it now
npx kaloko schedule pause <id>
npx kaloko schedule resume <id>
npx kaloko schedule remove <id>

Settings → Integrations on kaloko.app shows the same schedules.

Read the result #

The run appears among the others, compared with the scenario's baseline, with the same canvas, comments and approvals. Checks of selectors, texts, titles and meta tags are decided. Criteria that need the full walk say "not evaluated" and give the reason.

What hosted runs cannot do #

They only open pages and read them. Nothing is clicked, typed, submitted or created, so steps that do more than open a page are left out and named. Native apps, local or private addresses, credentials sent as headers and base URLs per language also need the full walk.

For those, export the schedule as a GitHub Actions workflow that walks the scenario with the full CLI:

npx kaloko schedule export github --scenario qa/flows/checkout.yml --env staging --every week

The workflow names the secrets it needs.

How pages are counted #

Every loaded page counts: each step in each language and screen size. A scenario of 10 pages in 2 languages on desktop and mobile loads 40 pages per run, so a daily run uses about 1,200 pages a month.

Invite people and guests

A run is only useful once the people who decide have seen it. Bring colleagues into the organization, invite a client to a single project, or let everyone from your company domain read along. Viewers are free on every plan, so nobody has to ration who may look.

Invite a colleague to the organization #

1. Open Settings → People and choose Invite to the organization.

   

2. Enter the e-mail address and pick a role. A Viewer reads, comments and approves; the other roles are explained in Roles and API tokens.
3. The person gets an e-mail with a link and joins the organization with it.

The list in Settings shows every member with their role and when they were last seen, and pending invitations with who sent them and when they expire.

Invite someone to one project #

Clients, freelancers and colleagues from other teams often need one project, not the whole organization.

1. Open the project and choose Share.
2. Type the e-mail address, also one outside your organization, and pick a role.
3. Choose Invite. The invitation is valid for 14 days.

An address outside your organization joins as a guest. Guests see only the projects they were invited to: not the other projects, not the people of the organization, not settings or billing. The Share dialog also shows who already has access and whether the project is open to every member or restricted to the people listed.

On Business you can set a date when a guest's access ends; on that day they leave the project and get an e-mail. Guests with the Designer or Developer / Tester role are part of Enterprise; on other plans a guest is a Viewer.

Let your company domain sign in #

Instead of inviting colleagues one by one, add your company domain under Settings → People → Sign-in domains. Prove you control it with the TXT record Kaloko shows; until the domain is verified, nobody joins through it. Anyone with an e-mail on a verified domain can then sign in with the domain's default role (Viewer unless you choose another). Public mailbox domains such as gmail.com cannot be claimed.

Readers from your domain #

On Business, the Readers from our domain switch in the same place lets everyone who signs in from a verified domain read the changelog and docs of every product, also where the project is restricted. They join as Viewers and take no seat. See Publish a changelog with pictures.

Who takes a seat #

Viewers never do. Designers, developers / testers and admins take one seat each, once for the whole organization, members and guests alike. An invitation with such a role holds a seat until it is accepted or revoked.

Roles and API tokens

Every person in an organization has a role, and every token acts for a person with no more rights than that role. Agents and CI use tokens; they never take a seat.

Roles #

A project can have its own project admin, who invites people to that project, changes their roles and can restrict or open it. Developers / testers and admins can create projects; whoever creates one becomes its admin. Change a member's role in Settings → People.

Create a token in the app #

1. Open Settings → API tokens.

   

2. Name the token after where it lives, for example "laptop CLI" or "GitHub Actions".
3. Tick what it may do. Reading is always included.
   • comment and approve: comments and verdicts, as you
   • design: kaloko sync of design drafts
   • upload runs: kaloko share from the CLI or CI
   • everything my role allows
4. Pick how long it lives: 30 days, 90 days or a year.
5. Optionally limit it to chosen projects; it then sees only those and cannot create new ones.
6. Copy the token once and put it into the project's .env as KALOKO_TOKEN.

The list shows each token's owner, role, prefix, last use and expiry. Revoke ends a token at once.

Sign in from the terminal #

On your own machine there is a shorter way:

npx kaloko login

The browser opens; you approve the code, choose the organization and what the token may do. The token is valid for at most 30 days and is saved to ~/.config/kaloko/.env. --print prints it instead, --no-open shows the link without opening a browser.

Tokens for CI #

Create a token with the upload runs scope, ideally limited to the project, and store it as a KALOKO_TOKEN secret in your CI. npx kaloko ci github and ci gitlab write workflows that read it; see the CLI reference.

Rules an admin can set #

Under Settings → Security an admin can allow only admins to create tokens with every right, turn off tokens limited to projects, and decide whether guests may create tokens at all. A guest's token sees only their projects. On Enterprise, service accounts hold tokens for automation that belongs to no person.

Set up company sign-in

For organization admins. Kaloko works with the sign-in your company already uses: personal sign-in on every plan, an organization-wide second factor from Business, and your identity provider with SCIM on Enterprise.

Sign-in on every plan #

People sign in with a one-time link sent by e-mail, or with Google, Microsoft or GitHub. Each person can add a passkey or an authenticator app as a second step in their account settings.

On Business an admin can require the second step for the whole organization and set limits for sessions, under Settings → Security. People without a second step are asked to add one before they continue.

Verify your domain first #

Company sign-in and SCIM act only on addresses from verified domains.

1. Open Settings → People and add your domain under Sign-in domains.
2. Add the TXT record Kaloko shows to your DNS, or prove the domain by signing in with Google Workspace or Microsoft 365.
3. Choose Verify now. Until the domain is verified, nobody joins through it.

Connect your identity provider (Enterprise) #

1. Open Settings → Security → Company sign-in.
2. Choose SAML (Okta, Entra ID, Google Workspace and others) and paste your provider's metadata, or OIDC with the issuer, client ID and secret. Kaloko's SP metadata, with its certificate, is on the same page.
3. Choose Test connection. It shows what the provider sent and whether Kaloko would accept it; nobody is signed in by the test.
4. Enforce it. People from your verified domains then sign in only through your provider. Admins with a passkey or an authenticator app keep a way in for the day the provider is down.

Optional settings on the same page:

• Encrypted assertions: create the organization's key pair; a new pair keeps the old one working until you remove it.
• Single logout: signing out at the provider signs people out of Kaloko, and the other way round.
• Sign-in from the app tile of your provider, off until you turn it on, with the page people land on.
• Your provider's second step can count as Kaloko's, for sign-ins through your provider.

Guests from other companies keep signing in as before and see only the projects they were invited to.

Provision people with SCIM (Enterprise) #

Paste the SCIM base URL and token from Settings into Okta or Entra ID. Assigned people join with their domain's default role, pushed groups map to projects and roles, and removing someone ends their sessions, revokes their tokens and takes them off every project. SCIM never makes anyone an admin.

Limit where people come from (Enterprise) #

Under Settings → Security you can allow IP ranges for the app and, separately, for the API, the CLI and MCP. The audit log can be streamed to a signed webhook, Splunk or a Datadog-compatible endpoint.

Where your data lives and who processes it: Security and data.

Publish a changelog with pictures

Your accepted runs already hold every screen of a release. Kaloko turns CHANGELOG.md in your repository into a page with before and after pictures that clients, support and marketing can read. Kaloko's own changelog is made this way and is public: kaloko.app/p/sinfin/kaloko/changelog.

Set it up once #

npx kaloko init --docs=changelog

It adds CHANGELOG.md (Keep a Changelog) and the changelog block in kaloko.config.yml. A repository with several apps lists them under products:; npx kaloko products sync puts them on Kaloko before their first release.

Put pictures in the entry #

A picture names a step instead of a file:

- The pricing page compares every plan in one table.
  ![Before](kaloko:KALOKO-public/pricing?version=prev) ![After](kaloko:KALOKO-public/pricing)

KALOKO-public is the scenario id and pricing the step id. Without ?version= the reference shows the newest accepted screen; ?version=prev shows the one from the release before.

Draft the entry #

npx kaloko changelog draft            # prints the Unreleased entry
npx kaloko changelog draft --apply    # writes it into CHANGELOG.md

The draft collects the pull requests and commits since the last release, groups them into Added, Changed, Fixed and Removed, and adds before and after pictures of the screens accepted runs show differently. Rewrite the lines for people: what they can do now, one change per line. --since <tag> drafts from another release.

Record the release #

npx kaloko release 2.4.0

Unreleased becomes 2.4.0, the release is recorded with the accepted runs that are its pictures, and the command prints the changelog page. Without a version, Kaloko reads it from git tags, package.json, release-please or changesets.

When release-please, changesets or semantic-release make your releases, they keep writing CHANGELOG.md and Kaloko only reads it. npx kaloko ci github --release adds a workflow that records every published GitHub release.

Who reads it #

The changelog is under Products in the app. Everyone who sees one of the product's projects reads it, guests included. On Business and Enterprise an admin of those projects can choose on the page who else reads it:

• Readers from our domain: everyone who signs in from a verified domain of your organization.
• Anyone with the link: a public page with an Atom feed. Search engines index it only when you allow it.

Readers can also ask for a "What's new" e-mail on the changelog page (Business and up): each new release arrives with its pictures in their language.

How readers use the page: Read what's new. To keep guides current the same way, see Publish living docs.

Publish living docs and catch stale sections

Guides in your repository can show the screens of your accepted runs instead of pasted screenshots, so their pictures never go stale. When a screen changes after the text was written, Kaloko tells you which section to check. The pages you are reading now are made this way: kaloko.app/p/sinfin/kaloko/docs.

Set up the folder #

npx kaloko init --docs=full

It creates docs/ with the four Diátaxis folders, each with a template:

index.md is the start page. Front matter is optional: title (else the first heading) and order (the position in the sidebar). Link pages with relative .md links.

Write a page #

Pictures are step references, as in the changelog:

1. Choose **Sign in** in the top bar.
   ![The sign-in page](kaloko:KALOKO-public/login)

A translation sits next to its original: pay.cs.md beside pay.md, with the languages listed in the config (docs.languages: [en, cs], originals first). Pictures follow the language of the text.

One action per list item with its picture right under it also makes the page a walkthrough: readers choose Walk through it step by step and see one step at a time, and can tick I tried it.

Publish #

npx kaloko docs publish                     # the docs of latest
npx kaloko docs publish --version 2.4.0     # the docs of a release (Business)

Readers get a version selector, language and screen switches and search, and every picture opens its step on the canvas. kaloko release publishes the docs folder for its version on its own unless you pass --no-docs. Who reads the docs follows the same rules as the changelog: see Publish a changelog with pictures.

Catch stale sections #

When you publish, Kaloko remembers the screen behind every reference. When a later accepted run shows a different screen, the section says "The screen changed in 2.4 — check the text." to the people who write the docs (Business).

npx kaloko docs check             # against the published docs; needs a token
npx kaloko docs check --offline   # only the local files

Errors: broken references, steps the scenario files do not have, references without an accepted picture and stale sections. Warnings: missing or outdated translations and changelog items that people will see but that have no picture. The command exits with 1 on errors; --strict also on warnings, which suits CI.

For each stale section, open the step, compare it with the text and rewrite what no longer fits; the note clears at the next kaloko docs publish. If the text still fits, kaloko docs publish --reviewed clears the notes without a change. Use it only after you have read them.

To get the docs out as files for another site, see Export docs and the changelog.

Export docs and the changelog

Your docs site may already have its own build, or a client wants the guide as a PDF. kaloko export writes the docs and the changelog with their pictures as plain files, so nothing stays locked in Kaloko.

Pick a format #

npx kaloko export --format docusaurus --out website/docs

The step references become ordinary image files, in the language and screen size of the export.

Choose what goes in #

• --what docs or --what changelog exports only one of them; the default is both.
• --version 2.4.0 exports the docs of a release instead of the latest.
• --lang cs exports one language.
• --product shop picks a product when the config has several.
• --out names the folder or file to write.

Published or local #

By default the export takes what was published with kaloko docs publish. --local takes the files in your repository instead, without publishing them, which is handy for checking a change before it goes out.

Keep a docs site in step #

An existing site keeps its own build. Either export into its folder and commit the result, or run the export in the site's CI before it builds:

npx kaloko export --format vitepress --what docs --out docs-site/guide

The token in CI needs only read access; see Roles and API tokens.

Download from the docs page #

Readers who prefer a file can use Download on the docs or changelog page. It offers the same formats; on plans below Business only Markdown and JSON are available there too.

Every export opens without Kaloko. For the source of the pages and how to keep them current, see Publish living docs and catch stale sections.

CLI commands

Every command of the kaloko CLI with what it does. Run them with npx kaloko <command> in a project that has the package installed. npx kaloko help prints the same list with every option.

Options that work everywhere: --run <id|dir> (or KALOKO_RUN) picks the run, --config <file> the config file, and KALOKO_AGENT=<name> labels what an agent writes.

Set up a project #

Walk and capture #

Results and review #

Shared runs #

Jira, Linear and hosted runs #

Products, changelog and docs #

Design #

kaloko help prints all of this with every option.

Scenario file

A scenario is a YAML file (usually in qa/flows/) that describes one flow: its steps, the arrows between them and what each step must meet. Its format is qawalk.scenario.v1; kaloko validate <file> checks a file against it. How to write one step by step: Write a scenario.

schema: qawalk.scenario.v1
id: WEB-12-checkout
title: Checkout
locales: [en, cs]
viewports: { desktop: [1280, 800], mobile: iphone-15 }
environments: [local, staging]
nodes:
  - id: cart
    kind: screen
    title: { en: Cart, cs: Košík }
    path: /cart
    purpose: The shopper sees what they are buying and can continue.
    execution: { strategy: playwright, script: steps/cart.mjs, instructions: Add one product and open the cart. }
    criteria:
      - { id: AC1, text: Shows the total, check: { type: deterministic, assert: exists, selector: .cart-total } }
  - id: payment
    kind: screen
    title: Payment
    path: /checkout/payment
    packs: [a11y]
edges:
  - { from: cart, to: payment }

Top-level keys #

schema, id, title, nodes and edges are required.

Step keys (nodes) #

Every step needs id, kind and title.

Criteria #

Each criterion has an id (AC1, AC12b), a text for people, an optional highlight (a CSS selector to outline on the screenshot) and exactly one check: deterministic, semantic or manual. Every check type and option is listed in Criteria, checks and packs.

Edges #

Capture #

A step's capture wins over the scenario's, and the scenario's over the config's.

Criteria, checks and packs

A criterion is one thing a step must meet. It has an id (AC1), a text for people and exactly one check. How to add them to a scenario: Add criteria and packs. How the results turn into a verdict: How verdicts are decided.

Kinds of check #

Numbers, dates and counts belong in deterministic checks, never in a semantic question. Ask one thing per question.

Deterministic checks (assert) #

Common options: selector (CSS), pattern (a regex) with flags (i, m, s), value, min, max, all (every match must pass, not just one).

Page content #

Address, metadata and links #

Styles and values in the page #

Accessibility (the a11y pack uses these) #

Performance, apps and design #

Packs #

packs: [seo, a11y] on a step adds a fixed set of criteria. A criterion of your own with the same id replaces the pack's, which is how you change a limit (AC401 with max: 4000).

The a11y pack covers part of WCAG 2.2 AA. It does not replace an audit with a screen reader.

Configuration file

kaloko.config.yml sits in the root of the repository and tells the CLI where your flows run. kaloko init writes a commented one; kaloko doctor checks it. Its format is qawalk.config.v1, and schema, org and environments are required.

schema: qawalk.config.v1
org: acme
project: shop
scenarios: [qa/flows/*.yml]
environments:
  local:
    base_url: http://localhost:3000
  staging:
    base_url: https://staging.acme.test
    access:
      basic: { user_env: STAGING_USER, password_env: STAGING_PASSWORD }
    accounts:
      member: { user: qa@acme.test, password_env: MEMBER_PASSWORD }
  production:
    base_url: https://acme.com
    readonly: true
    blocked_paths: ["/account/**"]
walk:
  browsers: [chromium]
share:
  expires_days: 30

Top-level keys #

Environments #

Each key under environments is a name you use with kaloko start --env.

Evaluators #

Docs and products #

Without products, the project is the product.

Secrets #

Secrets never go into the config. Wherever one is needed, the config names an environment variable (password_env: STAGING_PASSWORD, { env: CF_ACCESS_CLIENT_SECRET }). Kaloko reads the value from:

1. the environment of the process,
2. .env next to kaloko.config.yml,
3. ~/.config/kaloko/.env.

A variable already set wins over the files. Values are removed from captured HTML, page text and console output. kaloko doctor says which named variables are missing. The token for sharing is KALOKO_TOKEN unless share.token_env says otherwise; see Roles and tokens.

MCP tools

Kaloko's MCP server lets an agent read runs, feedback, designs, the changelog and docs, and answer reviewers. It is part of the Team plan and up. Capturing, evaluating and uploading stay in the CLI.

Connect #

The server is https://kaloko.app/mcp (MCP over Streamable HTTP). Authenticate with an organization API token in the Authorization header; how to create one: Roles and tokens.

claude mcp add --transport http kaloko https://kaloko.app/mcp --header "Authorization: Bearer $KALOKO_TOKEN"

Other agents (Codex, Cursor) take the same URL and header as an HTTP MCP server. A read-only token is enough to read; tools that write need a token with that scope, as noted below. Screenshots and files are not sent over MCP: the answers carry their URLs, which the same token can fetch.

Runs and review #

Feedback and issues #

Design #

Screens, changelog, docs and products #

Errors come back as tool results marked as errors (an unknown run, a missing argument). A token that is not valid gets HTTP 401; an organization below Team is told the MCP server is not in its plan.

How Kaloko maps to your applications

One repository is rarely one app. A monorepo holds several apps, one app spans several repositories, an agency has a project per client, and every team names its environments differently. Kaloko uses a small set of concepts that you map to your situation once.

The concepts #

Products are optional #

Without products, a project is the product and its changelog is the project's changelog. Most teams start there. When you need more, list products under products: in kaloko.config.yml, and scenarios name theirs with product: and module:. Nothing has to be moved.

A project can cover several products, and a product can be worked on in several projects. Docs and changelog entries refer to steps, so they know their product and module from the scenario.

Versions belong to products #

An environment only says where a run happened. A run is tied to a product version, and the docs for version 2.4 show the accepted runs of 2.4. The pictures come from the environment the product names as its reference (reference_env), for example staging for an internal tool or production-cz for a Czech-only portal.

Four common set-ups #

1. One app, one team. No products; the project is the app and has one changelog. This is the default.
2. A monorepo with several apps. One product per app; projects per team or per app.
3. An agency. A project per client and a product per deliverable. Client guests read only the changelog and docs of their product.
4. A large app with modules. One product with modules per team. Each team keeps its module's docs, and the release has one combined changelog with a section per module.

kaloko init proposes a product for each app under apps/ that has its own package.json. kaloko products sync puts them on Kaloko before their first release, and Products in the app's menu lists every product you read.

Every piece works alone #

You can use only the changelog, only the docs, only one module's docs, only the pictures for a docs site you already have (kaloko export), or only the "What's new" e-mail. Nothing requires the whole set.

How verdicts are decided

Every criterion on the canvas ends as pass, partial, fail or waiting for a person. Three kinds of judge take part, in a fixed order: checks that count, an AI evaluator for what needs judgement, and people, who have the last word.

Checks that count come first #

Whatever can be measured is checked in the live page while the step is captured: a text is there, a field has a label, the address after the step matches, a colour equals a design token, the page loads under a time limit. Each result keeps its evidence, such as the element that matched or the measured value, so you can see why it passed or failed. These checks run without any AI and never cost an evaluator question.

An evaluator for judgement #

Criteria such as "the error message tells the person what to do" go to a semantic evaluator. The default is JEV by TypeSafe. It reads a reduced outline of the page, not the screenshot, and returns a score between 0 and 1. Above the pass threshold (0.85 by default) the criterion passes, below the fail threshold (0.15) it fails, and in between it is marked for a person to decide. The thresholds are set in kaloko.config.yml.

Criteria marked vision: true, and answers in the unsure band, get a second look from a vision evaluator that sees the screenshot. It also says in one sentence what decided it; that sentence shows as "Why" on the canvas.

The evaluator runs with your own key in any plan, or through Kaloko from the Team plan up, within a monthly number of questions. Answers are kept, so evaluating an unchanged screen again asks nothing new. Without either, these criteria stay "not evaluated" until a reviewer decides. kaloko calibrate compares the evaluator with your reviewers' verdicts and suggests thresholds.

People decide #

A reviewer approves a step, returns it with a note, or decides a criterion the evaluator was unsure about. At the end they accept or return the whole run. Every decision carries a name.

• Required approvers (Team plan and up): a run counts as accepted only after the named people approved it.
• Approvals carry over. When a new run is shared, a step whose screenshots and criteria did not change since the last accepted run, and whose checks still pass, keeps the approval a person gave it, with who and when. Only the rest waits in review mode. Anyone can revoke a carried approval. A carried approval never accepts the run and never counts for required approvers.
• Agents never approve for people. Decisions made by an agent or a token are marked as theirs and do not count as a person's.

Why in this order #

Counting is cheap and exact, so it goes first. Judgement is expensive and sometimes wrong, so it only handles what counting cannot, and it says when it is unsure. People see the evidence and the reason next to every verdict, and their decision is the one that is recorded as acceptance.

Security and your data

Kaloko runs where your code and your agent are: the CLI walks, captures and checks on your machine or in your CI. The service at kaloko.app stores what you share, shows the canvas and collects decisions. This page explains what travels where. Where the data is stored and which companies process it is on the trust page, which is kept up to date.

What leaves your machine #

Nothing until you share. kaloko walk, evaluate and preview work locally, and the local canvas opens from disk. kaloko share uploads the run: screenshots, the captured HTML, criteria with their evidence, and recordings where your plan keeps them. Files travel by content hash, so a capture that did not change is not sent again.

What the evaluator sees #

For text criteria the evaluator gets a reduced outline of the page with e-mail addresses and tokens masked, never the raw HTML. A screenshot goes to the vision evaluator only for criteria marked vision: true and for answers the text evaluator was unsure about. With your own key the request goes from your machine to that provider; without one, it goes through Kaloko.

Secrets stay out #

Secrets are never written in kaloko.config.yml. The config names an environment variable of your choice (password_env: STAGING_PASSWORD), and the value is read from the environment, the project .env or ~/.config/kaloko/.env. Values of those variables are removed from captured HTML, page text and console output before anything is stored. Sessions saved with kaloko auth save stay on your machine and are never uploaded.

Production is read-only #

An environment marked as production is always read-only in the CLI: public pages as a visitor, no sign-in to back offices, no data created. A scenario must declare readonly: true to run there.

Runs expire #

Every shared run has an expiry date, 30 days by default, at most your plan's retention, and is deleted after it. kaloko keep keeps a run for good; on Business and Enterprise accepted runs do not expire while the plan lasts. From the Team plan up you can download a run as a ZIP at any time with kaloko export.

How files are served #

Uploaded files are served from a separate domain under short-lived signed links. Captured HTML is shown in a sandbox without scripts.

Who gets in #

People sign in with a one-time e-mail link or with Google, Microsoft or GitHub, and can add a passkey or an authenticator app as a second step. Enterprise adds company sign-in over SAML or OIDC and SCIM.

Roles decide what a person may do: a Viewer reads, comments, approves and returns; a Designer or a Developer / Tester also uploads; an Admin also manages members, tokens and settings. Viewers are free in every plan. API tokens carry their own scopes and expiry, and the CLI's token from kaloko login lasts at most 30 days.

Questions a security review asks #

The trust page answers where data lives, how it is encrypted, the list of subprocessors and where the SOC 2 preparation stands. For a vulnerability report or the data processing agreement, write to the contact given there.

Changelog