Configuration file
kaloko.config.yml sits in the root of the repository and tells the CLI where your flows run. kaloko init writes a commented one; kaloko doctor checks it. Its format is qawalk.config.v1, and schema, org and environments are required.
schema: qawalk.config.v1
org: acme
project: shop
project_name: E-shop
scenarios: [qa/flows/*.yml]
environments:
local:
base_url: http://localhost:3000
staging:
base_url: https://staging.acme.test
access:
basic: { user_env: STAGING_USER, password_env: STAGING_PASSWORD }
accounts:
member: { user: qa@acme.test, password_env: MEMBER_PASSWORD }
production:
base_url: https://acme.com
readonly: true
blocked_paths: ["/account/**"]
walk:
browsers: [chromium]
share:
expires_days: 30
Top-level keys
| Key | What it holds |
|---|---|
schema | always qawalk.config.v1 |
org | the organization's slug on kaloko.app |
project | the Kaloko project for runs (its slug, such as eco-podcasty); default: the repository's name. A scenario someone moved to another project (in the app or with kaloko scenario move) stays there: the move wins over this key |
project_name | the name people see on Kaloko (ECO Podcasty). The first kaloko share creates a missing project with it; kaloko projects sync renames an existing one. A share never renames, it only says when the names differ |
product | the slug of a product on Kaloko this project is part of; kaloko projects sync adds the project to it |
scenarios | glob patterns of scenario files |
output | the run folder (default tmp/kaloko) |
environments | where flows run (below) |
browser | channel (a Playwright channel such as chrome; empty = the bundled Chromium) and port of the shared browser (default 9333) |
walk | defaults for every run: concurrency (languages at once, 1–16), browsers, color_schemes, reduced_motion, forced_colors, display_modes |
capture | defaults for every scenario: record, trace |
evaluate | budget.calls: at most this many paid evaluator calls per evaluation (below) |
evaluators | the semantic evaluator jev, your own model, and the vision evaluator (below) |
pii | allow: values that look like personal data but are not (a company contact), * for any text; neither masked nor stopped by kaloko share |
share | url (default https://kaloko.app), token_env (default KALOKO_TOKEN), expires_days (default 30) |
docs | changelog and docs settings (below) |
products | what you release and version as a whole (below) |
traffic | Search Console and GA4 numbers per page, to rank findings by traffic (below); a scenario's traffic replaces it |
Environments
Each key under environments is a name you use with kaloko start --env.
| Key | What it holds |
|---|---|
kind | app (default) or design (a folder of live HTML steps) |
base_url | the address, or one per language with a default fallback; ${VAR} is filled from the environment (review apps) |
readonly | true for production: the CLI only reads, never signs in to back offices, never creates data |
pii | redact or pseudonymize: every run hides personal data; required: no run is created unless the scenario or kaloko start --pii hides it |
blocked_paths | path globs a read-only run never opens |
allowed_post_paths | path globs a read-only run may still POST to (a cart) |
may_create_data | false forbids creating data (default true) |
access | protection in front of the whole environment: basic (user_env, password_env), headers (service tokens), client_certificate, origins that receive them |
accounts | people who sign in, keyed like the scenario's account: user or user_env, password_env, totp_env, storage_state (a sign-in saved by kaloko auth save) |
mail | the mailbox: provider (mailpit, script, none), url, user_env, password_env, address, script, options, auth |
app | the app under test per platform: android, ios, electron, desktop, windows |
serve, fixtures, default_token_mode | design environments: the folder of steps, JSON fixtures, the default token mode (tokens_mode, its older name, still works) |
token_modes | design environments: the token modes every step is rendered in (default every mode of the set; false only default_token_mode) |
origins | design environments: the addresses the design loads fonts, stylesheets and images from (https://fonts.googleapis.com); the organization allows them in Settings → Security |
html_view | experimental: keep a static HTML view of every capture |
Evaluators
| Key | What it holds |
|---|---|
evaluators.jev.api_key_env | the variable with your own evaluator key (default TYPESAFE_API_KEY) |
evaluators.jev.hosted | without a local key, evaluate through your plan's allowance (default true); false uses only your key |
evaluators.jev.thresholds | pass and fail scores |
evaluators.jev.model, base_url, max_state_chars, signals, concurrency | finer settings, rarely needed |
evaluators.vision | the second evaluator that looks at the screenshot (provider anthropic, key in ANTHROPIC_API_KEY); false turns it off |
evaluators.<name> | your own text evaluator: kind: openai_compatible, base_url, model, optionally api_key_env, concurrency (default 2), max_state_chars (default 24000; the page text is cut to it), timeout_ms, response_format (json_schema, json_object or none) |
evaluators.text | which evaluator answers the semantic questions first: jev (default) or the name of your own one |
evaluators.escalate_to | jev: answers of your own model that are not clear enough for a verdict are asked again of JEV. Only this way round |
evaluate.budget.calls | uncached JEV calls per evaluation (your key or through Kaloko), escalations included; calls to your own model are not counted. Steps go first, then each stack's sample, chosen by priority (pages failing a check, then those checked longest ago). What does not fit stays not_evaluated with a note; kaloko evaluate --budget N overrides it |
evaluators.triage | experimental: Clef names the kind of problem on each screen and in recording frames, advisory only. model (clef or the faster clef-flash), keep (shown from this probability, default 0.7), frames_per_step (0 skips recordings), cascade (let Clef settle criteria the text evaluator is unsure about); false turns it off |
A local model in Ollama answers first and hands the unclear answers to JEV:
evaluate:
budget: { calls: 300 }
evaluators:
ollama:
kind: openai_compatible
base_url: http://localhost:11434/v1 # LM Studio: http://localhost:1234/v1
model: qwen3:8b
text: ollama
escalate_to: jev
Your own evaluator is used only when text names it, and the page text goes only to its base_url. Its verdicts are marked "evaluated by your own model" on the canvas and in the pull request comment.
Docs and products
| Key | What it holds |
|---|---|
docs.dir | docs sources (default docs) |
docs.changelog | the changelog file (default CHANGELOG.md) |
docs.style | keep-a-changelog or release-notes |
docs.version_source | auto, git-tag, package.json, release-please, changesets, semantic-release, pyproject, cargo, gemspec, manual |
docs.languages | languages of the docs; the first is the originals' |
docs.framework | auto, docusaurus, vitepress, mkdocs, nextra, plain |
docs.publish | who reads: members, domain or public (the last two need an admin) |
docs.images | viewport and locale (reader or a language) of the pictures |
products.<slug> | name (text or per language), version (source, path, pattern), docs, project or projects, modules, reference_env, default_scenario, publish |
Without products, the project is the product.
Traffic data
traffic joins search and analytics numbers onto the pages a run walks, so a missing description on the best-selling product comes before the same mistake on a page nobody finds. It is off until you add it, and Kaloko asks Google for nothing unless gsc or ga4 is set.
traffic:
file: qa/traffic/search-console-pages.csv # an export from Search Console or GA4
gsc:
property: sc-domain:acme.com # or https://www.acme.com/
credentials_env: GOOGLE_APPLICATION_CREDENTIALS
ga4:
property: "123456789"
low_ctr: { min_impressions: 100, ratio: 0.5 }
| Key | What it does |
|---|---|
file | one or more CSV exports, relative to the file that names them: Search Console's Pages table or a GA4 report by landing page. Kaloko reads English and Czech headers (Page, URL, Landing page, Stránka, Vstupní stránka; Clicks/Kliknutí, Impressions/Zobrazení, CTR, Position/Pozice, Sessions/Relace, Conversions or Key events/Konverze) and numbers as either language writes them (1 234, 1,234, 3,5 %) |
gsc.property | the Search Console property. Kaloko signs in as a service account, asks searchAnalytics.query for clicks, impressions, CTR and position by page over the 28 days that end two days ago (days changes the length), and caches the answer per run |
ga4.property | the GA4 property id. Sessions and key events (conversions) by landing page over the 28 days that end yesterday, through the Data API |
credentials_env | the variable with the path of the service account's JSON key, or the JSON itself (a CI secret). Default GOOGLE_APPLICATION_CREDENTIALS. Add the service account's e-mail to the Search Console property as a user and to GA4 as a viewer |
low_ctr | thresholds of HC127 (below): pages with at least min_impressions (default 100) whose CTR is below ratio (default 0.5) times the site's median |
Pages are matched by address: the host without www., the path without a trailing slash and the query without tracking parameters (utm_*, gclid, …); the scheme and the hash do not matter. A GA4 path (/cs/eshop/chair) matches that path when the stack walks one host; a stack across several hosts leaves such rows out and says so in stack.traffic.note. A walk on a host the data has no rows for at all (staging, a preview) finds production numbers by the path when the data names it on one host only. A CTR column without % signs is read as percentages when any value in it is above 1, otherwise as fractions. When a page is both in an export and in the API answer, the API's numbers are used, not the sum. The API window is counted from the start of the run, so every build of one run uses the same numbers.
What you get:
- every page of a stack has
traffic: { clicks, impressions, ctr, position, sessions, conversions }initems.json, and the page index on the canvas gets Impressions and Clicks columns you can sort by; kaloko export findingsadds those columns andimpact, and lists the findings with the highest impact first.impact = weight × (impressions + 10 × (clicks + sessions) + 100 × conversions), where the weight is 3 for an error or a failed acceptance criterion and 1 for a warning. A page without numbers has impact 0;- a stack with the
seopack gets HC127: pages with many impressions and a CTR below half the site's median CTR at similar positions (1–3, 4–10, 11–20, each with at least five pages to compare), which usually means the title and description in the results need a rewrite. Without traffic data the check is not there at all.
A source that cannot be read is reported on stderr and left out; the run still builds. The service account key never leaves your machine and is never written into the run; see Security and your data.
Secrets
Secrets never go into the config. Wherever one is needed, the config names an environment variable (password_env: STAGING_PASSWORD, { env: CF_ACCESS_CLIENT_SECRET }). Kaloko reads the value from:
- the environment of the process,
.envnext tokaloko.config.yml,~/.config/kaloko/.env.
A variable already set wins over the files. Values are removed from captured HTML, page text and console output. kaloko doctor says which named variables are missing. The token for sharing is KALOKO_TOKEN unless share.token_env says otherwise; see Roles and tokens.
Kaloko · latest · 2026-11-21