Webhooks
Kaloko can call your URL when something happens to a run: it is shared, accepted or returned, a step is approved or returned, someone comments or resolves a thread, its evidence is ready, or its health changes. Your docs pipeline can rebuild from the newest captures, and a QMS portal can show the verdict and the open comments next to its test cases without asking Kaloko every minute. Webhooks are part of the Business plan and up.
Set one up
An organization admin adds the URL in Settings → Integrations → Webhooks and ticks its events and projects. Each webhook has its own signing secret, shown next to it. An organization can have up to 10 webhooks.
A webhook takes every open project (and runs without a project), or only the projects you tick. Events of a restricted project reach only the webhooks that name it, so a client's portal hears about its own project and nothing else. A guest's comment counts for the project of the run they commented on.
The same settings work over the API with an admin token:
| Request | What it does |
|---|---|
GET /api/v1/orgs/<org>/webhooks | Your webhooks with their events, projects and secrets |
POST /api/v1/orgs/<org>/webhooks | Adds one: { "url", "events": [...], "projects": [...], "all_projects": true } |
PATCH /api/v1/orgs/<org>/webhooks/<id> | Changes its events or projects; what you leave out stays |
DELETE /api/v1/orgs/<org>/webhooks/<id> | Removes it |
GET /api/v1/orgs/<org>/webhooks/<id>/deliveries | The newest deliveries: event, state, tries, last answer |
In projects, an empty string stands for runs without a project. Without projects and all_projects, a new webhook takes every open project.
Events
| Event | When |
|---|---|
run.shared | A run was shared (uploaded and finished) |
run.accepted | A run was accepted |
run.returned | Someone returned the whole run, with an excerpt of their note |
review.step_changed | A step was approved or returned, or its decision was revoked: deleted, resolved, or a carried approval withdrawn |
comment.added | A new comment on a run: a thread, a reply, or a returned step with its note |
comment.resolved | A thread was resolved, in Kaloko or by its Jira or Linear issue being done |
comment.changed | A comment was edited or deleted, or its thread reopened (change says which) |
evidence.ready | A run's or a release's evidence can be fetched |
health.worse | The health of a scheduled or CI run got worse than the walk before it |
health.changed | A shared run has health problems that are new or fixed since the run before it |
Approving a step is not a comment, so it raises only review.step_changed. Returning a step raises both: the decision and the thread that explains it.
What a delivery looks like
Each delivery is a JSON POST with these headers:
| Header | Value |
|---|---|
Kaloko-Event | The event, e.g. comment.added |
Kaloko-Signature | t=<unix time>,v1=<hex HMAC-SHA256 of "<t>.<body>" with the webhook's secret> |
User-Agent | Kaloko-Webhooks/1 |
The same values also go out as QAwalk-Event and QAwalk-Signature for receivers written before the rename.
The body is an envelope around the event's data:
{
"id": "evt_Xb3kQ9vT2mLp",
"type": "comment.added",
"created_at": "2026-10-09T10:14:03.512Z",
"org": "acme",
"data": {
"run": {
"id": "2026-10-09-checkout-a1b2c3",
"scenario": "checkout",
"project": "shop",
"canvas": "https://kaloko.app/acme/runs/2026-10-09-checkout-a1b2c3/",
"api": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-09-checkout-a1b2c3"
},
"step": "payment",
"locale": "en",
"viewport": "desktop",
"thread_id": "k3Jd9QpZr",
"comment_id": "k3Jd9QpZr",
"kind": "comment",
"author": { "email": "pam@acme.example", "via": "session" },
"excerpt": "The total shows 0 Kč after the coupon. Ask [email] about the rounding rule.",
"resolved": false,
"at": "2026-10-09T10:14:03.401Z",
"url": "https://kaloko.app/acme/runs/2026-10-09-checkout-a1b2c3/#step=payment&comments=step&thread=k3Jd9QpZr",
"comments_api": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-09-checkout-a1b2c3/comments"
}
}
Answer with any 2xx within 5 seconds. Kaloko sends after it has answered the person who made the change, so a slow receiver never holds anyone up.
Verify the signature
Compute the HMAC over the timestamp, a dot and the raw body exactly as it arrived, compare it in constant time and refuse old timestamps:
import { createHmac, timingSafeEqual } from 'node:crypto'
export function verify (rawBody, header, secret, toleranceSeconds = 300) {
const { t, v1 } = Object.fromEntries(header.split(',').map((p) => p.split('=')))
if (!t || !v1 || Math.abs(Date.now() / 1000 - Number(t)) > toleranceSeconds) return false
const expected = createHmac('sha256', secret).update(`${t}.${rawBody}`).digest('hex')
return expected.length === v1.length && timingSafeEqual(Buffer.from(expected), Buffer.from(v1))
}
import hashlib, hmac, time
def verify(raw_body: bytes, header: str, secret: str, tolerance: int = 300) -> bool:
parts = dict(p.split("=", 1) for p in header.split(","))
t, v1 = parts.get("t"), parts.get("v1")
if not t or not v1 or abs(time.time() - int(t)) > tolerance:
return False
expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, v1)
Retries and repeats
When your endpoint does not answer 2xx, Kaloko tries twice more within a few seconds, then again over about a day at growing intervals (from a few minutes up to 12 hours); after that it gives up. Each try is signed afresh, but the event id stays the same, so keep the ids you have handled and skip repeats. The same change is never sent twice as two events: one comment is one comment.added.
Settings shows the last answer of each webhook and its recent deliveries (delivered, will try again, gave up), and the API lists them under …/deliveries. Deliveries are listed for 30 days.
Payloads
Payloads carry ids, states, short excerpts and links. Screenshots, attachments and whole comments stay behind the API: fetch them with your own token from the api and comments_api URLs. An excerpt is one line of at most 140 characters, with e-mail addresses replaced by [email].
Every event about a run has run: id, scenario, project (or null), canvas (the run in Kaloko) and api. Times are ISO 8601 in UTC. via says how a person acted: session (in the app), token (CLI, CI, MCP), agent or integration (Jira, Linear).
run.returned
{
"run": { "id": "…", "scenario": "checkout", "project": "shop", "canvas": "…", "api": "…", "environment": "staging", "verdict": "returned" },
"by": { "email": "ada@acme.example", "via": "session" },
"at": "2026-10-09T11:02:41.118Z",
"note": "Checkout total is wrong on the coupon step."
}
review.step_changed
{
"run": { "id": "…", "scenario": "checkout", "project": "shop", "canvas": "…", "api": "…" },
"step": "payment",
"locale": "en",
"viewport": "desktop",
"state": "returned",
"by": { "email": "ada@acme.example", "via": "session" },
"at": "2026-10-09T10:20:12.006Z",
"comment_id": "Qm2xYt7Lc",
"note": "Total shows 0 Kč",
"url": "https://kaloko.app/acme/runs/…/#step=payment",
"review_queue_api": "https://kaloko.app/api/v1/orgs/acme/runs/…/review-queue"
}
state is approved, returned or revoked. comment_id names the decision; a withdrawn carried approval has "carried": true instead. locale and viewport appear when the decision was made on one capture. note comes with returned steps.
comment.added, comment.resolved, comment.changed
| Field | Meaning |
|---|---|
step | The step id, or null for a comment on the whole run |
locale, viewport | The capture the comment was made on, when there is one |
thread_id | The thread's first comment; equal to comment_id for a new thread |
comment_id | The comment the event is about |
kind | comment, reply or returned (a returned step's note) |
change | comment.changed only: edited, deleted or reopened |
author | Who wrote the comment: email, via and, for agents, agent |
by | Who resolved, edited, deleted or reopened it |
excerpt | The start of the comment; left out when it was deleted |
resolved | Whether the thread is resolved now |
url | The thread on the canvas |
comments_api | All comments of the run, for your token |
The url opens the step with its comments and the thread in view: https://kaloko.app/<org>/runs/<run>/#step=<step>&comments=step&thread=<thread>. A comment on the whole run links to …/runs/<run>/#comments=run.
run.shared, run.accepted, evidence.ready, health.worse
run.shared and run.accepted carry the run with its steps and where to fetch the newest captures and the changes since the previous accepted run (steps_url, changes_url). evidence.ready names the run or release and the URLs of its evidence, package and trace matrix: Quality evidence. health.worse carries the run, the previous one and the new findings: Check the health of your app.
health.changed
Sent after every shared run of a scenario whose health checks moved against the previous run in the same environment: something fails (or the evaluator is unsure) that did not before, or something that failed is fine now. It counts per page, language and check, so it suits a weekly SEO walk over thousands of pages. A first run, and a run where nothing moved, send nothing.
{
"run": { "id": "2026-10-12-seo-weekly-a1b2c3", "scenario": "seo-weekly", "title": "Weekly SEO", "environment": "production", "started_at": "2026-10-12T05:00:00.000Z", "canvas": "https://kaloko.app/acme/runs/2026-10-12-seo-weekly-a1b2c3/#health" },
"previous": { "id": "2026-10-05-seo-weekly-d4e5f6", "started_at": "2026-10-05T05:00:00.000Z" },
"counts": { "new": 14, "fixed": 3, "still_open": 120, "unchecked": 0, "worse": 1, "expired": 0 },
"open": 134,
"new": [
{ "id": "eshop|de|HC112|sklep-kubek", "step": "eshop", "locale": "de", "check": "HC112", "text": "The page carries structured data (JSON-LD)", "page": "sklep-kubek", "url": "https://shop.example/de/eshop/kubek", "severity": "warning", "first_seen": { "run": "2026-10-12-seo-weekly-a1b2c3", "at": "2026-10-12T05:00:00.000Z" } }
],
"fixed": [],
"aggregate_url": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-12-seo-weekly-a1b2c3/health/aggregate",
"diff_url": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-12-seo-weekly-a1b2c3/health/diff"
}
new and fixed list the first 20; the rest is behind diff_url. The fields are explained in Health API for dashboards.