Docs

Kaloko

Download
Markdown with pictures (.zip)Docusaurus folder (.zip)VitePress folder (.zip)MkDocs folder (.zip)Offline HTML (.zip)PDFChangelog as JSON
Pages

Webhooks

Kaloko can call your URL when something happens to a run: it is shared, accepted or returned, a step is approved or returned, someone comments or resolves a thread, its evidence is ready, or its health changes. Your docs pipeline can rebuild from the newest captures, and a QMS portal can show the verdict and the open comments next to its test cases without asking Kaloko every minute. Webhooks are part of the Business plan and up.

Set one up

An organization admin adds the URL in Settings → Integrations → Webhooks and ticks its events and projects. Each webhook has its own signing secret, shown next to it. An organization can have up to 10 webhooks.

A webhook takes every open project (and runs without a project), or only the projects you tick. Events of a restricted project reach only the webhooks that name it, so a client's portal hears about its own project and nothing else. A guest's comment counts for the project of the run they commented on.

The same settings work over the API with an admin token:

RequestWhat it does
GET /api/v1/orgs/<org>/webhooksYour webhooks with their events, projects and secrets
POST /api/v1/orgs/<org>/webhooksAdds one: { "url", "events": [...], "projects": [...], "all_projects": true }
PATCH /api/v1/orgs/<org>/webhooks/<id>Changes its events or projects; what you leave out stays
DELETE /api/v1/orgs/<org>/webhooks/<id>Removes it
GET /api/v1/orgs/<org>/webhooks/<id>/deliveriesThe newest deliveries: event, state, tries, last answer

In projects, an empty string stands for runs without a project. Without projects and all_projects, a new webhook takes every open project.

Events

EventWhen
run.sharedA run was shared (uploaded and finished)
run.acceptedA run was accepted
run.returnedSomeone returned the whole run, with an excerpt of their note
review.step_changedA step was approved or returned, or its decision was revoked: deleted, resolved, or a carried approval withdrawn
comment.addedA new comment on a run: a thread, a reply, or a returned step with its note
comment.resolvedA thread was resolved, in Kaloko or by its Jira or Linear issue being done
comment.changedA comment was edited or deleted, or its thread reopened (change says which)
evidence.readyA run's or a release's evidence can be fetched
health.worseThe health of a scheduled or CI run got worse than the walk before it
health.changedA shared run has health problems that are new or fixed since the run before it

Approving a step is not a comment, so it raises only review.step_changed. Returning a step raises both: the decision and the thread that explains it.

What a delivery looks like

Each delivery is a JSON POST with these headers:

HeaderValue
Kaloko-EventThe event, e.g. comment.added
Kaloko-Signaturet=<unix time>,v1=<hex HMAC-SHA256 of "<t>.<body>" with the webhook's secret>
User-AgentKaloko-Webhooks/1

The same values also go out as QAwalk-Event and QAwalk-Signature for receivers written before the rename.

The body is an envelope around the event's data:

{
  "id": "evt_Xb3kQ9vT2mLp",
  "type": "comment.added",
  "created_at": "2026-10-09T10:14:03.512Z",
  "org": "acme",
  "data": {
    "run": {
      "id": "2026-10-09-checkout-a1b2c3",
      "scenario": "checkout",
      "project": "shop",
      "canvas": "https://kaloko.app/acme/runs/2026-10-09-checkout-a1b2c3/",
      "api": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-09-checkout-a1b2c3"
    },
    "step": "payment",
    "locale": "en",
    "viewport": "desktop",
    "thread_id": "k3Jd9QpZr",
    "comment_id": "k3Jd9QpZr",
    "kind": "comment",
    "author": { "email": "pam@acme.example", "via": "session" },
    "excerpt": "The total shows 0 Kč after the coupon. Ask [email] about the rounding rule.",
    "resolved": false,
    "at": "2026-10-09T10:14:03.401Z",
    "url": "https://kaloko.app/acme/runs/2026-10-09-checkout-a1b2c3/#step=payment&comments=step&thread=k3Jd9QpZr",
    "comments_api": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-09-checkout-a1b2c3/comments"
  }
}

Answer with any 2xx within 5 seconds. Kaloko sends after it has answered the person who made the change, so a slow receiver never holds anyone up.

Verify the signature

Compute the HMAC over the timestamp, a dot and the raw body exactly as it arrived, compare it in constant time and refuse old timestamps:

import { createHmac, timingSafeEqual } from 'node:crypto'

export function verify (rawBody, header, secret, toleranceSeconds = 300) {
  const { t, v1 } = Object.fromEntries(header.split(',').map((p) => p.split('=')))
  if (!t || !v1 || Math.abs(Date.now() / 1000 - Number(t)) > toleranceSeconds) return false
  const expected = createHmac('sha256', secret).update(`${t}.${rawBody}`).digest('hex')
  return expected.length === v1.length && timingSafeEqual(Buffer.from(expected), Buffer.from(v1))
}
import hashlib, hmac, time

def verify(raw_body: bytes, header: str, secret: str, tolerance: int = 300) -> bool:
    parts = dict(p.split("=", 1) for p in header.split(","))
    t, v1 = parts.get("t"), parts.get("v1")
    if not t or not v1 or abs(time.time() - int(t)) > tolerance:
        return False
    expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, v1)

Retries and repeats

When your endpoint does not answer 2xx, Kaloko tries twice more within a few seconds, then again over about a day at growing intervals (from a few minutes up to 12 hours); after that it gives up. Each try is signed afresh, but the event id stays the same, so keep the ids you have handled and skip repeats. The same change is never sent twice as two events: one comment is one comment.added.

Settings shows the last answer of each webhook and its recent deliveries (delivered, will try again, gave up), and the API lists them under …/deliveries. Deliveries are listed for 30 days.

Payloads

Payloads carry ids, states, short excerpts and links. Screenshots, attachments and whole comments stay behind the API: fetch them with your own token from the api and comments_api URLs. An excerpt is one line of at most 140 characters, with e-mail addresses replaced by [email].

Every event about a run has run: id, scenario, project (or null), canvas (the run in Kaloko) and api. Times are ISO 8601 in UTC. via says how a person acted: session (in the app), token (CLI, CI, MCP), agent or integration (Jira, Linear).

run.returned

{
  "run": { "id": "…", "scenario": "checkout", "project": "shop", "canvas": "…", "api": "…", "environment": "staging", "verdict": "returned" },
  "by": { "email": "ada@acme.example", "via": "session" },
  "at": "2026-10-09T11:02:41.118Z",
  "note": "Checkout total is wrong on the coupon step."
}

review.step_changed

{
  "run": { "id": "…", "scenario": "checkout", "project": "shop", "canvas": "…", "api": "…" },
  "step": "payment",
  "locale": "en",
  "viewport": "desktop",
  "state": "returned",
  "by": { "email": "ada@acme.example", "via": "session" },
  "at": "2026-10-09T10:20:12.006Z",
  "comment_id": "Qm2xYt7Lc",
  "note": "Total shows 0 Kč",
  "url": "https://kaloko.app/acme/runs/…/#step=payment",
  "review_queue_api": "https://kaloko.app/api/v1/orgs/acme/runs/…/review-queue"
}

state is approved, returned or revoked. comment_id names the decision; a withdrawn carried approval has "carried": true instead. locale and viewport appear when the decision was made on one capture. note comes with returned steps.

comment.added, comment.resolved, comment.changed

FieldMeaning
stepThe step id, or null for a comment on the whole run
locale, viewportThe capture the comment was made on, when there is one
thread_idThe thread's first comment; equal to comment_id for a new thread
comment_idThe comment the event is about
kindcomment, reply or returned (a returned step's note)
changecomment.changed only: edited, deleted or reopened
authorWho wrote the comment: email, via and, for agents, agent
byWho resolved, edited, deleted or reopened it
excerptThe start of the comment; left out when it was deleted
resolvedWhether the thread is resolved now
urlThe thread on the canvas
comments_apiAll comments of the run, for your token

The url opens the step with its comments and the thread in view: https://kaloko.app/<org>/runs/<run>/#step=<step>&comments=step&thread=<thread>. A comment on the whole run links to …/runs/<run>/#comments=run.

run.shared, run.accepted, evidence.ready, health.worse

run.shared and run.accepted carry the run with its steps and where to fetch the newest captures and the changes since the previous accepted run (steps_url, changes_url). evidence.ready names the run or release and the URLs of its evidence, package and trace matrix: Quality evidence. health.worse carries the run, the previous one and the new findings: Check the health of your app.

health.changed

Sent after every shared run of a scenario whose health checks moved against the previous run in the same environment: something fails (or the evaluator is unsure) that did not before, or something that failed is fine now. It counts per page, language and check, so it suits a weekly SEO walk over thousands of pages. A first run, and a run where nothing moved, send nothing.

{
  "run": { "id": "2026-10-12-seo-weekly-a1b2c3", "scenario": "seo-weekly", "title": "Weekly SEO", "environment": "production", "started_at": "2026-10-12T05:00:00.000Z", "canvas": "https://kaloko.app/acme/runs/2026-10-12-seo-weekly-a1b2c3/#health" },
  "previous": { "id": "2026-10-05-seo-weekly-d4e5f6", "started_at": "2026-10-05T05:00:00.000Z" },
  "counts": { "new": 14, "fixed": 3, "still_open": 120, "unchecked": 0, "worse": 1, "expired": 0 },
  "open": 134,
  "new": [
    { "id": "eshop|de|HC112|sklep-kubek", "step": "eshop", "locale": "de", "check": "HC112", "text": "The page carries structured data (JSON-LD)", "page": "sklep-kubek", "url": "https://shop.example/de/eshop/kubek", "severity": "warning", "first_seen": { "run": "2026-10-12-seo-weekly-a1b2c3", "at": "2026-10-12T05:00:00.000Z" } }
  ],
  "fixed": [],
  "aggregate_url": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-12-seo-weekly-a1b2c3/health/aggregate",
  "diff_url": "https://kaloko.app/api/v1/orgs/acme/runs/2026-10-12-seo-weekly-a1b2c3/health/diff"
}

new and fixed list the first 20; the rest is behind diff_url. The fields are explained in Health API for dashboards.

On this pageOn this page