Kaloko

All guides › By use case

By use case

E-mail flows: capture the message next to the screen that sent it

Sign-up confirmation, order receipt, password reset. Kaloko captures the e-mail from Mailpit or letter_opener as a step of the flow, checks its subject and links, and shows it on the canvas next to the form.

The form works. The e-mail says "Hello {{first_name}}". Every team has shipped this once; Kaloko makes the message a step of the flow so it is looked at with the screens around it.

The problem

E-mails live outside the browser, so end-to-end tests skip them or assert on a subject string. Nobody sees the rendered message in both languages until a customer does.

What changes with Kaloko

An email step in the scenario. kaloko mail waits for the message in the test mailbox (Mailpit on staging, letter_opener locally in Rails projects), captures it like a screen, and the criteria check subject, sender, links and copy. On the canvas the e-mail sits right after the form that triggered it.

How it works

  1. Unique address per run

    kaloko start prints a test address (qa+<run>@…); the agent uses it in the form so the run finds its own messages.

  2. Trigger and capture

    after the form step, kaloko mail --step confirmation --locale cs waits for the message and stores it with rendered HTML and the plain text.

  3. Check it

    deterministic criteria on subject and links (the confirmation link must point at your host), a semantic question for tone and completeness, a screenshot for the humans.

  4. Review

    the reviewer sees the form and the e-mail side by side per locale and approves or returns them like any other step.

Skills and prompts

Walk the sign-up flow on local including the confirmation e-mail in cs and en, and check that the link in the mail opens the account page.

Expected outcome: screens of the form and the account page, an e-mail step per locale with subject and sender, criteria on the link target, a shared canvas.

What you get

FAQ

Which mailboxes are supported?

Mailpit (mail.provider: mailpit with its URL) and Rails letter_opener (mail.provider: letter_opener, folder tmp/letter_opener). Others via a small adapter; ask.

Can the agent click the link inside the e-mail?

Yes: the message body is available to the step script, so a Playwright step can open the link and the next screen becomes a step of its own.

More guides

Accepting a task with an AI agent, in the pull requestRegression before a release: compare the run with the accepted baselineSEO and landing-page checks on production, read-only, every dayMobile apps: walk Android and iOS screens like web pages

Install once, then work through your agent

Kaloko runs where your code and your agent are. The service stores and versions the results, shows the canvas and collects approvals.

  1. Add the CLI to the project
    npm install --save-dev kaloko

    Needs Node 20 or newer. Update later with npm update kaloko.

  2. Create the config and install the skill
    npx kaloko init --agent claude --org <your-org>

    The skill is copied to .claude/skills/kaloko. npx kaloko doctor checks Chrome, the config and the token.

  3. Create your organization and a token

    Create an organization; you become its admin. The start page offers a tester token in one click, later under Settings → API tokens. Put it into the project .env:

    KALOKO_TOKEN=qwk_…
    TYPESAFE_API_KEY=…   # optional: semantic evaluator

Then just ask your agent

The skill teaches your agent the whole loop: it writes the acceptance plan and the scenario from the task, walks the screens, evaluates, shares the canvas, reads what reviewers said and fixes it. You do not type the commands; you review the canvas.

What the agent runs (or run it yourself, e.g. in CI)

The same loop by hand:

npx kaloko start --scenario docs/tasks/TASK-123/qa/scenario.yml --env local
npx kaloko walk        # playwright steps; agent/manual steps: kaloko capture
npx kaloko evaluate
npx kaloko share --pr
npx kaloko feedback    # what reviewers said, with ids to answer