Kaloko

All guides › By agent

By agent

QA with Codex: the skill in skills/qawalk and a pointer in AGENTS.md

How Codex runs Kaloko: where the skill lives, how AGENTS.md points to it, the prompts that work, and how Codex reads reviewer feedback before it continues.

Codex reads AGENTS.md and follows it. kaloko init --agent codex puts the skill into skills/kaloko and adds one paragraph to AGENTS.md, so acceptance walkthroughs become part of how the agent works in the repository.

The problem

Codex is fast at code and silent about screens. It will happily merge a form that overflows on phones, because nothing asked it to look.

What changes with Kaloko

AGENTS.md says: for acceptance plans, scenario runs and the canvas use skills/kaloko/SKILL.md. The skill defines done, what is allowed without asking and what is not, and links the references for writing the plan, executing steps, reading evaluations and handling environments.

How it works

  1. Install

    npm install --save-dev kaloko, npx kaloko init --agent codex --org <your-org>, npx kaloko doctor. The skill lands in skills/kaloko, AGENTS.md gets the pointer, kaloko.config.yml is created.

  2. Ask

    the same prompts as for any agent; Codex runs the CLI commands from the skill in its sandbox with network access to your local app and to the Kaloko service.

  3. Steps Codex cannot script

    mark them agent in the scenario; Codex follows the instructions, then kaloko capture --step <id> records what it reached.

  4. Feedback

    kaloko feedback and kaloko inbox in the terminal, or the MCP server as an HTTP tool source, before the next iteration.

Skills and prompts

Follow skills/kaloko/SKILL.md: walk the acceptance plan of TASK-123 on local, evaluate, build and share it, then post the canvas link into the PR.
Turn this task description into ACCEPTANCE.md and qa/scenario.yml; one criterion per check, semantic questions only for judgement.

Expected outcome: the files in the task folder, a validated scenario (kaloko validate), a shared run and one PR comment.

What you get

FAQ

Sandbox and network?

The CLI talks to your local app, to the shared Chrome over CDP and to kaloko.app over HTTPS. Allow those in the Codex sandbox; nothing else is needed.

Can Codex and Claude Code share a repository?

Yes: kaloko init --agent all installs both locations; runs are pinned per agent with --run <id>.

What it looks like

A real run of Kaloko on its own public pages, refreshed daily. This is the canvas your team gets.

10 of 10 steps passed · 2026-09-29Open the run in Kaloko ↗

More guides

QA with Claude Code: install the skill, ask in plain words, read the canvasKaloko in CI: headless runs, read-only tokens, one comment per scenario

Install once, then work through your agent

Kaloko runs where your code and your agent are. The service stores and versions the results, shows the canvas and collects approvals.

  1. Add the CLI to the project
    npm install --save-dev kaloko

    Needs Node 20 or newer. Update later with npm update kaloko.

  2. Create the config and install the skill
    npx kaloko init --agent claude --org <your-org>

    The skill is copied to .claude/skills/kaloko. npx kaloko doctor checks Chrome, the config and the token.

  3. Create your organization and a token

    Create an organization; you become its admin. The start page offers a tester token in one click, later under Settings → API tokens. Put it into the project .env:

    KALOKO_TOKEN=qwk_…
TYPESAFE_API_KEY=…   # optional: semantic evaluator

Then just ask your agent

The skill teaches your agent the whole loop: it writes the acceptance plan and the scenario from the task, walks the screens, evaluates, shares the canvas, reads what reviewers said and fixes it. You do not type the commands; you review the canvas.

What the agent runs (or run it yourself, e.g. in CI)

The same loop by hand:

npx kaloko start --scenario docs/tasks/TASK-123/qa/scenario.yml --env local
npx kaloko walk        # playwright steps; agent/manual steps: kaloko capture
npx kaloko evaluate
npx kaloko share --pr
npx kaloko feedback    # what reviewers said, with ids to answer