Kalokoby

All guides › Set up for the team

Set up for the team

Jira, Linear, Slack and Teams: returned steps where the team works

File a returned step as a Jira or Linear issue with its screenshot and notes, link runs to issues by key, and send each project's events to its own Slack or Teams channel.

A reviewer returns a step at four in the afternoon. By five the note has been copied into Jira without the screenshot, and the developer asks in Slack which screen it was about. Kaloko files the issue from the step itself and tells the right channel.

The problem

Review happens in one tool, planning in another and talk in a third. Every copy between them loses something: the screenshot, the criterion, the language the bug was in. Notifications go to one busy channel where nobody reads them.

What changes with Kaloko

An admin connects Jira Cloud or Linear and the chat tools once, in Settings → Integrations, and maps Kaloko projects to Jira projects or Linear teams. From then on an issue starts from the step and comes back to it when it is done, and each project reports to the channel its team actually reads. Both come with the Team plan and up.

How it works

  1. Connect the tracker

    in Settings → Integrations an admin connects Jira Cloud (OAuth or an API token) or Linear (OAuth or an API key) and maps each Kaloko project to a Jira project and issue type, or to a Linear team.

  2. File from the step

    on the canvas, in review mode or with kaloko issue create --step checkout --to jira --criterion AC2. The issue gets the screenshot, the criterion with its evidence, the reviewers' open notes and a link back to the step.

  3. Let it come back

    when the issue is done or cancelled in the tracker, the comment on the step gets a reply and is resolved; reopened, it opens again. For Jira the admin adds the webhook Kaloko shows; Kaloko also checks every 30 minutes.

  4. Link runs by key

    a branch or pull request title that names the issue (feat/WEB-12-gift-cards, Linear's eng-42) links the run to it, and the issue gets a comment when the run is shared.

  5. Pick channels per project

    add a Slack channel with *Add to Slack*, or a Teams channel through a Workflows URL. Each channel chooses its events (new run, returned, accepted, comment, mention, release recorded) and quiet hours; messages that arrive during quiet hours wait until they end.

Skills and prompts

File the returned checkout step of the latest TASK-123 run in Jira with the failing criterion, then tell me the issue key.

Expected outcome: kaloko issue create runs once for that step and the agent answers with the key and the link. Without a connected tracker it says an admin has to set it up.

List the open Kaloko issues for the checkout scenario.

Expected outcome: kaloko issue list --scenario checkout --state open with keys, steps and states.

What you get

FAQ

Do we need both Jira and Linear?

No. Connect the one you use; the CLI and the canvas offer only what is connected.

Who can file issues?

Developers and testers, on the canvas or with a token that may comment. Viewers see the issues on the steps.

Does the organization-wide Slack channel still work?

Yes. Channels per project come on top of it.

More guides

Company sign-in: SAML or OIDC, SCIM and the controls around themAccepting a task with an AI agent, in the pull requestRegression before a release: compare the run with the accepted baseline

Install once, then work through your agent

Kaloko runs where your code and your agent are. The service stores and versions the results, shows the canvas and collects approvals.

  1. Add the CLI to the project
    npm install --save-dev kaloko

    Needs Node 20 or newer. Update later with npm update kaloko.

  2. Create the config and install the skill
    npx kaloko init --agent claude --org <your-org>

    The skill is copied to .claude/skills/kaloko. npx kaloko doctor checks Chrome, the config and the token.

  3. Create your organization and a token

    Create an organization; you become its admin. The start page offers a tester token in one click, later under Settings → API tokens. Put it into the project .env:

    KALOKO_TOKEN=qwk_…
    TYPESAFE_API_KEY=…   # optional: semantic evaluator

Then just ask your agent

The skill teaches your agent the whole loop: it writes the acceptance plan and the scenario from the task, walks the screens, evaluates, shares the canvas, reads what reviewers said and fixes it. You don’t type the commands; you look at the canvas.

What the agent runs (or run it yourself, e.g. in CI)

The same loop by hand:

npx kaloko start --scenario docs/tasks/TASK-123/qa/scenario.yml --env local
npx kaloko walk        # playwright steps; agent/manual steps: kaloko capture
npx kaloko evaluate
npx kaloko share --pr
npx kaloko feedback    # what reviewers said, with ids to answer