---
title: Evals
description: Run agents on repeatable tasks and compare their scores.
---

An evaluation runs an agent on tasks whose verifiers score the result. Evals is experimental and off by default. Enable `evals` under **Settings → Feature flags**, then open **Evals** in your project. Project file storage must be available.

## A first run

1. Run `beeuni evals init` in your linked project to create a Harbor-format example task.
2. Add the suite below to `beeuni.yaml`. Replace `worker` with an agent in your project.
3. Publish the project changes with `beeuni ship`, then start the suite.

```yaml
evals:
  smoke:
    tasks: evals/tasks
    agents: [worker]
```

```bash
beeuni evals init
beeuni evals run smoke --attempts 3 --wait
beeuni evals show <job-id>
beeuni evals compare <baseline-job-id> <new-job-id>
```

A single-step task has a `task.toml`, an `instruction.md`, and a verifier at `tests/test.sh`. For multi-step tasks, each step has its own instructions and verifier under `steps/<name>/`. The verifier is copied into the trial environment after the agent stops and writes `/logs/verifier/reward.txt` or `reward.json`.

Each trial creates its own session and sandbox. Trials consume credits like other sessions. Start with a small task set and few attempts.

All steps of a trial use the same workspace. Trials use the project workspace by default. To give each trial its own workspace, set `workspace = "fresh"` under `[metadata.beeuni]` in `task.toml`.

## Results and controls

Read the job and trial pages for scores, errors and verifier output. Result files are also available under `/Evals/<job>/` in **Files**. Compare runs with the same tasks and judge model before deciding a change improved an agent.

| Command | Effect |
| --- | --- |
| `beeuni evals cancel <job-id>` | Cancel the job and stop running trials. |

The SDK provides `planProjectEvalJob` to check a run without starting trials, `startProjectEvalJob` to start it, and `getProjectEvalJob` to read progress. Reuse an `idempotencyKey` for retries of the same start request. Run `beeuni evals --help` for datasets, trial actions and model-judge commands.
