> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Use the CLI

> Install the oqoqo CLI, sign in, choose a project, and run experiments from your terminal, scripts, and CI.

The `oqoqo` CLI can do everything the [Oqoqo plugin](https://docs.oqoqo.ai/operate/plugin.md) and [Ask Oqo](https://docs.oqoqo.ai/operate/ask-oqo.md) can, with stable JSON output for scripts, CI jobs, and AI assistants. You still read results and enter secrets in the dashboard. For every command and flag, see the [CLI reference](https://docs.oqoqo.ai/cli/overview.md).

## Install the CLI

```bash theme={null}
curl -fsSL https://oqoqo.ai/install | sh
```

The dashboard shows the same install and sign-in commands under **Automate → Oqoqo CLI**:

![Install the Oqoqo CLI page with the install command and the sign-in command.](https://mintcdn.com/oqoqo/YO1ysyUx4SPwBwoz/images/screenshots/install-cli-light.png?fit=max\&auto=format\&n=YO1ysyUx4SPwBwoz\&q=85\&s=a8d5c58209300f157139eaa3350f5abc)

## Sign in

```bash theme={null}
oqoqo login
```

[`oqoqo login`](https://docs.oqoqo.ai/cli/sign-in#oqoqo-login) opens your browser and prints a sign-in link and a confirmation code. Confirm the code, and the CLI saves your sign-in. To sign in on another device, pass `--no-browser` and open the link there. [`oqoqo whoami`](https://docs.oqoqo.ai/cli/sign-in#oqoqo-whoami) shows who you're signed in as, and [`oqoqo logout`](https://docs.oqoqo.ai/cli/sign-in#oqoqo-logout) removes the saved sign-in.

## Use an API key

Use a personal API key where nobody can sign in through a browser, such as in a CI job or on the machine that an agent runs in.

**1. Create a personal API key**

In the dashboard, open **Profile → API keys** and [create a key](https://docs.oqoqo.ai/setup/profile.md#api-keys).

**2. Set the key where the CLI runs**

Store the key as a CI secret exposed as `OQOQO_API_KEY`.

```bash theme={null}
export OQOQO_API_KEY="<your personal API key>"
oqoqo whoami
```

`oqoqo whoami` confirms that the CLI is using an API key.

While `OQOQO_API_KEY` is set:

* Every command uses the key, which acts as you in its organization, instead of your saved sign-in.
* `oqoqo login` doesn't sign in. Unset the variable to sign in with your browser.
* `oqoqo logout` removes only the saved sign-in, and commands keep using the key.
* The CLI doesn't update itself. Run `oqoqo upgrade` when you want a newer release.

An invalid or revoked key gives an auth error (exit code `3`) that points to the **API keys** page. Organization API keys can't use the CLI.

## Choose a project

Most commands work on one project. List your projects and save the one to use:

```bash theme={null}
oqoqo projects list
oqoqo projects use prj_2b7e90c4
oqoqo projects current
```

To create a project, give it a name and an attempt ID. Running the command again with the same attempt ID returns the same project.

```bash theme={null}
oqoqo projects create --name "Checkout agents" --mutation-attempt-id checkout-agents-1
```

To use a different project for one command, pass `--project-id`. It doesn't change the saved project. Learn more in [`oqoqo projects`](https://docs.oqoqo.ai/cli/projects).

## Run an experiment from the terminal

This workflow uses only CLI commands, so an AI assistant can run it too.

**1. Find your tasks, treatments, and agents**

```bash theme={null}
oqoqo catalog list --kind task,treatment,agent --output json
```

[`catalog list`](https://docs.oqoqo.ai/cli/catalog#oqoqo-catalog-list) returns each item's stable ID and its current version ID. A launch plan uses each task's version ID (`tskver_…`), not its task ID.

**2. Write a launch plan**

If you've already run an experiment, copy its plan and edit it:

```bash theme={null}
oqoqo experiments clone-plan exp_8f2c41d0 --output json | jq '.data.plan' > plan.json
```

Otherwise, start from the [example plan](https://docs.oqoqo.ai/cli/experiments.md#example) and fill in the IDs from the catalog.

The plan lists the task versions, treatments, agents, trials, and Evals and Insights settings. Give each new launch its own `launchAttemptId`, and reuse an ID only to retry the same launch. `oqoqo schema experiments launch` shows every field.

**3. Check the plan**

```bash theme={null}
oqoqo experiments preflight --plan-file plan.json
```

[Preflight](https://docs.oqoqo.ai/cli/experiments.md#oqoqo-experiments-preflight) changes nothing and uses no runs. If a tool or machine still needs a value, such as an API key, it lists the dashboard link where you enter it.

**4. Launch it**

```bash theme={null}
oqoqo experiments launch --plan-file plan.json
```

The [launch](https://docs.oqoqo.ai/cli/experiments.md#oqoqo-experiments-launch) uses runs from your run balance. The CLI prints the launch ID, the experiment ID once it exists, the dashboard link, and the command to watch it.

**5. Watch it finish**

```bash theme={null}
oqoqo experiments watch adm_5f1c9e20
```

[`watch`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-watch) takes the launch ID (`adm_…`) or the experiment ID (`exp_…`), waits until the experiment ends, and prints its progress counts. To check once without waiting, use [`oqoqo experiments status`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-status).

**6. Read the results**

```bash theme={null}
oqoqo results analyze summarize_scope --experiment-id exp_8f2c41d0
oqoqo results analyze compare_treatments --experiment-id exp_8f2c41d0
oqoqo runs list --experiment-id exp_8f2c41d0
oqoqo runs profile run_4c7e19a2
```

[`results analyze`](https://docs.oqoqo.ai/cli/runs#oqoqo-results-analyze) summarizes pass and fail counts and compares treatments, and `runs list` pages through every run. [`runs profile`](https://docs.oqoqo.ai/cli/runs#oqoqo-runs-profile) shows where one run's tool calls, time, and tokens went, and which treatment tools it used.

**7. Read the frictions**

```bash theme={null}
oqoqo frictions list --experiment-id exp_8f2c41d0 --fix-in product --recovery unrecovered
oqoqo frictions get frc_7d20e6b4
oqoqo runs list --friction-id frc_7d20e6b4
```

[`frictions list`](https://docs.oqoqo.ai/cli/frictions#oqoqo-frictions-list) ranks frictions by the number of runs in which the agent didn't recover. [`frictions get`](https://docs.oqoqo.ai/cli/frictions#oqoqo-frictions-get) reads one friction: what goes wrong, the suggested fix, where it happens most, and recent runs.

To run a finished experiment again, use [`oqoqo experiments rerun`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rerun). To run it on a schedule, use [`oqoqo schedules create`](https://docs.oqoqo.ai/cli/schedules#oqoqo-schedules-create) and [track the results over time](https://docs.oqoqo.ai/guides/track-agents-over-time.md).

## Run an experiment in CI

A CI job can launch an experiment, wait for it, and fail when the result falls below a threshold you choose. The CLI doesn't decide pass or fail. Your job reads the counts and applies the threshold.

**1. Set the API key and project**

Store a [personal API key](#use-an-api-key) as a CI secret named `OQOQO_API_KEY`. A key has no saved project, so pass `--project-id` on every command.

```bash theme={null}
P=prj_2b7e90c4
```

**2. Stop if the plan is blocked**

Keep a plan file in your repository.

```bash theme={null}
oqoqo experiments preflight --plan-file plan.json --project-id "$P" --require-ready --output json > preflight.json
```

Preflight exits with code `0` even when the plan is blocked. With `--require-ready`, it exits with code `6` instead, which fails the job. `preflight.json` still lists the `blockers` and `issues` to fix.

**3. Launch with a new attempt ID**

A plan in your repository goes out of date when agents, models, or providers change, and the launch then fails with `configuration_changed`. Preflight returns a current copy in `data.plan`, so launch that. Give each pipeline run its own `launchAttemptId`, or you get the first launch back instead of a new experiment.

```bash theme={null}
jq --arg id "ci-$CI_PIPELINE_ID" '.data.plan | .launchAttemptId = $id' preflight.json > launch.json
LAUNCH_ID=$(oqoqo experiments launch --plan-file launch.json --project-id "$P" --output json | jq -r '.data.launchId')
```

If the launch is refused, read `error.code`. For example, `quota_exceeded` means your run balance doesn't cover the plan, and a person must add runs.

**4. Wait for the experiment to end**

```bash theme={null}
oqoqo experiments watch "$LAUNCH_ID" --project-id "$P" --timeout-seconds 7200 \
  --fail-on completed_with_errors,cancelled,errored,admission_failed --output json > status.json
```

`watch` exits with code `0` for every end state, including errored and cancelled. With `--fail-on`, it exits with code `6` when the experiment ends in a state you list, which fails the job. `status.json` still has the result. `completed` means every run finished, so the example lists every other end state. If the timeout passes first, it exits with code `4` and `error.code` `watch_timeout`, and the experiment keeps running.

**5. Fail the job below your threshold**

```bash theme={null}
EXP_ID=$(jq -r '.data.experiment.experimentId' status.json)
oqoqo results analyze summarize_scope --experiment-id "$EXP_ID" --project-id "$P" --output json > summary.json
jq -e '.data.result.totals | (.passedRuns + .failedRuns) > 0 and (.passedRuns / (.passedRuns + .failedRuns)) >= 0.8' summary.json
```

This fails the job when no run has a verdict, or when fewer than 80% of the runs with a verdict passed. The result is in `data.result`:

* `totals` has `runs`, `passedRuns`, `failedRuns`, `evalsFailedRuns`, `infraBlockedRuns`, `inconclusiveRuns`, and `pendingRuns`.
* `treatments[]` has `treatmentKey`, `runCount`, `passedRuns`, and `failedRuns` for each treatment.

Compute a pass rate as `passedRuns / (passedRuns + failedRuns)`. Blocked and inconclusive runs, and runs whose Evals failed, have no verdict.

To compare a treatment with the baseline in a script, run [`oqoqo results analyze compare_treatments`](https://docs.oqoqo.ai/cli/runs#oqoqo-results-analyze). Each treatment has `passedRuns`, `failedRuns`, `failureRate`, and `deltaFromReference.failureRate`, its difference in failure rate from the baseline.

If the tasks and agents don't change, `oqoqo experiments rerun exp_8f2c41d0 --attempt-id "ci-$CI_PIPELINE_ID"` is a shorter way to launch the same experiment again (see [attempt IDs](https://docs.oqoqo.ai/cli/overview.md#retry-safely-with-attempt-ids)).

## Script the CLI

* **JSON output:** Pass `--output json` to any command except `oqoqo acp`, which writes only protocol messages. Stdout then has one envelope: `{"ok": true, "command": …, "data": …}` on success, or `{"ok": false, "command": …, "error": {"code": …, "message": …}}` on failure.
* **Exit codes:** Check the exit code, not the text: `0` success, `1` unexpected error, `2` usage error, `3` auth error, `4` API error, `5` network error, and `6` when the result matched `--fail-on` or `--require-ready`. See [Exit codes](https://docs.oqoqo.ai/cli/overview.md#exit-codes).
* **Command contracts:** `oqoqo schema` lists every command, and `oqoqo schema <command>` prints its flags and output as JSON Schema.
* **Confirmation:** A command that removes, cancels, disconnects, unpublishes, or replaces data asks you to confirm in a terminal. With `--output json`, or when there's no terminal, pass `--yes`. Without it, the command exits with code `2` and changes nothing.
* **Retries:** Commands that launch or create take an attempt ID. If you run one again with the same attempt ID, you get the same result instead of a second launch.
* **Secrets:** The CLI never takes a secret on the command line. It returns the dashboard link instead, and [`oqoqo setup links`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-setup-links) lists each machine and tool that still needs a value.

The [CLI reference](https://docs.oqoqo.ai/cli/overview.md) covers global flags, paging, and environment variables.

## Keep the CLI current

After a command, the CLI checks for a new release (at most once an hour) and installs it in the background, so your next command uses it. To update now, run:

```bash theme={null}
oqoqo upgrade
```

`oqoqo upgrade` doesn't need a sign-in. The CLI doesn't update itself in CI, while `OQOQO_API_KEY` is set, or when run from source. The reference explains how to [turn off background updates](https://docs.oqoqo.ai/cli/overview.md#keep-the-cli-current).

### When Oqoqo refuses an old release

When Oqoqo no longer accepts the installed release, commands that contact Oqoqo fail with exit code `4` and this message:

```text theme={null}
oqoqo: This oqoqo release is older than the server supports: run `oqoqo upgrade`, then retry.
```

With `--output json`, `error.code` is `client_update_required`. The CLI reference has more on [refused releases](https://docs.oqoqo.ai/cli/overview.md#when-oqoqo-refuses-an-old-release).

If you [run Ask Oqo as a custom agent](https://docs.oqoqo.ai/guides/ask-oqo-custom-agent.md) from a pinned CLI download, it doesn't pick up a newer release. Copy the `url` and `sha256` of a current release into the agent's **Download URL** and **SHA-256** fields, save the agent, and test it again.

## Next steps

> [card] [CLI reference](https://docs.oqoqo.ai/cli/overview.md)
>
> Look up every command, flag, and environment variable.

> [card] [Measure a tool's impact](https://docs.oqoqo.ai/guides/measure-tool-lift.md)
>
> Run a lift experiment end to end.

> [card] [Install the Oqoqo plugin](https://docs.oqoqo.ai/operate/plugin.md)
>
> Use the same capabilities inside your AI assistant.

> [card] [Launch an experiment](https://docs.oqoqo.ai/experiments/launch.md)
>
> Learn what goes into a launch plan.

> [card] [Find and fix frictions](https://docs.oqoqo.ai/guides/fix-frictions.md)
>
> Turn the frictions you read into fixes.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.