> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom agents

> Add an Agent Client Protocol (ACP) agent to your organization, test it, and run it in experiments next to the built-in agents.

> [!NOTE]
>
> Custom agents are in private beta. Email [support@oqoqo.ai](mailto:support@oqoqo.ai) to enable them for your organization.

A **custom agent** lets you run your own agent, or one Oqoqo doesn't list, on your tasks next to the built-in agents. It's any agent that supports the [Agent Client Protocol](https://agentclientprotocol.com/) (ACP), the open standard editors use to talk to AI agents. Use one to:

* Evaluate an ACP agent that isn't built in.
* Test your own agent on your tasks before you publish it.
* Compare an agent's ACP version with the built-in agent.

Connecting a custom agent is optional step 4 of the [setup checklist](https://docs.oqoqo.ai/setup/organization.md#setup-checklist).

Oqoqo installs the agent in each run's machine, where it works with the task's files and services and the treatment's tools. It's an agent, not a treatment: you choose it at launch, give it [treatments](https://docs.oqoqo.ai/experiments/treatments.md), and read its results in the same places as a built-in agent.

> [!NOTE]
>
> An agent that runs its tools on its own computer, such as Ask Oqo, can't use the run's files, services, or tools, so give it tasks that don't need them ([Run Ask Oqo as a custom agent](https://docs.oqoqo.ai/guides/ask-oqo-custom-agent.md)).

## Who can see a custom agent

Your organization owns each custom agent, and every project lists it under **Manage → Agents** with a **Custom** tag. It's on in the project where you created it and off elsewhere until you turn it on. Every run uses the same environment variable values, in every project and for every member.

## Connect a custom agent

> [tab] Dashboard
>
> 1. Open **Manage → Agents** and select **Connect custom agent**.
> 2. Enter a unique name of 120 characters or fewer in **Name this agent**. Optional: select **Add a description**.
> 3. Under **Install**, choose **ACP Registry**, **npm package**, **Python package**, **Download**, or **Import JSON** (explained below).
> 4. Under **Environment variables**, select **Add variable** for each value the agent reads.
> 5. Select **Connect agent**, then [test the agent](#test-agent) on the page that opens.

> [tab] CLI
>
> Create the agent with its variable names. For example, from the ACP Registry:
>
> ```bash theme={null}
> oqoqo project ai apply --operation '{"action":"create_custom_agent","name":"Gemini CLI","source":{"kind":"registry","registryId":"gemini"},"env":["GEMINI_API_KEY"]}' --output json
> ```
>
> Or from an npm package:
>
> ```bash theme={null}
> oqoqo project ai apply --operation '{"action":"create_custom_agent","name":"My agent","source":{"kind":"manual","install":{"kind":"npm","package":"my-agent@1.4.2"},"args":"acp --verbose"},"env":["MY_AGENT_API_KEY"]}' --output json
> ```
>
> The result gives the agent key (`acp-…`), the agent page, and a dashboard link for each value to set ([`oqoqo project ai apply`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-apply)).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to connect the custom agent. It calls **Create custom agents and change agents, models, and Oqoqo AI settings** ([`manage_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#manage_project_ai)) and returns a dashboard link for each value to set.
>
> ```text theme={null}
> Add the Gemini CLI from the ACP Registry as a custom agent in Oqoqo.
> ```

The **Connect custom agent** form has the install choices, the ACP Registry search, the environment variables, and **Advanced**.

![Connect custom agent form with the name, the Install choices, the ACP Registry search list, and Environment variables.](https://mintcdn.com/oqoqo/YO1ysyUx4SPwBwoz/images/screenshots/custom-agent-new-light.png?fit=max\&auto=format\&n=YO1ysyUx4SPwBwoz\&q=85\&s=6c1447de6c27face4b5686e9637583e7)

### ACP Registry

Under **Agent**, search the registry by name, ID, or description. Each row shows the agent's version and description.

* Oqoqo pins the agent's current version when you choose it, and runs use that version until you change it. When the registry has a newer version, the agent page shows **Update to**. Select it, then select **Save changes**.
* The registry can pause an agent with a problem, and paused agents aren't listed. If you choose one from the CLI or the plugin, the command fails and shows the registry's reason. Choose another agent, or install it as a package.
* For agents that Oqoqo recognizes, it adds the variable names the agent usually reads, with empty values. Remove the ones you don't need. For example, **goose** adds `GOOSE_PROVIDER` and `GOOSE_MODEL`, with a note to set the API key of the provider `GOOSE_PROVIDER` names, such as `ANTHROPIC_API_KEY`.
* Without your own description, Oqoqo uses the registry's.
* An **Also built in** tag means Oqoqo has a built-in version. Add the registry version only to compare the two.

### npm package, Python package, or Download

Use these for your own agent or one that isn't in the registry. The CLI and the plugin call all three the `manual` source.

| Install | What you enter |
| - | - |
| **npm package** | The package with its exact version, such as `my-agent@1.4.2` |
| **Python package** | The package with its exact version, such as `my-agent==1.4.2` |
| **Download** | The **Download URL** (HTTPS) of a Linux x64 archive or binary, its **SHA-256**, and the **Command**: the file to run in the download |

Then enter the **Arguments**, separated by spaces, with quotes around any argument that has spaces.

* Packages need an exact version, which keeps results comparable. If the package or version doesn't exist, the install fails with the npm or uv error.
* Oqoqo checks a download against its SHA-256 (64 hexadecimal characters) before running it.
* A `.tar.gz`, `.tgz`, `.tar.bz2`, or `.zip` download is an archive: Oqoqo extracts it, and **Command** is the path to run inside it. Any other download is a binary that Oqoqo saves under the **Command** name and runs.

### Import JSON

Paste a standard ACP Registry `agent.json`, or select **Choose file**, to test an entry before you publish it.

* The entry needs an `id`, `name`, `version`, and a Linux x64 distribution: `npx`, `uvx`, or `binary` with a `linux-x86_64` target and its SHA-256.
* A `local` command can't run, because nothing is preinstalled in a run's machine.

## Set environment variables

An ACP agent can't sign in through a browser during a run, so give it its credentials, such as its model provider's API key, as environment variables (up to 64).

* Names use capital letters, digits, and underscores, and start with a letter. Don't use names Oqoqo sets for the run, such as `PATH`, `HOME`, and anything starting with `OQO_`.
* A saved name doesn't change. To rename a variable, remove it and add a new one.
* Values are secret, and you enter them only in the dashboard. The CLI and the plugin return a dashboard link for each value instead. Every run of the agent can read them, and Oqoqo removes them from each run's saved recorded work.

The ACP Registry doesn't say which variables an agent needs, so an empty list doesn't mean it needs none. Most agents need their model provider's API key. If the agent can't sign in, **Test agent** shows the sign-in methods it reports.

## Test agent

**Test agent** checks that the agent installs, starts, and answers. It uses one run (**Uses 1 run**) with:

* the **Agent check** task, which asks the agent for its name and what it can help with, in one sentence
* the **Agent default** model
* the **Raw Agent** treatment (no tools)
* Evals and Insights off

While it runs, the status shows **Testing**, and **Open test run** opens the run. The status updates when the run ends.

A test is a check, not an experiment: it sends no notice and doesn't show on Home or in the experiments list, though the CLI and the plugin list it. It has no project files, so if your agent needs them, run an experiment with a task that has them.

> [tab] Dashboard
>
> On the agent page, under **Status**, select **Test agent**.

> [tab] CLI
>
> ```bash theme={null}
> oqoqo project ai test-agent acp-0123456789abcdef
> oqoqo experiments watch <launch-id>
> ```
>
> The CLI prints an attempt ID. Pass it as `--attempt-id` to retry without a second run ([`oqoqo project ai test-agent`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-test-agent)).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to test the custom agent. It calls **Test a custom agent** ([`test_custom_agent`](https://docs.oqoqo.ai/mcp-server/tools#test_custom_agent)), reuses the same `launch_attempt_id` to retry without a second run, and polls **Get launch status** ([`get_launch_status`](https://docs.oqoqo.ai/mcp-server/tools#get_launch_status)). A test uses a run, so turn on tool approval in your AI assistant.

When Test agent can't start, the disabled button, the CLI, and the plugin say why:

| Message | What to do |
| - | - |
| Save or discard your changes first. | Save the agent page, or discard the changes. |
| A test with the current settings is running. Open its run. | Select **Open test run**. |
| Turn on this agent in this project. | On the agent page, turn on **In this project**. |
| Set *NAME*. | Enter each value under **Environment variables**. |
| Turn on Agent default. Test agent uses it. | Turn on **Agent default** under **Models**. |

### Statuses

The status shows on the agent page, **Manage → Agents**, and **New experiment**. When two apply, the higher one in this table wins.

| Status | Meaning |
| - | - |
| **Needs values** | An environment variable has no value. Set a value for each one. |
| **Testing** | A Test agent run with the current settings is in progress. |
| **Needs attention** | The agent did not answer its latest run with the current settings. |
| **Not tested** | No run with the current settings has finished. |
| **Ready** | The agent answered its latest run with the current settings. |

Any finished run with the current settings, in any project, updates the status, except a cancelled run or one where the agent refused the treatment's MCP servers. Saving a change to the settings or a value resets it to **Not tested**. **Open latest run** opens the run that set the status, and the CLI and the plugin also give the agent's reported version and the kinds of MCP server it accepts.

### When the agent needs attention

**Status** shows the step that failed, the agent's own error, and the fix, with a link such as **Go to Models** to the right section or the run. When the agent asks for a sign-in, the page lists its sign-in methods, which often name the variable it reads.

| What failed | Fix |
| - | - |
| The agent did not accept the run's MCP servers | Run this agent with a treatment that has no MCP servers. |
| The agent asks for a sign-in | Set the sign-in values this agent reads in its environment variables. Then test the agent again. |
| The agent did not accept the model | Use a model that the agent lists. Turn this model off in **Models**, or run with **Agent default**. |
| The download does not match its SHA-256 | Check that the download URL names the right file. Then enter that file's SHA-256. |
| An npm or Python package agent could not be installed | Check the package name and its exact version. The error shows the message from npm or uv. |
| An npm or Python package agent did not start | Check the package name and its exact version, and the arguments. The error shows the message from npm or uv. |
| The agent can't run on the machine's Node.js | [Change Node.js in the machine's setup](https://docs.oqoqo.ai/reference/troubleshooting.md#a-run-could-not-start-because-of-its-machine), or choose another agent. |
| A **Download** agent could not be installed or did not start | Check the download URL, its SHA-256, and the command. |
| An **ACP Registry** agent could not be installed | Test the agent again. If it fails again, change its install to its npm package, Python package, or download. |
| An **ACP Registry** agent did not start or connect | Read the last lines of the agent's log in the run. |
| The agent did not start a session or answer the prompt | Read the agent's error. If it asks you to sign in, set the sign-in values that this agent reads in its environment variables. Then test the agent again. |

## Confirm the agent's models

Oqoqo learns the agent's models from each run, so **Models** has no **Refresh models** button.

* **Agent default** is always first. Oqoqo doesn't choose a model, so the agent uses its own default. Some agents read the model from a variable, such as `GOOSE_MODEL` for goose.
* After a run, the list shows each model the agent offered. Turn on the ones you want to launch with. You can't add a model that the agent doesn't offer.

| Model state | Meaning |
| - | - |
| **Current** | A run with the current settings listed this model, or it is **Agent default**. |
| **Not confirmed** | The agent listed this model with older settings. Test the agent to confirm it. |
| **Unavailable** | The agent did not accept this model with the current settings. |

An agent with no model option has only **Agent default**. Custom agents have no effort setting.

## Run a custom agent in an experiment

**New experiment** lists custom agents after the built-in ones, with the **Custom** tag and, unless the agent is **Ready**, its status. Each model you turned on is a choice.

Selecting **New experiment** on the agent's page opens an experiment with this agent on **Agent default** beside the built-in agents, for comparison. To run only this agent, select **Clear** beside the agent search and select it again. Nothing runs until you select **Launch**.

* **New experiment** is the page's main action once the agent is **Ready**. Before that, **Test agent** is the main action.
* The page has no **New experiment** while the agent is off in the project (turn on **In this project**) or has no models turned on (follow **Go to Models**).
* An experiment that you open another way doesn't select a custom agent for you. When you select one, it starts on **Agent default**.

| Launch state | Cause | What to do |
| - | - | - |
| Blocked | A selected custom agent has a variable with no value | Set the value in the dashboard |
| Blocked | A treatment has an MCP server the agent can't accept | Remove the agent or the treatment |
| Allowed, with a notice | The agent is **Not tested** or **Needs attention** (for example, "My agent is not tested yet." with **Open agent**) | Test the agent before a full experiment |

A launch from the CLI or the plugin must name its agents: find the model IDs with `oqoqo catalog list --kind agent --projection full` (plugin: `inspect_catalog` with `kinds: ["agent"]`).

Before each run, Oqoqo copies the treatment's skills into the agent's skills folder, `~/.agents/skills` by default. If the agent uses another folder, set **Advanced → Skills folder** on the agent page.

## Read a custom agent's results

Results work as for a built-in agent, with these differences:

* Models show under the agent's own names, and **Agent default** adds the reported model, such as **Agent default: opus**.
* An answer that ends early shows a note, such as "Stopped early: the agent reached its token limit." Evals score it as usual.
* Tokens and cost show only when the agent reports them. Otherwise the **Runs** cost cell is blank, because Oqoqo never estimates a custom agent's cost.
* When the agent fails before it answers, the run status says "The agent failed before it answered." A banner shows the failed step, the agent's error, and the fix, with a link such as **Go to Models**. Evals don't score the run, and as an errored run it's left out of the pass rate and doesn't use your [run balance](https://docs.oqoqo.ai/setup/billing.md).

### Where a cost comes from

Every cost names its source, beside **Cost** on a run or on hover in the Runs list.

| Label | Meaning |
| - | - |
| **Reported by the agent** | The agent reported its own total. |
| **Estimated from list prices** | Oqoqo calculated the cost from token counts and list prices. |
| No label | The run's record does not say where the cost came from. |

A Matrix cell whose runs mix sources shows **Mixed sources**, with the mean cost per source rather than one combined value.

## Change or remove a custom agent

Change settings on the agent page and select **Save changes**. You can also change how it installs: another install choice, another ACP Registry agent, or a newer version. Past runs keep their launch settings, and the status resets to **Not tested**.

> [tab] Dashboard
>
> 1. Open the agent page.
> 2. Open the **⋯** menu and select **Remove**.
> 3. Select **Remove agent**.

> [tab] CLI
>
> ```bash theme={null}
> oqoqo project ai delete --operation '{"action":"remove_custom_agent","agentKey":"acp-0123456789abcdef","expectedRevision":4}'
> ```
>
> Pass the current `revision` from `oqoqo project ai inspect --view agents`. In a terminal, the CLI asks you to confirm. With `--output json` or outside a terminal, pass `--yes` ([`oqoqo project ai delete`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-delete)).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to remove the custom agent. It calls **Remove custom agents or provider connections** ([`delete_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#delete_project_ai)), and it asks you to approve the call.

Removing an agent takes it off the Agents list and out of new experiments, and Oqoqo clears its variable values. Past results keep its name. You can't remove an agent while one of its runs, a test or an experiment, is in progress, so wait for it to finish or cancel it first.

## From the CLI and the plugin

The CLI and the plugin offer the same operations, except that variable values are set in the dashboard: a create or update returns a link for each one.

| Task | CLI | Plugin (MCP) |
| - | - | - |
| List agents with each custom agent's status, variables, note about variables (`envNote`), and fix | [`oqoqo project ai inspect --view agents`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-inspect) | [`inspect_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#inspect_project_ai) with `view=agents` |
| Find an agent in the ACP Registry | [`oqoqo project ai inspect --view agent_registry --query gemini`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-inspect) | [`inspect_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#inspect_project_ai) with `view=agent_registry` |
| Create or change a custom agent | [`oqoqo project ai apply`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-apply) with `create_custom_agent` or `update_custom_agent` | [`manage_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#manage_project_ai) |
| Test a custom agent | [`oqoqo project ai test-agent`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-test-agent) | [`test_custom_agent`](https://docs.oqoqo.ai/mcp-server/tools#test_custom_agent) |
| List its models | [`oqoqo project ai inspect --view models --agent-key`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-inspect) | [`inspect_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#inspect_project_ai) with `view=models` and `agent_key` |
| Turn a model on or off | [`oqoqo project ai apply`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-apply) with `set_model_enabled` | [`manage_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#manage_project_ai) |
| Read a run's stop reason, reported model, and failure | [`oqoqo runs trajectory`](https://docs.oqoqo.ai/cli/runs#oqoqo-runs-trajectory), [`oqoqo runs profile`](https://docs.oqoqo.ai/cli/runs#oqoqo-runs-profile) | [`get_run_trajectory`](https://docs.oqoqo.ai/mcp-server/tools#get_run_trajectory), [`get_run_profile`](https://docs.oqoqo.ai/mcp-server/tools#get_run_profile) |
| Remove a custom agent | [`oqoqo project ai delete`](https://docs.oqoqo.ai/cli/agents-and-models#oqoqo-project-ai-delete) with `remove_custom_agent` | [`delete_project_ai`](https://docs.oqoqo.ai/mcp-server/tools#delete_project_ai) |

An update or removal needs the current `revision` from `oqoqo project ai inspect` as `expectedRevision`. If the revision is stale, the command changes nothing. To change how an agent installs, send `update_custom_agent` with a new `source`.

## Next steps

> [card] [Oqoqo AI features](https://docs.oqoqo.ai/setup/oqoqo-ai.md)
>
> Step 5: choose models for Evals, Insights, and Ask Oqo.

> [card] [Run Ask Oqo as a custom agent](https://docs.oqoqo.ai/guides/ask-oqo-custom-agent.md)
>
> Install Ask Oqo from a download and give it an API key.

> [card] [Compare agents and models](https://docs.oqoqo.ai/guides/compare-agents-and-models.md)
>
> Compare your custom agent with the built-in agents.

> [card] [Launch an experiment](https://docs.oqoqo.ai/experiments/launch.md)
>
> Choose your custom agent at launch.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.