Skip to main content
Custom agents are in private beta. Email support@oqoqo.ai to enable them for your organization.
Evaluate how accurately Ask Oqo answers questions about your Oqoqo projects by connecting oqoqo acp ask-oqo as a custom agent and grading its responses with tasks and rubrics.

Prerequisites

  • Ask Oqo works in the dashboard (has an Ask Oqo model assigned and you opened Ask Oqo once to initialize your chat connection).
  • Target project ID (oqoqo projects current or /projects/prj_… in the dashboard URL)—prefer a dedicated fixture project if tasks ask Ask Oqo to modify records, or set Max Concurrency to 1.
The personal API key acts as your user account in every run of this custom agent, including runs launched by teammates. Those runs can read and modify your projects and appear in your Ask Oqo chat history.

Steps

1

Create a personal API key

Because cloud runs cannot complete browser OAuth, open Profile → API keys and create a personal key to pass via OQOQO_API_KEY.
2

Get the Linux x64 CLI release

Inspect the latest.json manifest URL inside the https://oqoqo.ai/install shell script and copy the url and sha256 for linux-x64:
Open that latest.json URL to read the linux-x64 url and sha256 (pinned downloads do not auto-update; update these fields to test newer CLI builds).
3

Connect the custom agent

Under Manage → Agents, select Connect custom agent with:Under Environment variables, add OQOQO_API_KEY with your personal key, then select Connect agent.
4

Test the agent

Select Test agent so Oqoqo verifies startup and discovers Ask Oqo’s available models. Once Status reaches Ready, enable the models you want to compare under Models.
5

Write tasks about your Oqoqo projects

Because Ask Oqo calls Oqoqo APIs directly rather than reading files or tools inside the run container, write tasks asking about your project (such as “How many tasks does this project have?”) with rubrics grading the reply.
6

Launch the experiment

Open New experiment, select your tasks, choose Ask Oqo with the models to compare, keep Raw Agent (treatment tools do not reach Ask Oqo), and select Launch.

Interpret the results

Compare pass rates (Evals), Cost, Tokens, and Duration on Matrix across models to pick the best Ask Oqo model under Oqoqo AI defaults. If every model fails a task, inspect Traces to see which Oqoqo tools Ask Oqo called; startup failures appear under Custom agent in Frictions.

From the CLI and the plugin

Create the custom agent headlessly, open the returned setupUrl in the dashboard to enter OQOQO_API_KEY, and then run Test agent:
From the plugin, pass the same payload to manage_project_ai and run test_custom_agent.

Next steps

Custom agents

Connect any ACP agent from npm, Python, downloads, or the registry.

Ask Oqo

Configure Ask Oqo in the dashboard, Slack, and editors.

Use an API key

Authenticate the CLI with OQOQO_API_KEY.