Skip to main content
Oqoqo records quantitative performance data for every trial automatically. These metrics let you go beyond pass/fail and understand how agents are completing tasks — how many steps they needed, how much it cost, how long it took, and which tools they leaned on most. When you compare across treatments, agents, or versions, metrics are what tell you whether a change is actually an improvement.

​
What metrics are captured

MetricDescriptionExample
StepsTotal number of agent steps in the trial218
TokensTotal tokens consumed across the run8.4M
CostEstimated cost based on model pricing$5.25
DurationWall-clock time from start to finish4m 32s
Tool callsTotal number of tool invocations23
Tool call breakdownPercentage of calls by tool typebash 30.8%, browser.click 30.8%, fs.read 23.1%, search 15.3%
All metrics are captured per trial, so you can compare across parallel runs within an experiment as well as across separate experiments.
Metrics are captured for every trial, so you can compare across parallel runs as well as across experiments.

​
Comparing metrics

The Compare view shows metrics side by side across treatments, agents, or versions. Use it to answer questions like:
  • Which treatment uses fewer tokens to achieve the same pass rate?
  • Does a newer model version cost more or less than the baseline?
  • Which agent finishes the task fastest?
  • Does adding a skill reduce step count or increase it?
Sorting and filtering in the Compare view lets you isolate the dimensions that matter for your specific experiment.
The Compare view works across any combination of agents, treatments, and trials in your experiment. You’re not limited to pairwise comparisons.

​
What to look for

Metrics become most useful when you read them alongside rubric results. Patterns to watch for:
The agent is likely stuck in a loop — retrying the same action or exploring paths that don’t lead to a solution. Look at the trace to find where the loop starts, then check whether a friction point (a failed tool call, a confusing error) is causing the repetition.
The treatment is adding overhead without adding value. Compare the tool call breakdown to see if the agent is doing more search or context-loading than necessary. A high search percentage often indicates the agent is struggling to locate the right API surface.
The agent is spending a disproportionate amount of time trying to find the right method, endpoint, or schema. This is a signal that your agent-facing interface lacks discoverability — naming, examples, or introspection surfaces need attention.
A cheaper model is performing as well as an expensive one. This is the signal to swap the model on that workflow and redirect spend elsewhere.

​
Diffs

Alongside the quantitative metrics, Oqoqo captures the exact files changed by the agent during the trial as a diff. This gives you a ground-truth record of what the agent actually modified — useful for verifying correctness beyond what the rubric checks. 
auth.ts
+ cache.set(key, mod)
- return require(path)
+ return require(resolve(path))
Diffs appear in the trial output alongside answer.md. You can review them directly in the UI or access them via the API.
When a trial passes the rubric but the diff looks wrong, read the reasoning hook entries in the trace. The agent may have solved the stated task in an unexpected way — the diff shows you exactly what changed and where.