The June 2026 benchmark revealed a massive "startup token tax" – the fixed prompt and tool description payload each harness ships before any reasoning begins. That baseline ranged from roughly 700 tokens for Aider’s architect mode to about 26,000 tokens for OpenClaw, and because the same payload is resent on every turn, total token consumption per solved task varied by up to 70‑fold. Practitioners see the same pattern in later studies: higher prompt floors translate directly into higher per‑task costs, even when raw model performance is comparable.
Token overhead across harnesses
All three recent benchmarks (June token study, Composio’s workflow run, and Artificial Analysis index) measured the same phenomenon. The June suite ran twelve harnesses on identical Python tasks using DeepSeek V4 Flash and Nvidia’s Nemotron 3 Ultra. Solved‑task token counts spanned from ~3,500 (Aider architect) to ~292,000 (OpenClaw). The author linked this spread to the initial prompt size – 700 tokens versus 26,000 tokens – and to the number of interaction turns. With fifteen turns, OpenClaw’s 26,000‑token floor contributed roughly 390,000 input tokens solely to scaffolding, a figure that predicts total token use with an R² of 0.99.
Cache utilization and billing impact
Composio’s August run added a cost dimension. Across 30 enterprise workflows (240 runs, 129 successes), per‑success cost ranged from $0.028 (Pi Agent) to $0.195 (Claude Code). DeepAgents matched Claude Code’s pass rate while costing about a quarter as much. The study highlighted cache behavior: Claude Code drew only 1.5 % of its input from cache, whereas Codex and OMP leveraged 70 % and 57 % respectively. Fresh input is roughly five times more expensive than cached input, so Claude Code’s low cache share inflated its invoice despite similar raw token consumption.
The June benchmark observed a similar asymmetry. Codex consumed over a million tokens, but 77 % were cache reads billed at roughly one‑tenth the normal rate, making Codex cheaper per solved task than Claude Code, which used half the raw tokens but received almost no cache discount. The author attributed Claude Code’s cache deficit to its exclusive use of the Anthropic‑style messages endpoint on OpenRouter, which apparently reduces cache hits compared to the OpenAI‑style endpoint used by other harnesses.
Operational considerations for AI agents
From an engineering standpoint, two levers dominate cost and performance:
- Prompt floor size: Smaller system prompts and concise tool descriptions reduce the per‑turn token burden.
- Turn count: Fewer interaction cycles limit the number of times the floor is resent.
Secondary levers include:
- Choosing API endpoints that preserve cache eligibility.
- Designing harnesses to maximize cacheable fragments (e.g., reusing static library definitions).
- Monitoring token usage per turn to detect unexpected growth.
Related CloudNinjas coverage: AI engineering.
What This Means For Practitioners
When evaluating or building AI coding agents, start by measuring the harness’s initial token payload and the expected number of turns for typical tasks. Prefer harnesses with a low prompt floor and consider consolidating tool descriptions to avoid repeated large payloads. Verify which API endpoint your provider uses and test cache hit rates; a switch from an Anthropic‑style to an OpenAI‑style endpoint can cut token bills dramatically. Finally, instrument token consumption and cache statistics in your CI/CD pipelines so cost anomalies surface early, allowing you to iterate on harness design before they affect production budgets.