DeepSeek’s open‑source coding agent, DeepSeek Harness, introduced a built‑in habit of shutting down background processes when a task completes. In the Terminal‑Bench benchmark this behavior caused the harness to fail three tasks that require long‑running processes, reducing its pass rate to two out of 36 attempts while competing agents succeeded on all 36. The resulting score penalty of roughly 1.5–1.7 percentage points pushed DeepSeek Harness from a statistical tie for first place across the benchmark suite down to sixth on the Terminal‑Bench slice.
Benchmark findings that matter
Researcher Joël Niklaus added DeepSeek Harness to a coding‑agent leaderboard that runs ten harnesses against the same underlying models (DeepSeek‑V4.1‑Flash, GLM‑5.3‑Flash, and Flint) with identical reasoning effort. The leaderboard ran 28,849 trials across nine harnesses. While DeepSeek Harness tied for first on four of the five scored evaluations, its shutdown behavior caused a noticeable dip on Terminal‑Bench.
Other harnesses – OpenCode, Pi, and Nous Research’s Hermes Agent – completed all 36 attempts for the same tasks, highlighting that the difference stemmed from orchestration rather than model capability.
Why harness behavior matters to engineers
For AI engineers, cloud/platform engineers, and SREs, the results illustrate a practical truth: the surrounding harness can dominate overall system performance and cost. In this case, a habit that developers normally view as good practice – cleaning up after a job – directly conflicted with the benchmark’s expectations.
Cost data from the same evaluation reinforce the point. Pi achieved a 49.4 % solve rate at $0.29 per solved task, whereas OpenCode cost $3.35 per solved task. Even when models and providers are held constant, harness choice can swing both effectiveness and expense.
Architectural and operational implications
- Job‑lifecycle design: DeepSeek’s documentation describes its local job registry as a deliberately short‑lived component, intended for jobs that should die with the harness process. Engineers must match job persistence requirements to the appropriate backend rather than relying on the default registry.
- Benchmark‑aligned testing: When evaluating new agents or harnesses, include scenarios that keep background processes alive. Automated test suites should cover both short‑lived and long‑running job patterns.
- Cost awareness: The same model can exhibit widely varying cost per solved task depending on harness implementation. Monitoring spend at the harness layer is essential for budgeting and scaling decisions.
- Operational observability: Shutting down processes silently can mask failures. Instrumentation that logs job termination events and correlates them with task outcomes helps detect unintended side effects early.
Related CloudNinjas coverage: AI engineering.
What This Means For Practitioners
When integrating an AI coding agent, treat the harness as a first‑class component. Verify that its job‑management policy aligns with the workload’s durability needs, and include explicit tests for long‑running background tasks. Track both success rates and per‑task cost at the harness level to avoid hidden inefficiencies. Finally, consider configuring or extending the harness to use a persistent backend for jobs that must survive process restarts, rather than relying on the default short‑lived registry.
