Anthropic’s latest release, Claude Duet, extends the Claude Opus 5.5 model with two new operational pieces: an offline grader that scores how well the agent orders tasks, and a runtime classifier that validates each action against the user’s intent. Both were introduced after the public debut on October 21 and are now being exercised on internal workloads.
Task‑prioritization grading
Duet’s graders run offline against a curated set of internal use cases. They measure whether the agent correctly distinguishes urgent items—such as looming deadlines or reply‑needed emails—from work that can wait. Anthropic notes that urgency is inherently subjective, so the evaluation is “imperfect but quite helpful.” The graders are not exposed to external users yet, and the public launch does not disclose error rates.
Separate intent‑validation classifier
To mitigate the risk of an agent acting on hidden instructions, Duet splits the execution path. The model that performs a task is paired with an independent check built on the “auto mode classifier” from Claude Code, extended for Duet. This check runs alongside each action and decides if the result matches the user’s expressed intent. Because the classifier is isolated from the primary model, embedded prompt‑injection strings in emails, documents, or web pages cannot directly influence its decision.
Anthropic reports that an external firm performed penetration testing on Duet, but the public materials do not quantify how reliably the classifier blocks out‑of‑scope actions. Researchers have shown that removing prompt‑defining sections from Claude Code prompts increases out‑of‑scope behavior, suggesting the classifier’s effectiveness is a key variable.
Operational and cost considerations
Duet consumes the same token allowance as regular Claude conversations, meaning background checks count against a subscriber’s plan limits. The agent polls connected apps every few hours rather than continuously, a design choice that balances resource usage against the chance of missing time‑sensitive events. Practitioners should be aware that increasing poll frequency will raise token consumption, while decreasing it could leave urgent items unnoticed.
Connected‑app access is scoped to the user’s Claude account. Disconnecting an app stops new data pulls, but any information already cached in Duet’s memory persists until the user explicitly resets it. The system does not ingest passwords, browser cookies, or sign‑in tokens, and it cannot store them.
Security posture and user controls
Duet is designed to request explicit approval before performing high‑impact actions such as payments or deletions. Sharing sensitive health or financial data also requires user consent, though an explicit request can serve as that consent. Actions that involve trading securities, moving money between accounts, or accessing another user’s account are categorically disallowed.
Users can view completed and pending tasks via a task list, but they cannot watch the agent’s browser session in real time or undo actions after they are executed. The lack of a live inspection view means operators must rely on post‑action reviews and the separate intent check for safety guarantees.
Related CloudNinjas coverage: AI engineering.
What This Means For Practitioners
When building or adopting AI agents that act autonomously across SaaS tools, consider implementing a dedicated validation layer that runs independently of the generative model. Use offline grading datasets that reflect real‑world priority mixes, but treat the results as guidance rather than absolute correctness. Account for the token or compute cost of periodic polling, and expose clear user‑facing controls for approving high‑risk actions. Finally, verify that any external security assessment reports concrete detection rates for out‑of‑scope behavior before relying on the agent in production environments.