Managed Agents
Anthropic’s Managed Agents system is a hosted service designed to run long-horizon agents, focusing on interfaces that remain stable as underlying implementations change. This approach addresses the challenge of building systems for “programs as yet unthought of” by virtualizing agent components into stable abstractions.
Key Takeaways
- Decoupling Components: The system separates the LLM ‘brain’ (Claude and its harness) from the ‘hands’ (sandboxes and tools) and the ‘session’ log.
- Stable Interfaces: Key interfaces for session, harness, and sandbox allow implementations to be swapped without affecting the overall system.
- Virtualization Analogy: Similar to operating systems virtualizing hardware into processes and files, Managed Agents virtualizes agent components.
- Improved Scalability & Reliability: Decoupling allows components to fail or be replaced independently, reducing downtime and simplifying recovery.
- Enhanced Security: Separating the sandbox from credentials prevents prompt injection attacks from compromising sensitive tokens.
Main Concept
The core idea behind Managed Agents is to create a robust and scalable platform for AI agents by applying principles of abstraction and decoupling. By defining stable interfaces for the agent’s session (event log), harness (LLM interaction loop), and sandbox (execution environment), Anthropic ensures that the system can adapt to future improvements in LLMs and infrastructure without requiring fundamental architectural changes. This mirrors how operating systems provide stable abstractions over evolving hardware.
Practical Use
For DevSecOps Architects, Managed Agents offers a model for building secure and scalable AI-powered automation. The principles of decoupling the execution environment (sandbox) from the core logic (harness) and state management (session) are crucial for:
- Security: Preventing LLM-generated code from accessing sensitive credentials or infrastructure directly.
- Reliability: Ensuring that agent failures (e.g., sandbox crashes) do not lead to data loss or system instability.
- Scalability: Allowing individual components to be scaled or replaced independently to meet demand.
- Maintainability: Adapting to new LLM versions or execution environments without major refactoring.
Implementation Notes
- Pets vs. Cattle: The system evolved from a coupled design (a ‘pet’ container) to a decoupled one where components like sandboxes are treated as ‘cattle’ – interchangeable and easily replaceable.
- Security Boundary: Credentials are never directly exposed to the sandbox. Access tokens are either bundled with resources or stored in vaults, with access mediated by proxies or during sandbox initialization (e.g., Git tokens wired into remotes).
- Session as Durable Context: The session log acts as an append-only record of events, serving as durable context that lives outside the LLM’s context window. This allows agents to recover state and resume long-running tasks.
- Time-to-First-Token (TTFT) Improvement: Decoupling reduced TTFT by approximately 60% (p50) and over 90% (p95) by provisioning sandboxes only when needed.
- Many Brains, Many Hands: The architecture supports multiple LLM instances (brains) interacting with multiple execution environments (hands), enabling complex reasoning and task delegation.
Connected Notes
- Large Language Models - Managed Agents are built to leverage advancements in LLMs.
- Abstraction - The core principle of Managed Agents is virtualization and abstraction.
- Decoupling - The system’s success hinges on decoupling agent components.
- Prompt Injection - The architecture directly addresses security concerns related to prompt injection.
Questions
- How can the session log be effectively queried and transformed for different LLM context requirements?
- What are the specific security implications of using MCP tools and external service proxies?
- How does the system handle complex interdependencies between multiple ‘hands’ or sandboxes?