Skip to content

Design an agent infrastructure platform

  • System design
  • Specialist topic or differentiator
  • About 2 h 30 min to learn
  • 4 vetted resources
Build a plan that includes this

What this topic covers

You design a platform that runs agents other people wrote, at scale and safely. Interviewers ask it to see whether you treat model-generated code as untrusted and plan for failure. A strong answer puts tool execution in a strong sandbox (a microVM such as Firecracker, or gVisor), cuts off network paths that could leak data, and keeps credentials out of the sandbox behind a proxy. It isolates tenants, stores session state outside the worker so a crashed run can resume, records traces you can replay, and adds rate limits, retries with backoff for tool calls, and hard spend caps.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Explains why one container per session is fragile
  • Designs a durable session log so a run can resume after a crash
  • Keeps credentials out of the sandbox and explains why scoping alone isn’t enough
  • Treats the sandbox as a replaceable tool that can fail and be reprovisioned
  • Adds per-tenant limits and spend caps on top of the design
  • Explains why filesystem isolation without network isolation still leaks data, and the reverse
  • Describes routing all egress through a proxy with an allowlist
  • Designs a scoped-credential proxy so the real token never enters the sandbox
  • Compares containers, gVisor-style user-space kernels and microVMs on isolation and overhead
  • Quotes rough boot time and memory overhead for a microVM

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    Scaling Managed Agents: Decoupling the brain from the hands (opens in a new tab)

    Anthropic · Lance Martin, Gabe Cemaj, Michael Cohen · 30 min · Free · checked 21 Oct 2026

    Why
    This is a real write-up of a platform that runs agents at scale. It shows the split between session log, harness and sandbox that a strong design answer needs.
    What to read
    Read all of it: “Don’t adopt a pet”, “Decouple the brain from the hands”, “The session is not Claude’s context window”, and “Many brains, many hands”. Note how a crashed harness resumes from the log, and how credentials stay in a vault behind a proxy.
    What to skip
    Skim the conclusion. Then add what the post leaves out yourself: tenant isolation, rate limits and retries for tool calls, and spend caps.

  2. Free alternative

    Beyond permission prompts: making Claude Code more secure and autonomous (opens in a new tab)

    Anthropic · David Dworken, Oliver Weller-Davies · 15 min · Free · checked 21 Oct 2026

    Why
    Interviewers will ask how you stop an agent that has been prompt-injected from leaking data. This post explains why you need both filesystem and network isolation, with a concrete design.
    What to read
    Read “Sandboxing: a safer and more autonomous approach” and both parts of “Two new sandboxing features”: the sandboxed bash tool and Claude Code on the web. Focus on the egress proxy and the git proxy with a scoped credential.
    What to skip
    Skip “Getting started”; it is setup instructions.

  3. Go deeper

    Firecracker: Lightweight Virtualization for Serverless Applications (opens in a new tab)

    USENIX NSDI ’20 · Alexandru Agache et al. · 60 min · Free · checked 21 Oct 2026

    Why
    When you say “run each agent in a microVM”, expect a follow-up on why not containers and what it costs. This paper from the AWS Lambda team gives you the answer with numbers.
    What to read
    Read section 2 (choosing an isolation solution), section 3.3 and 3.4 (rate limiters and security), section 4.2 (the role of multi-tenancy), and section 5.1 and 5.2 (boot time and memory overhead).
    What to skip
    Skip the device model and API details in 3.1 and 3.2, and the IO benchmarks in 5.3.

  4. Go deeper

    LangGraph (opens in a new tab)

    GitHub · langchain-ai · 60 min · Free · checked 22 Oct 2026

    Why
    LangGraph is the most widely deployed low-level agent orchestration library, and its checkpointing, interrupts and durable execution are the pieces an agent-platform design has to provide in some form.
    What to read
    Read the README’s “Why use LangGraph?” and then the docs pages it links on durable execution and interrupts. Build the docs Quickstart agent. In the repo, read libs/langgraph/langgraph/graph/state.py (StateGraph, add_node, add_conditional_edges, compile with a checkpointer), libs/langgraph/langgraph/types.py (interrupt, Command, Send) and libs/checkpoint/langgraph/checkpoint/base/__init__.py for what a checkpoint stores.
    What to skip
    Skip the LangSmith and Deep Agents sections, the JS port and the deployment platform docs.

Companies that test System design most

Put Design an agent infrastructure platform in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan