Skip to content

Design a coding copilot

  • System design
  • Asked often
  • About 2 h 30 min to learn
  • 3 vetted resources
Build a plan that includes this

What this topic covers

A coding copilot runs in two very different latency regimes. Inline completion has to show a suggestion within a few hundred milliseconds, so it relies on fast context ranking from open and nearby files, fill-in-the-middle prompts, a small model, and careful post-processing. Agent mode is slower and riskier: it indexes the repo, plans, runs tools and tests in a sandbox, and proposes a diff for a human to review. Interviewers want to hear you treat these as separate systems with separate budgets, and that you keep proprietary code private and limit what an agent can touch on disk and on the network.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Can lay out the stages: trigger, retrieve context, generate, post-process
  • Can explain how nearby-file context is ranked under a tight time budget
  • Can explain why fill-in-the-middle stops the model repeating code after the cursor
  • Can list three latency tricks, such as stop sequences, streaming with early stop, and request reuse
  • Can explain what a prompt library does when building a completion prompt
  • Can explain neighbouring tabs and why a low bar for including snippets helped
  • Can explain fill-in-the-middle and its measured effect
  • Can say how caching keeps added context from adding latency
  • Explains why filesystem isolation without network isolation still leaks data, and the reverse
  • Describes routing all egress through a proxy with an allowlist

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    The lifecycle of a code AI completion (opens in a new tab)

    Sourcegraph · Philipp Spiess · 35 min · Free · checked 21 Oct 2026

    Why
    This walks through the inline-completion pipeline stage by stage with real latency numbers, which is the half of the copilot case most candidates hand-wave.
    What to read
    Read “Code completions 101”, then “Planning”, “Retrieval”, “Generation”, “Post-processing”, and “Reliability”. Note the latency target, the fill-in-the-middle fix, and why they moved to a smaller code model.
    What to skip
    Skim “Data, data, data…” and the summary.

  2. Free alternative

    How GitHub Copilot is getting better at understanding your code (opens in a new tab)

    The GitHub Blog · Johan Rosenkilde · 15 min · Free · checked 21 Oct 2026

    Why
    A short account of how Copilot builds its prompt from your editor, with the A/B-tested gains from neighbouring tabs and fill-in-the-middle that you can quote in an interview.
    What to read
    Read “Why context matters”, “How GitHub Copilot understands your code”, and “Improving semantic understanding”.
    What to skip
    Skip “From OpenAI’s Codex model to GitHub Copilot”; it’s history.

  3. Go deeper

    Beyond permission prompts: making Claude Code more secure and autonomous (opens in a new tab)

    Anthropic · David Dworken, Oliver Weller-Davies · 15 min · Free · checked 21 Oct 2026

    Why
    Interviewers will ask how you stop an agent that has been prompt-injected from leaking data. This post explains why you need both filesystem and network isolation, with a concrete design.
    What to read
    Read “Sandboxing: a safer and more autonomous approach” and both parts of “Two new sandboxing features”: the sandboxed bash tool and Claude Code on the web. Focus on the egress proxy and the git proxy with a scoped credential.
    What to skip
    Skip “Getting started”; it is setup instructions.

Companies that test System design most

Put Design a coding copilot in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan