Skip to content

Design a ChatGPT-style assistant

  • System design
  • Asked often
  • About 2 h 30 min to learn
  • 3 vetted resources
Build a plan that includes this

What this topic covers

A general chat assistant looks simple but has to assemble the right context on every turn: the current conversation, saved facts about the user, summaries of past chats, uploaded files and images, and tool results. Interviewers use it to see whether you can reason about state, memory, and serving cost together. A strong answer explains how context is built and trimmed, how cross-session memory is stored and shown to the user with controls to view, delete, or turn it off, how streaming and moderation fit into the request path, and how prefix caching and model routing keep cost per daily active user in check.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Can list the layers of context assembled for each message
  • Can explain the difference between saved user facts and summaries of past chats
  • Can explain why summaries can be cheaper than RAG over full chat history, and what they lose
  • Can describe how the current session is trimmed when it gets long
  • Can explain how injected memory can change answers in ways the user didn’t expect
  • Can list user-visible controls: view, delete, archive, turn off, temporary chats
  • Can argue for scoping memory to a project
  • Can name the privacy risk of building a readable profile of the user
  • Can explain why a new chat turn shares a prefix with the previous one
  • Can describe how a radix tree lets sessions share a cached system prompt

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    I Reverse Engineered ChatGPT’s Memory System, and Here’s What I Found! (opens in a new tab)

    manthanguptaa.in · Manthan Gupta · 25 min · Free · checked 21 Oct 2026

    Why
    The heart of this case is how the assistant builds its context on each turn. This post lays out a plausible layered structure for a real product, which gives you a concrete design to present and critique.
    What to read
    Read all of it: “ChatGPT’s Context Structure”, “Session Metadata”, “User Memory”, “Recent Conversations Summary”, “Current Session Messages”, and “How It All Works Together”.
    What to skip
    Nothing major to skip. Keep in mind it’s inferred from experiments, not official documentation, so present it as one possible design.

  2. Free alternative

    I really don’t like ChatGPT’s new memory dossier (opens in a new tab)

    Simon Willison’s Weblog · Simon Willison · 15 min · Free · checked 21 Oct 2026

    Why
    Memory is a product and privacy question, not only a storage one. This post shows how cross-session memory can surprise users, which is exactly the trade-off an interviewer wants you to raise.
    What to read
    Read “We’re losing control of the context”, “How this actually works”, “Opting out”, and “There’s a version of this feature I would really like”.
    What to skip
    Skim the image-generation and research anecdotes at the top once you’ve got the point.

  3. Go deeper

    Fast and Expressive LLM Inference with RadixAttention and SGLang (opens in a new tab)

    LMSYS · Lianmin Zheng, Ying Sheng, et al. · 30 min · Free · checked 21 Oct 2026

    Why
    Multi-turn chat resends the whole conversation every turn, so reusing the KV cache across turns is one of the biggest levers on serving cost per user. This post explains how that reuse works.
    What to read
    Read “Backend: Automatic KV Cache Reuse with RadixAttention”, including the multi-turn chat walkthrough and the eviction policy, then skim “Benchmark”.
    What to skip
    Skip “Frontend: Easy LLM Programming with SGLang”, “Adoption”, and the citation sections.

Companies that test System design most

Put Design a ChatGPT-style assistant in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan