Skip to content

Design enterprise document search / a knowledge assistant

  • System design
  • Asked in most interview loops
  • About 2 h 30 min to learn
  • 3 vetted resources
Build a plan that includes this

What this topic covers

This is RAG over a company’s own data: wikis, drives, tickets, chat, and CRM, each pulled in through a connector. The question interviewers care most about is access control, because one leaked document ends the product. A strong answer mirrors each source’s permissions into the index and filters by the user’s identity at retrieval time, so the model never sees what the user can’t open. It also covers hybrid retrieval with reranking, citations the user can click to check, keeping the index fresh as documents and permissions change, and how you would evaluate answer quality and source precision.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Can explain why they paired lexical retrieval with reranking instead of pure vector search
  • Can describe the freshness trade-off between periodic syncs, webhooks, and re-indexing load
  • Can state a latency target and what it rules out
  • Can name evaluation metrics for answer quality and source precision and recall
  • Can explain why fine-tuning is a poor fit for permissions and fast-changing documents
  • Can describe a permission-aware crawler and index
  • Can explain why hybrid search with reranking beats vector search alone here
  • Can say how showing sources lets users check an answer
  • Can describe security trimming: store allowed users and groups per document, filter by caller identity
  • Can explain why filtering inside the search engine beats trimming results in the app

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    Building Dash: How RAG and AI agents help us meet the needs of businesses (opens in a new tab)

    Dropbox Tech Blog · Ranjitha Gurunath Kulkarni, James Johnson · 35 min · Free · checked 21 Oct 2026

    Why
    This is a real team explaining how they built a knowledge assistant over many business apps, with the trade-offs an interviewer will probe: retrieval choice, freshness, latency, and evaluation.
    What to read
    Read “Challenges in making an AI product that’s ready for businesses” (data diversity, fragmentation, modalities), then “Leveraging retrieval-augmented generation” including “Choosing the right retrieval system” and “Choosing the right model”, and “Lessons learned and future direction”.
    What to skip
    Skim “The role of AI agents” stages unless the interviewer asks about multi-step questions; the core case is retrieval.

  2. Free alternative

    How to build an AI assistant for the enterprise (opens in a new tab)

    Glean · Chau Tran, Mrinal Mohit, Arvind Jain · 20 min · Free · checked 21 Oct 2026

    Why
    It makes the central argument of this case in plain terms: fine-tuning on company data can’t respect permissions or keep up with change, so you need a permission-aware index feeding the model at query time.
    What to read
    Read “The challenges of fine-tuning LLMs on enterprise data” and all four parts of “Retrieval Augmented Generation: Vector search is not enough”, especially “Data: Scalable permission-aware indexing”.
    What to skip
    Skip the closing section; it’s a product pitch. Treat the connector counts as vendor claims.

  3. Go deeper

    Document-Level Access Control - Azure AI Search (opens in a new tab)

    Microsoft Learn · 30 min · Free · checked 21 Oct 2026

    Why
    When an interviewer asks “how exactly do you stop a user seeing a document they shouldn’t?”, this page gives you the concrete mechanics: permission metadata in the index and filtering by the caller’s identity at query time.
    What to read
    Read “Approaches for document-level access control”, “Choose an approach”, “Pattern for security trimming using filters”, and “Enforce document-level permissions at query time”. Note what it says about permission changes only showing up after sync.
    What to skip
    Skip the Purview sensitivity label and SharePoint setup details and the preview API version lists; the pattern matters, not the Azure specifics.

Companies that test System design most

Put Design enterprise document search / a knowledge assistant in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan