Skip to content

Embeddings and vector search

  • RAG
  • Asked in most interview loops
  • About 1 h 40 min to learn
  • 4 vetted resources
Build a plan that includes this

What this topic covers

An embedding model turns text (or images, or code) into a fixed-length vector so that similar meanings land close together. Search then becomes 'find the nearest vectors to the query vector', scored with cosine similarity or dot product. Exact search compares against every vector and stops scaling around millions of items, so production systems use approximate nearest neighbour indexes such as HNSW, trading a little recall for large speedups. Interviews ask you to pick a similarity metric, explain symmetric vs asymmetric search, describe how HNSW navigates its layered graph, and reason about the recall/latency/memory knobs in a RAG retriever.

Leads to

You’re ready when you can

What interviewers check, from the resources below.

  • Explain what an embedding vector represents and why its dimension is fixed
  • Compute cosine similarity between two small vectors and say why it ignores magnitude
  • Describe how related-content and semantic search work using only embeddings
  • Explain where embeddings sit in a basic RAG pipeline
  • Explain symmetric vs asymmetric semantic search and give an example of each
  • Implement top-k search over a small corpus with cosine similarity
  • Explain why normalizing embeddings lets you use dot product instead of cosine
  • Describe the speed vs recall trade-off of approximate nearest neighbour search
  • Explain how a probability skip list inspires HNSW's layered structure
  • Describe a greedy search from the top layer down to layer 0

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    Embeddings: What they are and why they matter (opens in a new tab)

    simonwillison.net · Simon Willison · 40 min · Free · checked 21 Oct 2026

    Why
    Every RAG or search design question rests on embeddings; you need a clear, concrete answer to “what is an embedding and how does similarity search use it?”.
    What to read
    Read “What are embeddings?” through “Vibes-based search”, then “Answering questions with Retrieval-Augmented Generation”.
    What to skip
    “The 38 minute video version”, the Symbex, CLIP and Faucet Finder demos, and the clustering/PCA sections.

  2. Free alternative

    Semantic Search (Sentence Transformers docs) (opens in a new tab)

    Sentence Transformers documentation · Sentence Transformers maintainers · 35 min · Free · checked 21 Oct 2026

    Why
    A short, code-first walk from embedding a corpus to top-k search to ANN libraries, covering the symmetric vs asymmetric distinction that interviewers use to check you've built one.
    What to read
    Read “Background” through “Approximate Nearest Neighbor”, running the “Manual Implementation” snippet yourself.
    What to skip
    “Elasticsearch” and “OpenSearch” (product setup) and the “Examples” list; “Retrieve & Re-Rank” only points to another page.

  3. Go deeper

    Hierarchical Navigable Small Worlds (HNSW) (opens in a new tab)

    Pinecone Learn (Faiss series) · 40 min · Free · checked 21 Oct 2026

    Why
    HNSW is the default index in most vector databases, and “how does your vector DB actually find neighbours?” is a common follow-up in RAG system design.
    What to read
    Read “Foundations of HNSW” (skip lists and navigable small worlds) and “Graph Construction”, then “HNSW Performance” including “Improving Memory Usage and Search Speeds”.
    What to skip
    Most of “Implementation of HNSW”: skim the Faiss parameters and skip the embedded notebook output.

  4. Go deeper

    Dense Passage Retrieval for Open-Domain Question Answering (opens in a new tab)

    arXiv · Vladimir Karpukhin et al. · 35 min · Free · checked 22 Oct 2026

    Why
    DPR is the canonical dual-encoder retriever, and its training tricks, in-batch negatives and BM25 hard negatives, are what interviewers probe when you say “we fine-tuned our embedding model”.
    What to read
    Read §3.1 (two encoders, dot-product similarity) and §3.2 on training with in-batch negatives and hard negatives. Then read §5.1 (dense beats BM25 on top-20 accuracy) and the §5.2 ablation on negatives and training size, plus §5.4 on run-time cost.
    What to skip
    Skip §4 data preprocessing, §6 end-to-end QA, §7 related work and §5.3 beyond a skim.

Companies that test RAG most

Put Embeddings and vector search in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan