Skip to content

LLMs in search and recommendations

  • System design
  • Specialist topic or differentiator
  • About 2 h to learn
  • 3 vetted resources
Build a plan that includes this

What this topic covers

How LLMs fit into search and recommender systems without replacing the funnel that keeps them fast. Interviewers ask it because many teams are adding LLMs to ranking and want people who know where the cost goes. A strong answer keeps cheap retrieval and ranking stages and uses the LLM on a small candidate set, or offline. It explains semantic IDs (item IDs built from content, which help new and long-tail items), using LLMs to write metadata and label training data, and distilling a large model into a small one that can serve real-time traffic.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Draws the retrieval → ranking funnel and says where an LLM fits and why
  • Explains semantic IDs and why they help cold start
  • Gives two examples of LLM-generated metadata or labels improving a model
  • Names a way to keep LLM cost off the real-time path (offline labelling, distillation, caching)
  • Explains why the cross-encoder teacher can’t serve real-time traffic
  • Describes using the teacher to label far more data than humans could
  • Explains how generated image captions enrich item text
  • Explains why random hashed IDs hurt new and long-tail items
  • Describes how RQ-VAE turns a content embedding into a short code
  • Explains why dense content embeddings alone lost quality versus IDs

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    Improving Recommendation Systems & Search in the Age of LLMs (opens in a new tab)

    eugeneyan.com · Eugene Yan · 60 min · Free · checked 21 Oct 2026

    Why
    This is the best single map of how real companies are adding LLMs to search and recommendations. It gives you named examples to cite in a design round.
    What to read
    Read section 1 (LLM- and multimodal-augmented architectures, starting with YouTube’s semantic IDs) and section 2 (LLM-assisted data generation: Bing, Indeed, Yelp, Spotify, Amazon). Skim section 4 for LinkedIn’s and Netflix’s unified models.
    What to skip
    Skim section 3 (scaling laws, transfer learning, LoRAs) and skip the references.

  2. Free alternative

    Improving Pinterest Search Relevance Using Large Language Models (opens in a new tab)

    arXiv (Pinterest) · Han Wang et al. · 30 min · Free · checked 21 Oct 2026

    Why
    A short, concrete case of an LLM teacher in a production search system. It shows the pattern interviewers want: big model offline, small model online.
    What to read
    Read section 3 (the LLM relevance model, Pin text representations, and distilling into the student model) and sections 4.1.3 and 4.2.4 (online serving and scaling up labels through distillation).
    What to skip
    Skip related work and the model-by-model comparison tables in 4.2.2.

  3. Go deeper

    Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations (opens in a new tab)

    arXiv (Google) · Anima Singh et al. · 45 min · Free · checked 21 Oct 2026

    Why
    If you mention semantic IDs, expect to be asked how they’re built and why not just use the content embedding. This paper from YouTube answers both.
    What to read
    Read the introduction, section 3.2 (RQ-VAE for semantic IDs) and section 3.3 (how the IDs feed the ranking model), then the results in 4.2.
    What to skip
    Skip related work and most of the appendix, except A.2 on how stable the IDs are over time.

Companies that test System design most

Put LLMs in search and recommendations in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan