Skip to content

Reasoning models and test-time compute

  • Inference
  • Asked often
  • About 2 h to learn
  • 6 vetted resources
Build a plan that includes this

What this topic covers

Reasoning models such as o1 and DeepSeek-R1 are trained to produce long chains of thought before answering, mostly through reinforcement learning on tasks with checkable answers, using algorithms like GRPO. Separately, you can spend more compute at inference time on any model with self-consistency (majority voting), best-of-N with a verifier, or search guided by a process reward model. Interviewers ask because choosing a reasoning model is now a routine product decision with real latency and cost consequences. A strong answer explains how R1-style training works, compares outcome and process reward models, and says when the extra tokens are worth paying for and when they are not.

Builds on

Leads to

You’re ready when you can

What interviewers check, from the resources below.

  • Explains R1-Zero (pure RL) versus R1 (SFT plus RL) and why the cold-start data helped
  • Describes verifiable rewards (accuracy and format) for math and code
  • Separates training-time methods from inference-time scaling
  • Gives two cases where a reasoning model is the wrong choice
  • Contrasts self-consistency, best-of-N and PRM-guided beam search
  • Explains process versus outcome reward models
  • Explains why the best method depends on problem difficulty and compute budget
  • Contrasts parallel sampling with sequential self-revision
  • Explains why chains of thought may not reflect the model’s real reasoning
  • Describes the risk of putting optimisation pressure on the chain of thought

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    Understanding Reasoning LLMs (opens in a new tab)

    Ahead of AI · Sebastian Raschka · 40 min · Free · checked 21 Oct 2026

    Why
    Interviewers expect you to explain how R1-style models are actually built and when you’d choose one, and this article lays out the four routes clearly.
    What to read
    Read “How do we define ‘reasoning model’?”, “When should we use reasoning models?”, “A brief look at the DeepSeek training pipeline”, and all four parts of “The 4 main ways to build and improve reasoning models” (inference-time scaling, pure RL, SFT + RL, pure SFT and distillation).
    What to skip
    Skim “Thoughts about DeepSeek R1” and “Developing reasoning models on a limited budget”; they are commentary and project notes.

  2. Free alternative

    Scaling test-time compute with open models (opens in a new tab)

    Hugging Face · Edward Beeching, Lewis Tunstall, Sasha Rush · 35 min · Free · checked 21 Oct 2026

    Why
    It shows the test-time methods side by side on the same task, so you can say with numbers when majority voting, best-of-N or verifier-guided search is worth it.
    What to read
    Read “Strategies for test-time compute scaling”, “Majority voting: a simple baseline”, “Beyond majority: Best-of-N”, “Beam search with process reward models”, “DVTS: boosting performance with diversity” and “The best of all worlds: compute-optimal scaling”.
    What to skip
    Skip “Experimental setup” beyond the model names, plus “Scaling up to larger models” and “Where to go from here?”.

  3. Go deeper

    Why We Think (opens in a new tab)

    Lil'Log · Lilian Weng · 75 min · Free · checked 21 Oct 2026

    Why
    A thorough survey of test-time compute that helps with follow-ups on faithfulness and on what happens when you optimise the chain of thought.
    What to read
    Read “Thinking in Tokens” in full: “Branching and Editing” (parallel sampling, sequential revision), “RL for Better Reasoning”, “External Tool Use” and “Thinking Faithfully”. Then read “Scaling Laws for Thinking Time”.
    What to skip
    Skip “Motivation” if you’re short on time, plus “Thinking in Continuous Space” and “Thinking as Latent Variables”, which cover research directions rarely asked about.

  4. Go deeper

    Let's Verify Step by Step (opens in a new tab)

    arXiv · Hunter Lightman et al. · 35 min · Free · checked 22 Oct 2026

    Why
    This is the source for process versus outcome reward models, a distinction interviewers raise when they ask how reasoning models are trained or how best-of-N reranking works.
    What to read
    Read §2.5 and §2.6 on outcome-supervised versus process-supervised reward models, §3 with Figure 3 (best-of-N on MATH with each), and §4.2 on active learning for labelling. Then read §6.1 on credit assignment.
    What to skip
    Skip §2.1–2.4 setup detail, §4.1 synthetic replication, §5 out-of-distribution results beyond the headline, §7 related work and the appendices.

  5. Go deeper

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (opens in a new tab)

    arXiv · DeepSeek-AI · 60 min · Free · checked 22 Oct 2026

    Why
    R1 showed that reinforcement learning with verifiable rewards alone can produce long chain-of-thought reasoning, and GRPO, R1-Zero and distillation into small models are now standard interview topics.
    What to read
    Read §2.1 (GRPO), §2.2 (rule-based accuracy and format rewards) and §2.3 on R1-Zero’s behaviour during training. Then §3 on the multi-stage R1 pipeline, and from the appendix A.3 (GRPO versus PPO), B.5 (reward hacking), F.1 (distillation versus RL) and G.2 (what didn’t work: PRMs and MCTS).
    What to skip
    Skip §4’s benchmark tables beyond the headline, §5 ethics, B.1 infrastructure, the D safety report and the per-category breakdowns in E.

  6. Go deeper

    Build A Reasoning Model (From Scratch) (opens in a new tab)

    GitHub · rasbt · 90 min · Free · checked 22 Oct 2026

    Why
    This repo implements the reasoning toolkit interviewers ask about, from verifiable evaluation and inference-time scaling to GRPO and distillation, in readable notebooks on a small Qwen3 model.
    What to read
    Run ch03/01_main-chapter-code/ch03_main.ipynb (verifying math answers) and ch04/01_main-chapter-code/ch04_main.ipynb (chain-of-thought prompting and self-consistency). Then read ch06/01_main-chapter-code/ch06_main.ipynb for RL with verifiable rewards via GRPO. Chapters 2 to 4 run on a CPU.
    What to skip
    Skip chapter 1 (no code), the setup tips, ch07’s advanced GRPO variants and ch08 distillation unless distillation is on your list. The companion book is paid but not needed for the code.

Companies that test Inference most

Put Reasoning models and test-time compute in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan