RAG vs fine-tuning vs prompting: choosing the approach
- System design
- Asked in most interview loops
- About 60 min to learn
- 3 vetted resources
What this topic covers
Most “should we fine-tune?” questions come down to what is missing. If the model lacks facts, or the facts change, retrieval is the fix. If it knows enough but behaves wrongly (format, tone, a niche syntax), fine-tuning is the fix. Interviewers ask this to see whether you escalate in a sensible order: prompt first with an eval set, then add retrieval, and fine-tune only once you’ve shown the cheaper options fall short. A strong answer weighs data volume, how fresh the data must be, the need for citations, latency and cost. It also notes that real systems often combine RAG and fine-tuning.
You’re ready when you can
What interviewers check, from the resources below.
- Explain the difference between context optimisation (what the model knows) and LLM optimisation (how it behaves)
- Map a failure such as stale facts, wrong format or inconsistent tone to the right lever
- Justify starting with prompts and an eval set before adding RAG or fine-tuning
- Describe a case where RAG and fine-tuning are combined, and why
- State what fine-tuning is good at (style, format, instruction following) and bad at (new facts)
- Retell the character-renaming experiment as evidence
- Name at least two alternatives to fine-tuning for domain adaptation
- Describe one case where a niche syntax or strict output format justified fine-tuning
- Explain why fine-tuning needs a domain-specific eval system first
- Say why some product types (general assistants, developer tools) rarely benefit
What to study, in order
Each link opened and checked by hand; the date says when.
Start here
Optimizing LLM Accuracy (opens in a new tab)
OpenAI · 45 min · Free · checked 21 Oct 2026
- Why
- “Would you use RAG or fine-tuning here?” is a standard system-design question. This guide gives you a clear two-axis way to answer it, instead of just naming a preference.
- What to read
- Read “LLM optimization context”, “Prompt engineering”, “Evaluation” and “Understanding the tools” (the RAG, Fine-tuning and “All of the above” parts). Then read “How much accuracy is ‘good enough’ for production”.
- What to skip
- Skim the worked language-translation and language-correction examples. One pass is enough to see the method.
Free alternative
Fine tuning is for form, not facts (opens in a new tab)
Anyscale · Waleed Kadous and Kourosh Hakhamaneshi · 20 min · Free · checked 21 Oct 2026
- Why
- Its one-line rule, fine-tuning shapes form and retrieval supplies facts, is a memorable way to open your answer. The experiment behind it gives you evidence to cite.
- What to read
- Read the whole post, including “Looking Beyond Fine-Tuning”. Note the experiment where the model was fine-tuned to rename a character and mostly failed to learn it.
- What to skip
- “Next Steps” is a pointer to their other posts; skip it.
Go deeper
Is Fine-Tuning Still Valuable? (opens in a new tab)
hamel.dev · Hamel Husain · 20 min · Free · checked 21 Oct 2026
- Why
- Interviewers like a counterpoint to “never fine-tune”. This gives you two real production cases where fine-tuning was the right call.
- What to read
- Read the whole post, focusing on “Examples where I’ve seen fine-tuning work well” (the Honeycomb query assistant and ReChat’s Lucy).
- What to skip
- Skip the embedded social posts and the newsletter box.
Companies that test System design most
Put RAG vs fine-tuning vs prompting in your plan
Tell us your interview date and hours a day; it lands on the right day, in the right order.