Skip to content

Shipping LLM features: prompt and model versioning, rollouts and migrations

  • System design
  • Asked often
  • About 1 h 30 min to learn
  • 4 vetted resources
Build a plan that includes this

What this topic covers

In an LLM feature, prompts and model versions behave like code and config: changing either changes behaviour. So they need versions, review and a way to roll back. Interviewers ask how you would ship a prompt change or move to a new model without breaking users. A strong answer covers a prompt registry with environment labels, pinned model snapshots, and eval gates in CI before anything ships. It then runs new versions in shadow mode or as a small canary, and rolls back fast when metrics drop. It also plans for provider deprecations and knows when self-hosting is worth the operational cost.

Builds on

You’re ready when you can

What interviewers check, from the resources below.

  • Explain why you pin a model snapshot and what a shadow pipeline on the newer model buys you
  • Describe how you would test a prompt before and after a model migration
  • Give two reasons to self-host and two reasons not to
  • Name the trigger for re-running your assertion tests (any change to prompt, model or context)
  • Explain the difference between an immutable version and a movable label such as production
  • Describe a rollback as moving a label back to an earlier version
  • Explain why prompts are fetched at runtime, and how client-side caching avoids extra latency
  • Contrast example-based tests with auto-evaluator (property-style) tests
  • Explain how a test suite makes prompt refactoring safe
  • Say where these tests sit in a CI pipeline and what blocks a release

What to study, in order

Each link opened and checked by hand; the date says when.

  1. Start here

    What We’ve Learned From A Year of Building with LLMs (opens in a new tab)

    Applied LLMs · Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, Shreya Shankar · 50 min · Free · checked 21 Oct 2026

    Why
    Interviewers ask how you ship and maintain LLM features safely. This is the practitioner consensus on pinning models, migrating prompts, and when to build your own infrastructure.
    What to read
    Read 1.4.1 (assertion tests run on every pipeline change), 2.1.1 (dev–prod skew), 2.2.2 (migrating prompts across models), 2.2.3 (version and pin your models, plus a shadow pipeline on newer models) and 2.2.4. Then read 3.1.3 (inference APIs vs self-hosting) and 3.2.3 (build LLMOps for faster iteration).
    What to skip
    Skip the Product and Team & Roles sections (2.3, 2.4) and section 4 for this topic. The prompting and RAG tactics in section 1 are covered elsewhere.

  2. Free alternative

    Prompt Version Control (opens in a new tab)

    Langfuse · 20 min · Free · checked 21 Oct 2026

    Why
    “Where do prompts live and how do you roll one back?” is a common follow-up. This shows the standard registry model of versions plus environment labels.
    What to read
    Read “Versions & Labels”, “Fetching by Label or Version” (including how label resolution works) and “Operational workflows” (rollbacks, prompt diffs, protected labels).
    What to skip
    Skip the SDK snippets for languages you don’t use. Ignore the plan tiers mentioned under protected labels.

  3. Go deeper

    Engineering Practices for LLM Application Development (opens in a new tab)

    martinfowler.com · David Tan and Jessie Wang · 35 min · Free · checked 21 Oct 2026

    Why
    It shows what eval gates look like as ordinary software tests on a real project. That is the bridge interviewers want between “we have evals” and “we ship safely”.
    What to read
    Read “Testing our LLMs” (example-based tests, auto-evaluator tests, adversarial tests) and “Refactoring prompts to sustain the pace of delivery”. Then read “Engineering basics saves us time”.
    What to skip
    Skim “What were we building?”, “Ethical guidelines” and the acknowledgements.

  4. Go deeper

    Promptfoo: LLM evals & red teaming (opens in a new tab)

    GitHub · promptfoo · 45 min · Free · checked 22 Oct 2026

    Why
    Promptfoo is a common way teams gate prompt and model changes: a config of test cases and assertions run side by side across models, locally or in CI. It makes “how do you know a model swap didn’t regress?” concrete.
    What to read
    Follow the README “Quick Start”: promptfoo init --example getting-started, read examples/getting-started/promptfooconfig.yaml (prompts, providers, tests with contains and icontains assertions), then promptfoo eval and promptfoo view. Add one llm-rubric assertion and a second provider, then read the CI/CD integration doc linked from the README.
    What to skip
    Skip the red-teaming and code-scanning guides for now (they belong with security prep) and the Node.js package docs.

Companies that test System design most

Put Shipping LLM features in your plan

Tell us your interview date and hours a day; it lands on the right day, in the right order.

Build my plan