ML pipelines, feature stores and reproducibility
- ML in production
- Asked often
- About 2 h 10 min to learn
- 3 vetted resources
What this topic covers
What sits around the model: data pipelines that build features, training pipelines that retrain and validate on a schedule or a trigger, and the metadata that records which data, code and settings produced each model. A feature store keeps one definition per feature, serves it from an offline store for training and a low-latency online store for inference, and joins features to labels as of each example’s timestamp so no future data leaks in. Interviewers check that you can explain point-in-time correctness with an example, say how a feature store reduces training–serving skew, describe what an experiment tracker records, and make a training run reproducible months later. Knowing the classic sources of ML technical debt, such as pipeline jungles and entangled features, helps in senior rounds.
Builds on
You’re ready when you can
What interviewers check, from the resources below.
- Describe MLOps levels 0, 1 and 2 and what changes at each
- Explain continuous training and name three triggers for a pipeline run
- Say what data validation and model validation each check before a model is promoted
- List what a metadata store records so any model can be traced and reproduced
- Explain offline versus online feature stores and how values get from one to the other
- Describe point-in-time correctness with an example of the leak it prevents
- Say how one shared feature definition reduces training–serving skew
- Argue when a team does not need a feature store yet
- Explain “changing anything changes everything” with an example of entangled features
- Describe a pipeline jungle and how a shared feature pipeline or feature store prevents it
ML pipelines, feature stores and reproducibility interview questions
Answer out loud, then open the points a strong answer covers. All Machine learning interview questions
What is point-in-time correctness in a feature store, and what goes wrong without it?
- Each training example must use feature values as they were known at that example’s timestamp, so labels are joined to features with an as-of join on time, not to the latest values.
- Joining on the latest values leaks future information into training, so offline metrics look strong while the model disappoints in production, where only past data is available.
- A feature store keeps timestamped history in an offline store for training and the latest values in a low-latency online store, both computed from one definition to avoid skew.
How would you make a training run reproducible six months from now?
- Version the code, the configuration and the exact data snapshot or query with its time range, since rerunning a query against a changed table silently trains on different data.
- Pin the environment with locked dependency versions or a container image and fix random seeds, accepting that GPU nondeterminism can still cause small differences between runs.
- Log each run’s parameters, data version, metrics and model artefact in an experiment tracker or metadata store, so every served model can be traced back to what produced it.
What to study, in order
Each link opened and checked by hand; the date says when.
Start here
MLOps: Continuous delivery and automation pipelines in machine learning (opens in a new tab)
Google Cloud Architecture Center · Google Cloud · 45 min · Free · checked 26 Oct 2026
- Why
- Its three maturity levels give a ready structure for “how would you automate retraining?”, and it puts the feature store, metadata store and pipeline triggers exactly where interviewers expect them.
- What to read
- Read “DevOps versus MLOps”, “Data science steps for ML”, “MLOps level 0: Manual process” (characteristics and challenges), all of “MLOps level 1: ML pipeline automation” including “Data and model validation”, “Feature store”, “Metadata management” and “ML pipeline triggers”, then “MLOps level 2: CI/CD pipeline automation”.
- What to skip
- “What’s next”, the contributors list and the links out to Vertex AI product pages.
Free alternative
Feature Stores: A Hierarchy of Needs (opens in a new tab)
eugeneyan.com · Eugene Yan · 40 min · Free · checked 26 Oct 2026
- Why
- Interviewers ask what a feature store is actually for, and this post answers from the needs upwards (reuse, serving, integrity, convenience, automation) with examples from Uber, Netflix, Airbnb, GoJek and others.
- What to read
- Read “The feature store hierarchy of needs”, then “Access”, “Serving” and “Integrity” closely (offline and online stores, train-serve skew, and Netflix’s time travel for point-in-time correctness), and skim “Convenience”, “Autopilot” (backfills) and “Conclusion: It depends on what you need”.
- What to skip
- The references list and the names of each company’s internal tools.
Go deeper
Hidden Technical Debt in Machine Learning Systems (opens in a new tab)
NeurIPS 2015 (Google) · D. Sculley et al. · 45 min · Free · checked 26 Oct 2026
- Why
- It is the standard reference for why ML systems cost so much to maintain, and its terms (entanglement, pipeline jungles, undeclared consumers, configuration debt) come up in senior design and production rounds.
- What to read
- Open the paper PDF from this page and read “1 Introduction”, “2 Complex Models Erode Boundaries”, “3 Data Dependencies Cost More than Code Dependencies”, “4 Feedback Loops”, “5 ML-System Anti-Patterns” (glue code, pipeline jungles, dead experimental codepaths), “6 Configuration Debt”, “7 Dealing with Changes in the External World” and the short “9 Conclusions: Measuring Debt and Paying it Off”.
- What to skip
- “8 Other Areas of ML-related Debt” except its paragraph on reproducibility debt.
Put ML pipelines, feature stores and reproducibility in your plan
Tell us your interview date and hours a day; it lands on the right day, in the right order.