Validate Batch Quality Prediction Model | Dataford Interview Questions

Your question is Validate Batch Quality Prediction Model. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Context

Avid Bioservices deployed a binary classification model to predict whether a manufacturing batch will fail final quality review before release. The model is used in the process development workflow to prioritize additional investigation on high-risk batches, but operations leadership is concerned that the current validation approach relies too heavily on overall accuracy.

Current Performance

Metric	Validation Set	Previous Baseline	Change
Accuracy	0.91	0.88	+0.03
Precision	0.64	0.55	+0.09
Recall	0.42	0.58	-0.16
F1 Score	0.51	0.56	-0.05
AUC-ROC	0.79	0.74	+0.05
Failed batch rate	0.12	0.12	0.00

The Problem

The model appears stronger on accuracy and AUC-ROC, but it is catching fewer truly failing batches than the previous baseline. In Avid Bioservices manufacturing, a missed failing batch can create release delays, rework, and client risk, while too many false alarms increase investigation workload.

Requirements

Explain how you would validate this model beyond a single metric like accuracy.
Interpret whether the current model is actually better than the baseline for this use case.
Use the metric table and confusion matrix to identify the main risk in deployment.
Recommend what additional validation checks you would run before approving the model.
Propose specific improvements to the model or thresholding strategy.

Constraints

Only 12% of batches fail, so the data is imbalanced.
Quality investigations are expensive, but missed failures are more costly than unnecessary reviews.
The model must be interpretable enough for manufacturing and quality stakeholders to trust its outputs.