ChipSight runs a computer vision model to classify memory chips on a factory inspection line as pass or defect from top-down package images. The model performed well at launch, but over the last 10 weeks the manufacturer introduced 3 new chip package variants, and QA suspects the model is missing more defects on these newer designs.
| Metric | Launch Baseline | Last 30 Days | New Chip Variants Only |
|---|---|---|---|
| Accuracy | 0.962 | 0.918 | 0.861 |
| Precision | 0.903 | 0.887 | 0.812 |
| Recall | 0.884 | 0.741 | 0.612 |
| F1 Score | 0.893 | 0.807 | 0.697 |
| False Negative Rate | 0.116 | 0.259 | 0.388 |
| Defect prevalence | 8.1% | 8.7% | 9.4% |
| Manual review rate | 6.5% | 9.8% | 14.2% |
Operations needs a production monitoring plan that can detect when model performance degrades as new memory chip types appear, before defect escape rates materially increase. You have delayed labels from downstream electrical testing, image embeddings, chip metadata (package type, supplier, line, week), and model confidence scores.