Postmortem: AI Incident Classifier Failed Due to Biased Training Data and Scikit-Learn 1.5

A production AI incident classifier misclassified 42% of critical security events as low priority for 72 hours in Q3 2024.

The failure was driven by biased training data and a Scikit-Learn 1.5 KMeans default change that destabilized the pipeline.

The incident caused $2.44M in direct losses, including SLA penalties and customer churn.

It highlights how data bias and unversioned library updates can combine to break ML systems in security and compliance workflows.