When Good Models Go Bad: The damage caused by wayward models and how to prevent it

Case Study

Delivering a bad model into production/serving is deceptively easy to do and can create significant and difficult-to-mitigate damage. Problems with models range from simple issues like incompatibility with the serving system to more subtle quality regressions. Using a hand analysis of approximately 100 incidents tracked over 10 years, we look carefully at cases where these models reached, or almost reached the serving system. We identify common causes and manifestations of these failures and provide some ideas for how to measure the potential damage of various failures. Most importantly, we propose a set of simple (and some more sophisticated) techniques for detecting the problems before they cause damage.

Session Speakers

Senior Director, ML SRE

Oops, please Login or Create Account to view On Demand

The good news is that it's both easy and free to register and get access.

Account Login

Create Account

Newsletter Consent(Required)
Terms and Privacy Consent
This field is for validation purposes and should be left unchanged.