Continuous evaluation and improvement of Deployed Models in Production – Francesc Joan Riera, The LEGO Group


This talk will introduce how we have setup a “frameless server” to train, maintain and update our deployed Machine Learning models for LEGO’s Moderation Service. Together with known tools for metrics and error logging, we now have added a Model Store, where all our models (deployed, in research, and failed) are stored, can be compared based on our set of standardized metrics, and deployed in a one-liner, independently of the ML framework used.

Key Takeaways

  • Learn and discover how we use a frameless ML toolkit for training, maintaining and updating our deployed ML models in the cloud
  • Visualize how our ML models have been evolving through time, based on this new frameless server
  • Discuss “what makes ML model X better than model Y, based on the logged metrics and parameters, that we specifically chose to keep in the Model Store
  • Key takeaway: get rid of specific Machine Learning libraries dependencies and allow your developers and researchers to train and test their models using their favorite library.

Add comment