New Call for Open Problems at the ICML 2020 Workshop on Deploying and Monitoring Machine Learning Systems

Workshop on "Challenges in Deploying and Monitoring Machine LearningSystems" at ICML 2020, on the 17th of July. Virtual event.

Important dates:
  • Paper submission deadline EXTENDED: 17th of June, 2020
  • Open problems submission deadline: 30th of June 2020 


We invite you to submit open challenges in deploying and monitoring ML systems which you feel are only partially or are wholly unsolved at the present time. These problems might be algorithmic, computational, programmatic or hardware-based in nature. However, in all cases, they should represent a significant blocker in the continual use of a machine learning system.

High-level topics submitted problems might relate to including:
  • Updating machine learning systems, or managing the life-cycle of machine learning systems, whether manually or automatically (potentially using machine learning itself)
  • Machine learning systems for resource-limited hardware, e.g. industrial devices or low memory/low power devices 
  • Detecting, and responding to changes in the data, e.g. data drift, or adversarial attacks
  • Detecting and responding to unexpected events, e.g. oddball or missing data
  • Ensuring an ML system is and continues to be compliant with any existing regulations, e.g. with respect to regulations about data privacy
  • Measuring the impact of deploying an ML system, e.g. how to measure the carbon-footprint of a deployed ML system

If possible, in addition to describing the problem, you should give examples where your submitted problem has proved a blocker to success deployment or monitoring of an ML system.

After the submission deadline, we will collate all submissions, and publish a register of problems on the workshop’s website. We will then invite the authors of the submissions representing the most important and challenging problems to participate in a live broadcast panel discussion during the workshop.

--> SUBMIT HERE by the 30th of June <--



We invite submissions of full papers on machine learning systems, deployment and testing, including:
  • MLOps for deployed ML systems
  • ethics around deploying ML systems; 
  • fairness, trust and transparency of ML systems; 
  • providing privacy and security on ML Systems;
  • useful tools and programming languages for deploying ML systems;
  • deploying reinforcement learning in ML systems
  • performing continual learning and providing continual delivery in ML systems;
  • data challenges for deployed ML systems

All papers submissions will be made in PDF format, with a limit of four pages, including figures and tables, excluding references and appendices.

Formatting instructions are provided in the ICML website. The reviewing process will be blind and the workshop allows re-submissions of already published work and double submission.

--> SUBMIT HERE by the 17th of June <--

If you are interested in contributing to the workshop as a reviewer please complete this form.
Invited speakers:
Ernest Mwebaze, Makerere University and Google
Nevena Lalic, Amazon
Leilani Gilpin, MIT
Mohammad Ghavamzadeh, Inria and Facebook AI
- Thomas Gunter, Apple

Organization committee:
Alessandra Tosi, Mind Foundry
Nathan Korda, Mind Foundry
Neil D Lawrence, University of Cambridge