It’s pretty rare for people to make explicit lists of open problems in their field. People instead spend most of their time staring at smaller, individual, paper-sized pieces.

Still, in 2006, the great researcher Tom Mitchell listed nine open problems in machine learning. I claim that four of them have since been solved completely (in the loose sense that we went from no working system to practical systems).



  1. Solved: Can unlabelled data be helpful for supervised learning? (Mikolov 2013, Devlin 2018, Radford 2019);


  1. Solved: How can we transfer what is learned for one task to improve learning in other related tasks? (Devlin 2018, Alammar, Radford 2019, Raffel 2020, Zhuang 2020);


  1. What is the relationship between different learning algorithms, and which should be used when? (some progress, but somewhat obviated by the dominance of the Transformer: Hu 2022, Tsai 2019);


  1. For learners that actively collect their own training data, what is the best strategy? (minor progress; Loshchilov 2016, Katharopoulos 2018, Jiang 2019, Mindermann 2022);


  1. Solved: To what degree can we have both data privacy and the benefits of data mining? (Bonawitz 2016, Kairouz 2021, McMahan 2022, Banse 2024);


  1. Can we build never-ending learners? (unsolved; Parisi 2019, Khetarpal 2022, Wang 2024);


  1. Can machine learning theories and algorithms help explain human learning? (minor progress; Shteingart 2014, Kudithipudi 2022, Parr 2021, Byrnes 2022, Levin 2024);


  1. Can we design programming languages containing machine learning primitives? (some progress; Weiss 2021, Pyro 2020);


  1. Solved: Will computer perception merge with machine learning? [i.e. multimodal systems] (Dosovitskiy 2021, Radford 2021, Radford 2022, Zia).



Bibliography