ED Mathématiques et Informatique
Applications of Large-Dimensional Random Matrix Theory and Free Probability in Statistical Learning by Neural Networks
by Issa-Mbenard DABO (IMB - Institut de Mathématiques de Bordeaux)
The defense will take place at 10h00 - Salle de conférence 351, cours de la Libération 33400 Talence
in front of the jury composed of
- Jeremie BIGOT - Professeur des universités - Université de Bordeaux - Directeur de these
- Jiangfeng YAO - Full professor - Chinese University of Hong Kong - Rapporteur
- Delphine FERAL - Maîtresse de conférences - Université de Bordeaux - Examinateur
- Jamal NAJIM - Directeur de recherche - Université Gustave Eiffel - Examinateur
- Camille MALE - Chargé de recherche - Université de Bordeaux - CoDirecteur de these
- Charles BORDENAVE - Directeur de recherche - Institut de Mathématiques de Marseille - Rapporteur
The functioning of machine learning algorithms relies heavily on the structure of the data they are given to study. Most research work in machine learning focuses on the study of homogeneous data, often modeled by independent and identically distributed random variables. However, data encountered in practice are often heterogeneous. In this thesis, we propose to consider heterogeneous data by endowing them with a variance profile. This notion, derived from random matrix theory, allows us in particular to study data arising from mixture models. We are particularly interested in the problem of ridge regression through two models: the linear ridge model and the random feature ridge model. In this thesis, we study the performance of these two models in the high-dimensional regime, i.e., when the size of the training sample and the dimension of the data tend to infinity at comparable rates. To this end, we propose asymptotic equivalents for the training error and the test error associated with the models of interest. The derivation of these equivalents relies heavily on spectral analysis from random matrix theory, free probability theory, and traffic theory. Indeed, the performance measurement of many learning models depends on the distribution of the eigenvalues of random matrices. Moreover, these results enabled us to observe phenomena specific to the high-dimensional regime, such as the double descent phenomenon. Our theoretical study is accompanied by numerical experiments illustrating the accuracy of the asymptotic equivalents we provide.