Eigenvalue Distributions in Random Confusion Matrices: Applications to Machine Learning Evaluation

Abstract

This paper examines the distribution of eigenvalues for a 2×2 random confusion matrix used in machine learning evaluation. We also analyze the distributions of the matrix’s trace and the difference between the traces of random confusion matrices. Furthermore, we demonstrate how these distributions can be applied to calculate the superiority probability of machine learning models. By way of example, we use the superiority probability to compare the accuracy of four disease outcomes machine learning prediction tasks.