A method for analyzing the dependence of random variables with a small number of observations

Abstract

With the nonlinear nature of the dependence of random variables, the use of the correlation coefficient can lead to incorrect conclusions. A more universal characteristic of the dependence is mutual information, which does not require the assumption of the linearity of the relationship of random variables. It can be interpreted as the average amount of information about a random variable contained in the distribution of some other random variable. The article describes a method for nonparametric estimation of the value of mutual information based on empirical data, which consists in minimizing a regularized quadratic functional in a Hilbert space with a reproducing kernel. The replacement of the scalar product in the standard Hilbert space by the value of a nonnegatively defined function of two variables is used. The analytical solution is a linear combination of the values of the reproducing kernel with coefficients calculated as the solution of a regularized system of linear equations. The results of the evaluation of the relationship between heart rate variability and the success of the operator of the control system using mutual information and using the correlation coefficient according to the data of the real experiment are presented. By the example of finding out the dependence of the success of the task by the operator of the control system on the variability of the heart rhythm on small time intervals, the effectiveness of using the proposed mutual information assessment is shown.