Machine learning in the problem of image quality evaluation in super-resolution systems

Abstract

This paper describes an approach to assessing image quality in super-resolution systems based on deep learning using a pre-trained neural network with an additional pre-processing layer that performs feature transfer to the frequency domain. Most of the existing quality metrics traditionally used in image processing problems do not allow capture changes of detalization occurred when using super-resolution algorithms. Such metrics do not allow adequately evaluate the results of super-resolution algorithms and compare them with each other. Therefore, the purpose of this work is to develop an algorithm for quantifying the change in image resolution that occurs due to processing by an arbitrary super resolution algorithm. The proposed quality assessment algorithm is based on methods of deep learning and transfer learning. Taking into account the physical meaning of the super-resolution effect, the processing of features in the neural network performed for the frequency domain, therefore, in this work author proposed the special preprocessing layer that performs such transition using the Fourier transform. The training scheme consists of two stages. In the first step, an imaging system model is created for modeling the acquisition of low-resolution images from arbitrary high-resolution images. It is shown that the use of a neural network for low-resolution acquisition is significantly superior in accuracy to conventional linear models. Next, the formation model is applied to increase the size of the training sample used to train the quality assessment algorithm. The resulting quality assessment algorithm is applicable both for comparing the results of the work of various super-resolution algorithms with each other, and for estimating the limitations of resolution increasing of an arbitrary super-resolution algorithm.