A modified method for assessing the quality of generative adversarial neural networks

Abstract

Evaluation of the efficiency and the quality of arbitrary generative adversarial neural networks (GAN) is a pressing problem. Currently, acceptable results have only been obtained for networks that generate images of a certain format and size. Other data types and structures can be assessed exclusively by an expert method, the disadvantage of which is their subjectivity, low productivity, and the impossibility of automating the processing of large amounts of information. The analysis of existing approaches to assessing the effectiveness of GAN identified the most common metrics, among which Inception Score and Fréchet Inception Distance should be noted. However, these metrics cannot be used to estimate GANs generating objects other than images of a certain format. This is due to the specifics of neural networks used to calculate these metrics. Therefore, the paper proposes a modified method for assessing the quality of the GAN, based on the Inception Score metric, which is characterized by the use of an arbitrary classifier to calculate the Kullback-Leibler distance, which makes it possible to analyse the generation quality of arbitrary objects. The article presents a mathematical description of the methods for assessing the quality of the GAN and the modifications introduced. Practical experiments were carried out on several well-known datasets: graphic datasets (MNIST) and numerical (Human Activity Recognition Using Smartphones, Epileptic Seizure Recognition) datasets. For each set, the developed modified GAN estimation method was tested. The obtained results confirm the possibility of using the method to evaluate arbitrary data sets. The use of the obtained theoretical and practical results in the implementation and training of GAN will improve the quality of samples generated by neural networks and automate the process of their assessment.