Algorithmic ways to obtain super-resolution of video data under applicative noise using deep neural networks

Abstract

The problem of constructing superresolution algorithms for video sequences is, the solution of which provides an increase in the resolution of the generated frames after processing several frames of the original low-resolution sequence. The peculiarity of the problem to be solved is the presence of distortions caused by the influence of applicative noise. The latter appear as distributed areas of anomalous observations or closing areas on the original frames and can be regarded as an additional factor the decreases the resolution of the input images. The existing approaches and algorithms for super-resolution of images and video data are analyzed, including those under the influence of applicative noise. Two approaches to video data super-resolution algorithms are considered. Both of them are based on the use of deep convolutional neural networks processing data in a time sliding window capturing several frames before and after the current frame and forming a higher quality image for it. The first algorithm is based on the use of a two-input neural network in the form of a directed acyclic graph and implements an iterative approach to form the next frame of the video sequence. The second algorithm is based on the modification of this neural network, taking into account the peculiarities of video data processing that allows to increase the processing speed by providing the input of all frames within a sliding window at once. Experimental study of synthesized algorithms has been conducted, the results of which showed that the first algorithm has a higher quality of restored images compared with the second one, but is inferior to it in terms of processing time. The problem of elimination of undesirable moving objects on the video sequences covering the areas with useful information is considered, which can also be considered as a problem of elimination of applicative interference. A relatively simple algorithm for segmentation of anomalous observation areas on video sequence frames based on comparing the next frame with a reference image and carrying out a sequence of morphological operations is proposed.