See Fine Color from the Rough Black-and-White

Image super-resolution and colorization are two important research fields in computer vision. In previous studies, they have been considered separately as two unrelated tasks. However, for the task of restoring gray video to high-definition color video, when the network learns to abstract features from low-resolution images and maps them to high-resolution images, the abstract understanding of images by the network is also useful for colorization task. Treating them as two unrelated tasks have to construct two different models, which needs more time and resources. In this paper, we propose a framework to combine the tasks of image super-resolution and colorization together. We design a new network model to directly map low-resolution gray images into high-resolution color images. Moreover, this model can obtain motion information of objects in the video by predicting surrounding frames with the current frame. Thus, video super-resolution and colorization can be realized. To support studying super-resolution and colorization together, we build a video dataset containing three scenes. As far as we know, this is the first dataset for such kinds of tasks combination.