48. Style Transfer for Videos with Audio

In the art of painting, from early era of beginning of the human civilization, human beings have been creating artistic images with content from the real world but style from their imagination. Consider one such art called The Starry Night by painter van Gogh. Here the mountains, moon and houses are content taken from the real world but the style of painting is totally from the painter’s imagination and is unique. Style Transfer is the problem of taking content of an image and style of other image to create third image having content of the former but style of the latter. Clearly, such work cannot be obtained by simple overlapping the two images. Until recently, due to not so optimized GPUs and slower hardware, image processing was a time consuming computation problem. But now we can use technological optimizations to use Convolutional Neural Networks (CNNs) to do the Style Transfer. In this paper, we discuss how style transfer can be done in videos having audio in them. We shall also compare one of the existing methods with our implementation. Our proposed work has potential applications in the domains of social media communication, entertainment industry and mobile applications.