Image Captioning System for Movie Subtitling Using Neural Networks and LSTM

With the advent of the Internet, the multimedia business has experienced explosive growth and is now available to consumers worldwide. Particularly important to this growth has been the dominance of over-the-top (OTT) platforms during the COVID-19 epidemic. One important component in this growth has been the implementation of state-of-the-art Machine Learning (ML) methods. These algorithms can generate captions from video frames mechanically, increasing the platform's accessibility. However, it can be difficult to meet the needs of users speaking different languages, as there are many films being streamed in languages that may be inaccessible to consumers in other parts of the world. There is an issue with English being the most spoken language in the world. Therefore, individuals who aren't fluent in English may have trouble finding videos in their native language. By automatically creating English subtitles for any movie, regardless of the language spoken on the original audio track, machine learning plays a significant role in removing this barrier. Neural Networks, used for processing each frame of the video, and LSTM models, used for caption synthesis, are the key models employed here from the realm of machine learning. After the models have been trained, they can be incorporated into the user interface (UI) using a programming language like Python. In the user interface, the created caption can be shown alongside the uploaded image.