26. An Application for Processing Large and Non-Uniform Media Objects on MapReduce-Based Clusters

Researchers at the Austrian Institute of Technology are exploring ways for the processing of media archives on large computer infrastructures using data intensive computing methods. The work is motivated by a strong demand for scalable methods that support the processing of media content such as can be found in archives of broadcasting or memory institutions. Data intensive computing frameworks leverage cloud technologies in order to generate and process large data sets on clusters of virtualized computers. MapReduce provides a highly scalable programming model in this context that has proven to be widely applicable for processing structured data. We have developed an approach and implementation that utilizes this model for the processing of audiovisual content. Here, we present an application that is capable of analyzing and modifying large audiovisual files using multiple computer nodes in parallel and thereby able to dramatically reduce processing times. The article provides detailed insights into the developed approach and the corresponding application. We summarize previous work and provide recent results that evaluate the application in a large-scale cloud environment.