5W1H Aware Framework for Representing and Detecting Real Events from Multimedia Digital Ecosystem

A digital media sharing platform (e.g., YouTube, Twitter, Facebook, and Flickr) is an advanced Digital Ecosystem that focuses on mobile device to share multimedia resources. Millions of users share different events (e.g., sport, earthquake, concerts, etc.) through social media platforms. As a result, the platforms host heterogeneous and a significant amount of user-generated multimedia documents (e.g., image, voice, video, text, etc.). In this paper, we introduce a general framework for representing events while keeping expressivity and capability to recognize events from Multimedia-based Digital Ecosystem. It takes as input: a collection of multimedia objects from heterogeneous sources, and then produces as output clustered real-world events. The proposed framework consists of two main components for: (i) defining and representing each dimension of multimedia objects (such as, participant (who), temporal (when), spatial (where), sematic (what) and causal (why)); (ii) detecting real events using scalable clustering algorithm in an unsupervised manner. To improve our clustering framework, we developed clustering comparison strategies using combination of dimensions (contextual features) of multimedia objects. We also showed how clustering comparison strategies can be used to detect real-world events and measured the quality of our clustering algorithm using F-score. The experimental results exhibited promising result.