An Unsupervised Approach to Multiple Speaker Tracking for Robust Multimedia Retrieval

Tagging multi media data based on who is speaking at what time, is important especially in the intelligent retrieval of recordings of meetings and conferences. In this paper an unsupervised approach to tracking more than two speakers in multi media data recorded from multiple visual sensors and a single audio sensor is proposed. The multi-speaker detection and tracking problem is first formulated as a multiple hypothesis testing problem. From this formulation we proceed to derive the multi speaker detection and tracking problem as a condition in mutual information. The proposed method is then evaluated for multi media recordings consisting of four speakers recorded on a multi media recording test bed. Experimental results on the CUAVE multi modal corpus are also discussed. The proposed method exhibits reasonably good performance as demonstrated by the detection (ROC) curves. The results of analysis based on the condition in mutual information are also encouraging. A multiple speaker detection and tracking system implemented using this approach gives reasonable performance in actual meeting room scenarios.