Differences in Video Search Behavior between Novices and Archivists

Improving users’ interactions with a video retrieval system requires the examination of the search behavior of real users. This paper presents a study that examines and compares the video search behavior of professional archivists and novice users. The comparison focuses on the use and effectiveness of different state-of-the-art video search methods offered by the VITALAS retrieval system, and also on the behavior of the two user groups during their interactions with the retrieval results. We conducted our experiments in the context of TRECVID’s 2009 interactive search task, using the provided collection and topics for our evaluation. The findings are based on a qualitative questionnaire analysis and a quantitative examination of the logged user actions on the search interface. The experimental results indicate that today’s visual search techniques have improved in effectiveness, confirming a trend found in previous user studies. To our surprise, professional archivists used visual concept search in many of their searches. Queries containing visual concepts were more effective, resulting in more relevant shots found than the alternative methods. Overall, we conclude that professional archivists are more focused on recall when carrying out their search tasks and are better at reflecting on their own search performance.