Designing and Evaluating a Fusion of Visible and Infrared Spectrum Video Streams for Remote Tower Operations

The research project INVIDEON evaluated requirements, technical solutions, and the benefit of fusing visible (VIS) and infrared (IR) spectrum camera streams into a single panorama video stream. In this paper, the design process for developing a usable and accepted fusion is described. As both sensors have strengthens and weaknesses, INVIDEON proposes a fused panorama optimized out of both sensors to be presented to the air traffic service officer (referred to as ‘operator’ in this context). This chapter gives an overview of the project and reports results of the operators’ perception performance with the visual, infrared, and fused panorama as well as results of the acceptance and usability of the INVIDEON solution. Main findings of requirements for fusing VIS and IR camera data for remote tower operations are highlighted and set into context with the air traffic control officer’s (ATCO’s) tasks. A specific fusion approach was developed within the project and evaluated by means of recorded IR and VIS data. For evaluation, a testbed was set up at a regional airport and data representing different visibility conditions were selected out of 70 days data recordings. Six operators participated in the final evaluation. The objectively possible detection performance and the recognition performance of the operator was determined, evaluated, and compared to the theoretical performance derived from the Johnson Criteria. Subjective data on perceived usability, situational awareness, and trust in automation was assessed. Furthermore, qualitative data on Human Machine Interface (HMI) design and optimization potential from debriefings and comments was collected and clustered.