An Integrated Visualization Framework to Enhance Human–Robot Collaboration in Facility Management

The global pandemic has sparked the popularity of robots in facility management tasks (FMTs) such as floor cleaning and disinfection. This trend brings many task scenarios where humans and robots need to cooperate with each other. Effective human–robot collaboration (HRC) relies on precise communication of the robots’ intentions (e.g., to move to a position, or to grasp an object), so that their human counterparts can adapt their behaviors/actions accordingly. However, little has been known on how this can be done in FMTs. Visualization technologies have potential to enhance HRC by communicating the robot intention in a visualized manner. This research aims to develop a framework that integrates the latest visualization technologies, e.g., building information modelling (BIM) and augmented reality (AR), to enhance HRC in facility management. The framework includes two complementary modules: (a) a remote monitoring module (RMM) that can remotely transmit and visualize robot information in a Web-based BIM to inform decision-making, and (b) an onsite collaboration module (OCM) that augments human co-workers with real-time robot intention to allow effective cooperation. Experiments were conducted to validate the proposed framework in typical FMTs. Results show that the integrated visualization framework can intuitively and unambiguously convey robots’ intentions to their human counterparts, significantly improving the performance of HRC. Future research is suggested to complement the framework with a reverse mechanism to effectively convey human intentions to robots.