Semantic 3D Object Maps for Everyday Robotic Retail Inspection

In retail field, customer culture is shifting towards in-store researching, and retailers need to re-evaluate their location services to better assist customer. In-store mapping help retailers learn how their employees are interacting and it satisfies user intent to search for products, something that is often ignored by retailers especially for the secondary placement, which contains offers and promotions that change very often. In this paper, we describe a retail robot that moves autonomously inside a store and gathers points cloud data for a semantic store mapping. With all the data collected, it is possible to build a 3D map of the store with the exact product locations. This retail robot combines the features of both Robotics and Artificial Intelligence. Three classification approach have been compared in order to achieve the best performances: a machine learning technique, PointNet++ and a novel Reflectance PointNet++ especially designed for this task. Experiments are performed in a real retail environment that is an Italian supermarket, during business hours. A dataset has been built and made publicly available. The application of our approach yields good results in terms of precision, recall and F1-score and demonstrates the effectiveness of the proposed approach.