An interactive handheld spherical 3D object display system

Traditional display systems usually display 3D objects on static screens (monitor, wall, etc.) and the manipulation of virtual objects by the viewer is usually achieved via indirect tools such as keyboard or mouse. It would be more natural and direct if we display the object onto a handheld surface and manipulate it with our hands as if we were holding the real 3D object. In this paper, we propose a prototype system by projecting the object onto a handheld foam sphere. The aim is to develop an interactive 3D object manipulation and exhibition tool without the viewer having to wear spectacles. In our system, the viewer holds the sphere with his hands and moves it freely. Meanwhile we project well-tailored images onto the sphere to follow its motion, giving the viewer a virtual perception as if the object were sitting inside the sphere and being moved by the viewer. The design goal is to develop a low-cost, real-time, and interactive 3D display tool. An off-the-shelf projector-camera pair is first calibrated via a simple but efficient algorithm. Vision-based methods are proposed to detect the sphere and track its subsequent motion. The projection image is generated based on the projective geometry among the projector, sphere, camera and the viewer. We describe how to allocate the view spot and warp the projection image. We also present the result and the performance evaluation of the system.