Latent-MVCNN: 3D Shape Recognition Using Multiple Views from Pre-defined or Random Viewpoints

The Multi-view Convolution Neural Network (MVCNN) has achieved considerable success in 3D shape recognition. However, 3D shape recognition using view-images from random viewpoints has not been yet exploited in depth. In addition, 3D shape recognition using a small number of view-images remains difficult. To tackle these challenges, we developed a novel Multi-view Convolution Neural Network, “Latent-MVCNN” (LMVCNN), that recognizes 3D shapes using multiple view-images from pre-defined or random viewpoints. The LMVCNN consists of three types of sub Convolution Neural Networks. For each view-image, the first type of CNN outputs multiple category probability distributions and the second type of CNN outputs a latent vector to help the first type of CNN choose the decent distribution. The third type of CNN outputs the transition probabilities from the category probability distributions of one view to the category probability distributions of another view, which further helps the LMVCNN to find the decent category probability distributions for each pair of view-images. The three CNNs cooperate with each other to the obtain satisfactory classification scores. Our experimental results show that the LMVCNN achieves competitive performance in 3D shape recognition on ModelNet10 and ModelNet40 for both the pre-defined and the random viewpoints and exhibits promising performance when the number of view-images is quite small.