Semisupervised Cross-Media Retrieval by Distance-Preserving Correlation Learning and Multi-modal Manifold Regularization

Due to the heterogeneous representation and incongruous distribution of cross-media data, like text, image, audio, video, and 3D model, how to capture the correlations of heterogeneous data for cross-media retrieval is a challenging problem. In order to handle with multiple media types, this paper proposes a novel distance-preserving correlation learning and multi-modal manifold regularization (DCLMM) approach to exploit the common representation of heterogeneous data. The method mines the distance-preserving correlation by minimizing (maximizing) the distances between media samples with positive (negative) semantic correlations, while most existing methods only focus on positive correlations of pairwise media types. DCLMM also utilizes an intrinsic multi-modal manifold to well describe the geometry distribution of both labeled and unlabeled heterogeneous cross-media data. Moreover, DCLMM incorporates the distance-preserving correlation and multi-modal manifold into a kernel based regularization framework to explore more rich complementary information from high dimensional space. Extensive experimental results on two widely-used cross-media datasets with up to five media types demonstrate the effectiveness of DCLMM for cross-media retrieval, compared with the state-of-the-art methods.