Visually aware recommendation with aesthetic features

Visual information plays a critical role in human decision-making process. Recent developments on visually aware recommender systems have taken the product image into account. We argue that the aesthetic factor is very important in modeling and predicting users’ preferences, especially for some fashion-related domains like clothing and jewelry. This work is an extension of our previous paper (Yu et al., in: WWW, pp 649–658, 2018), where we addressed the need of modeling aesthetic information in visually aware recommender systems. Technically speaking, we make three key contributions in leveraging deep aesthetic features. In Yu et al. (in: WWW, pp 649–658, 2018), (1) we introduced the aesthetic features extracted from product images by a deep aesthetic network to describe the aesthetics of products. We incorporated these features into recommender system to model users’ preferences in the aesthetic aspect. (2) Since in clothing recommendation, time is very important for users to make decision, we designed a new tensor decomposition model for implicit feedback data. The aesthetic features were then injected to the basic tensor model to capture the temporal dynamics of aesthetic preferences. In this extended version, we try to explore aesthetic features in negative sampling to get further benefit in recommendation tasks. In implicit feedback data, we only have positive samples. Negative sampling is performed to get negative samples. In conventional sampling strategy, uninteracted items are selected as negative samples randomly. However, we may sample potential samples (preferred but unseen items) as negative ones by mistake. To address this gap, (3) we use the aesthetic features to optimize the sampling strategy. We enrich the pairwise training samples by considering the similarity among items in the aesthetic space (and also in the semantic space and graphs). The key idea is that a user may likely have similar perception on similar items. We perform extensive experiments on several real-world datasets and demonstrate the usefulness of aesthetic features and the effectiveness of our proposed methods.