Trapit Bansal
ABSTRACT
In a variety of application domains the content to be recommended to users is associated with text. This includes research papers, movies with associated plot summaries, news articles, blog posts, etc. Recommendation approaches based on latent factor models can be extended naturally to leverage text by employing an explicit mapping from text to factors. This enables recommendations for new, unseen content, and may generalize better, since the factors for all items are produced by a compactly-parametrized model. Previous work has used topic models or averages of word embeddings for this mapping. In this paper we present a method leveraging deep recurrent neural networks to encode the text sequence into a latent vector, specifically gated recurrent units (GRUs) trained end-to-end on the collaborative filtering task. For the task of scientific paper recommendation, this yields models with significantly higher accuracy. In cold-start scenarios, we beat the previous state-of-the-art, all of which ignore word order. Performance is further improved by multi-task learning, where the text encoder network is trained for a combination of content recommendation and item metadata prediction. This regularizes the collaborative filtering model, ameliorating the problem of sparsity of the observed rating matrix.
Supplemental Material
- Ido Guy, Naama Zwerdling, Inbal Ronen, David Carmel, and Erel Uziel. Social media recommendation based on people and tags. In SIGIR, 2010. Google Scholar
Digital Library
- Owen Phelan, Kevin McCarthy, and Barry Smyth. Using twitter to recommend real-time topical news. In RecSys, 2009. Google ScholarDigital Library
- Trapit Bansal, Mrinal Das, and Chiranjib Bhattacharyya. Content driven user profiling for comment-worthy recommendations of news and blog articles. In RecSys, 2015. Google ScholarDigital Library
- Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. In RecSys, 2013. Google ScholarDigital Library
- Chong Wang and David M Blei. Collaborative topic modeling for recommending scientific articles. In SIGKDD, 2011. Google ScholarDigital Library
- Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, (8): 30--37, 2009. Google ScholarDigital Library
- Andriy Mnih and Ruslan Salakhutdinov. Probabilistic matrix factorization. In NIPS, 2007. Google ScholarDigital Library
- nd Shoham(1997)}balabanovic1997fabMarko Balabanović and Yoav Shoham. Fab: content-based, collaborative recommendation. Communications of the ACM, 40 (3): 66--72, 1997. Google ScholarDigital Library
- Raymond J Mooney and Loriene Roy. Content-based book recommending using learning for text categorization. In ACM conference on Digital libraries, 2000. Google ScholarDigital Library
- Chumki Basu, Haym Hirsh, William Cohen, et al. Recommendation as classification: Using social and content-based information in recommendation. In AAAI, 1998. Google ScholarDigital Library
- Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. Methods and metrics for cold-start recommendations. In SIGIR, 2002. Google ScholarDigital Library
- Justin Basilico and Thomas Hofmann. Unifying collaborative and content-based filtering. In ICML, 2004. Google ScholarDigital Library
- Hao Wang, Naiyan Wang, and Dit-Yan Yeung. Collaborative deep learning for recommender systems. In SIGKDD, 2015. Google ScholarDigital Library
- Prem Melville, Raymond J Mooney, and Ramadass Nagarajan. Content-boosted collaborative filtering for improved recommendations. In AAAI, 2002. Google ScholarDigital Library
- Prem K Gopalan, Laurent Charlin, and David Blei. Content-based recommendations with poisson factorization. In NIPS, 2014. Google ScholarDigital Library
- Deepak Agarwal and Bee-Chung Chen. Regression-based latent factor models. In SIGKDD, 2009. Google ScholarDigital Library
- Hanna M Wallach. Topic modeling: beyond bag-of-words. In ICML, 2006. Google ScholarDigital Library
- Paul J Werbos. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78 (10): 1550--1560, 1990.Google ScholarCross Ref
- Burget, Cernockỳ, and Khudanpur}mikolov2010recurrentTomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. Recurrent neural network based language model. INTERSPEECH, 2010.Google ScholarCross Ref
- Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP, 2014.Google ScholarCross Ref
- Robert M Bell and Yehuda Koren. Lessons from the netflix prize challenge. SIGKDD Explorations Newsletter, 9 (2): 75--79, 2007. Google ScholarDigital Library
- Guang Ling, Michael R Lyu, and Irwin King. Ratings meet reviews, a combined approach to recommend. In RecSys, 2014. Google ScholarDigital Library
- Amjad Almahairi, Kyle Kastner, Kyunghyun Cho, and Aaron Courville. Learning distributed representations from reviews for collaborative filtering. In RecSys, 2015. Google ScholarDigital Library
- Jason Weston, Samy Bengio, and Nicolas Usunier. Wsabie: Scaling up to large vocabulary image annotation. In IJCAI, 2011. Google ScholarDigital Library
- Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In ICDM, 2008. Google ScholarDigital Library
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In UAI, 2009. Google ScholarDigital Library
- Yue Shi, Martha Larson, and Alan Hanjalic. Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges. ACM Computing Surveys, 47 (1): 3, 2014. Google ScholarDigital Library
- Steffen Rendle. Factorization machines. In ICDM, 2010. Google ScholarDigital Library
- Zeno Gantner, Lucas Drumond, Christoph Freudenthaler, Steffen Rendle, and Lars Schmidt-Thieme. Learning attribute-to-feature mappings for cold-start recommendations. In ICDM, 2010. Google ScholarDigital Library
- Rich Caruana. Multitask learning. Machine learning, 28 (1): 41--75, 1997. Google ScholarDigital Library
- Ajit P Singh and Geoffrey J Gordon. Relational learning via collective matrix factorization. In SIGKDD, 2008. Google ScholarDigital Library
- Hao Ma, Haixuan Yang, Michael R Lyu, and Irwin King. Sorec: social recommendation using probabilistic matrix factorization. In CIKM, 2008. Google ScholarDigital Library
- Ralf Krestel, Peter Fankhauser, and Wolfgang Nejdl. Latent dirichlet allocation for tag recommendation. In RecSys, 2009. Google ScholarDigital Library
- Yoshua Bengio Ian Goodfellow and Aaron Courville. Deep learning. Book in prep. for MIT Press, 2016.Google Scholar
- Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton. Restricted boltzmann machines for collaborative filtering. In ICML, 2007. Google ScholarDigital Library
- Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. Autorec: Autoencoders meet collaborative filtering. In WWW, 2015. Google ScholarDigital Library
- Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. Collaborative denoising auto-encoders for top-n recommender systems. In WSDM, 2016. Google ScholarDigital Library
- Ali Mamdouh Elkahky, Yang Song, and Xiaodong He. A multi-view deep learning approach for cross domain user modeling in recommendation systems. In WWW, 2015. Google ScholarDigital Library
- Gintare Karolina Dziugaite and Daniel M Roy. Neural network matrix factorization. arXiv preprint arXiv:1511.06443, 2015.Google Scholar
- Aaron Van den Oord, Sander Dieleman, and Benjamin Schrauwen. Deep content-based music recommendation. In NIPS, 2013. Google ScholarDigital Library
- Xinxi Wang and Ye Wang. Improving content-based and hybrid music recommendation using deep learning. In International Conference on Multimedia, 2014. Google ScholarDigital Library
- Jason Weston, Sumit Chopra, and Keith Adams.# tagspace: Semantic embeddings from hashtags. 2014.Google Scholar
- R. He and J. McAuley. VBPR: visual bayesian personalized ranking from implicit feedback. In AAAI, 2016.Google ScholarDigital Library
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111--3119, 2013. Google ScholarDigital Library
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. JMLR, 12: 2493--2537, 2011. Google ScholarDigital Library
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104--3112, 2014. Google ScholarDigital Library
- Andrew M Dai and Quoc V Le. Semi-supervised sequence learning. In NIPS, pages 3061--3069, 2015. Google ScholarDigital Library
- Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. Neural Networks, 5 (2): 157--166, 1994. Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9 (8): 1735--1780, 1997. Google ScholarDigital Library
- Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.Google Scholar
- Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. An empirical exploration of recurrent network architectures. In ICML, 2015.Google ScholarDigital Library
- Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. Signal Processing, 45 (11): 2673--2681, 1997. Google ScholarDigital Library
- Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society., pages 1--38, 1977.Google Scholar
- Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.Google Scholar
- Misha Denil, Alban Demiraj, and Nando de Freitas. Extraction of salient sentences from labelled documents. arXiv preprint arXiv:1412.6815, 2014.Google Scholar
- Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. Visualizing and understanding neural models in nlp. 2016.Google Scholar
Index Terms
- Ask the GRU: Multi-task Learning for Deep Text Recommendations
Recommendations
User-Specific Feature-Based Similarity Models for Top-n Recommendation of New Items
Survey Paper, Regular Papers and Special Section on Participatory Sensing and Crowd IntelligenceRecommending new items for suitable users is an important yet challenging problem due to the lack of preference history for the new items. Noncollaborative user modeling techniques that rely on the item features can be used to recommend new items. ...
Naïve filterbots for robust cold-start recommendations
KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data miningThe goal of a recommender system is to suggest items of interest to a user based on historical behavior of a community of users. Given detailed enough history, item-based collaborative filtering (CF) often performs as well or better than almost any ...
Merging trust in collaborative filtering to alleviate data sparsity and cold start
Providing high quality recommendations is important for e-commerce systems to assist users in making effective selection decisions from a plethora of choices. Collaborative filtering is a widely accepted technique to generate recommendations based on ...
Comments