Steffen Rendle
Skip Abstract Section
Abstract
Factorization approaches provide high accuracy in several important prediction problems, for example, recommender systems. However, applying factorization approaches to a new prediction problem is a nontrivial task and requires a lot of expert knowledge. Typically, a new model is developed, a learning algorithm is derived, and the approach has to be implemented.
Factorization machines (FM) are a generic approach since they can mimic most factorization models just by feature engineering. This way, factorization machines combine the generality of feature engineering with the superiority of factorization models in estimating interactions between categorical variables of large domain. libFM is a software implementation for factorization machines that features stochastic gradient descent (SGD) and alternating least-squares (ALS) optimization, as well as Bayesian inference using Markov Chain Monto Carlo (MCMC). This article summarizes the recent research on factorization machines both in terms of modeling and learning, provides extensions for the ALS and MCMC algorithms, and describes the software tool libFM.
- Adomavicius, G., Sankaranarayanan, R., Sen, S., and Tuzhilin, A. 2005. Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Info. Syst. 23, 1, 103--145. Google Scholar
Digital Library
- Agarwal, D. and Chen, B.-C. 2009. Regression-based latent factor models. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’09). ACM, New York, NY, 19--28. Google ScholarDigital Library
- Baltrunas, L. and Ricci, F. 2009. Context-based splitting of item ratings in collaborative filtering. In Proceedings of the third ACM Conference on Recommender Systems (RecSys’09). ACM, New York, NY, 245--248. Google ScholarDigital Library
- Chang, C.-C. and Lin, C.-J. 2011. Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1--27:27. Google ScholarDigital Library
- Chen, T., Zheng, Z., Lu, Q., Zhang, W., and Yu, Y. 2011. Feature-based matrix factorization. Tech. rep. APEX-TR-2011-07-11, Apex Data & Knowledge Management Lab, Shanghai Jiao Tong University.Google Scholar
- Freudenthaler, C., Schmidt-Thieme, L., and Rendle, S. 2011. Bayesian factorization machines. In Proceedings of the NIPS Workshop on Sparse Representation and Low-rank Approximation.Google Scholar
- Gantner, Z., Drumond, L., Freudenthaler, C., Rendle, S., and Lars, S.-T. 2010. Learning attribute-to-feature mappings for cold-start recommendations. In Proceedings of the IEEE International Conference on Data Mining (ICDM’10). IEEE Computer Society, Los Alamintos, CA, 176--185. Google ScholarDigital Library
- Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. 2003. Bayesian Data Analysis 2nd Ed. Chapman and Hall/CRC.Google Scholar
- Gunawardana, A. and Shani, G. 2009. A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10, 2935--2962. Google ScholarDigital Library
- Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. 2009. The weka data mining software: An update. SIGKDD Explor. Newsl. 11, 10--18. Google ScholarDigital Library
- Harshman, R. A. 1970. Foundations of the parafac procedure: Models and conditions for an ‘exploratory’ multimodal factor analysis. UCLA Working Papers in Phonetics, 1--84.Google Scholar
- Jahrer, M., Töscher, A., and Legenstein, R. 2010. Combining predictions for accurate recommender systems. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10). ACM, New York, NY, 693--702. Google ScholarDigital Library
- Joachims, T. 1999. Making Large-Scale Support Vector Machine Learning Practical. MIT Press, Cambridge, MA, 169--184. Google ScholarDigital Library
- Karatzoglou, A., Amatriain, X., Baltrunas, L., and Oliver, N. 2010. Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). ACM, New York, NY, 79--86. Google ScholarDigital Library
- Koren, Y. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’08). ACM, New York, NY, 426--434. Google ScholarDigital Library
- Koren, Y. 2009a. The bellkor solution to the Netflix grand prize.Google Scholar
- Koren, Y. 2009b. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’09). ACM, New York, NY, 447--456. Google ScholarDigital Library
- Koren, Y. 2010. Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Trans. Knowl. Discov. Data 4, 1:1--1:24. Google ScholarDigital Library
- Lim, Y. J. and Teh, Y. W. 2007. Variational Bayesian approach to movie rating prediction. In Proceedings of the KDD Cup and Workshop.Google Scholar
- Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. Inter. Comput. IEEE 7, 1, 76--80. Google ScholarDigital Library
- Lipczak, M., Hu, Y., Kollet, Y., and Milios, E. 2009. Tag sources for recommendation in collaborative tagging systems. In Proceedings of the ECML-PKDD Discovery Challenge Workshop.Google Scholar
- Lipczak, M. and Milios, E. 2011. Efficient tag recommendation for real-life data. ACM Trans. Intell. Syst. Technol. 3, 1, 2:1--2:21. Google ScholarDigital Library
- Liu, N. N. and Yang, Q. 2008. Eigenrank: A ranking-oriented approach to collaborative filtering. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’08). ACM, New York, NY, 83--90. Google ScholarDigital Library
- Ma, H., King, I., and Lyu, M. R. 2011. Learning to recommend with explicit and implicit social relations. ACM Trans. Intell. Syst. Technol. Article 29. Google ScholarDigital Library
- Marinho, L. B., Preisach, C., and Schmidt-Thieme, L. 2009. Relational classification for personalized tag recommendation. In Proceedings of the ECML-PKDD Discovery Challenge Workshop.Google Scholar
- Paterek, A. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of the KDD Cup Workshop 13th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD’07). 39--42.Google Scholar
- Pilászy, I., Zibriczky, D., and Tikk, D. 2010. Fast als-based matrix factorization for explicit and implicit feedback datasets. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). ACM, New York, NY, 71--78. Google ScholarDigital Library
- Rendle, S. 2010. Factorization machines. In Proceedings of the 10th IEEE International Conference on Data Mining. IEEE Computer Society. Google ScholarDigital Library
- Rendle, S. 2012. Learning recommender systems with adaptive regularization. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining (WSDM’12). ACM, New York, NY, 133--142. Google ScholarDigital Library
- Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI’09). Google ScholarDigital Library
- Rendle, S., Freudenthaler, C., and Schmidt-Thieme, L. 2010. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM, New York, NY, 811--820. Google ScholarDigital Library
- Rendle, S., Gantner, Z., Freudenthaler, C., and Schmidt-Thieme, L. 2011. Fast context-aware recommendations with factorization machines. In Proceedings of the 34th ACM SIGIR Conference on Reasearch and Development in Information Retrieval. Google ScholarDigital Library
- Rendle, S. and Schmidt-Thieme, L. 2010. Pairwise interaction tensor factorization for personalized tag recommendation. In Proceedings of the third ACM International Conference on Web Search and Data Mining (WSDM’10). ACM, New York, NY, 81--90. Google ScholarDigital Library
- Robert, C. P. 1995. Simulation of truncated normal variables. Stat. Comput. 5, 121--125.Google ScholarCross Ref
- Salakhutdinov, R. and Mnih, A. 2008a. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In Proceedings of the 25th International Conference on Machine Learning. Google ScholarDigital Library
- Salakhutdinov, R. and Mnih, A. 2008b. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems 20.Google Scholar
- Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. ACM Press, New York, NY, 285--295. Google ScholarDigital Library
- Srebro, N. and Jaakkola, T. 2003. Weighted low rank approximation. In Proceedings of the 20th International Conference on Machine Learning (ICML’03).Google Scholar
- Srebro, N., Rennie, J. D. M., and Jaakola, T. S. 2005. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 17, MIT 1329--1336.Google Scholar
- Stern, D. H., Herbrich, R., and Graepel, T. 2009. Matchbox: Large-scale online Bayesian recommendations. In Proceedings of the 18th International Conference on World Wide Web (WWW’09). ACM, New York, NY, 111--120. Google ScholarDigital Library
- Takács, G., Pilászy, I., Németh, B., and Tikk, D. 2009. Scalable collaborative filtering approaches for large recommender systems. J. Mach. Learn. Res. 10, 623--656. Google ScholarDigital Library
- Tucker, L. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279--311.Google ScholarCross Ref
- Xiong, L., Chen, X., Huang, T.-K., Schneider, J., and Carbonell, J. G. 2010. Temporal collaborative filtering with Bayesian probabilistic tensor factorization. In Proceedings of the SIAM International Conference on Data Mining (SIAM). 211--222.Google Scholar
- Zhang, N., Zhang, Y., and Tang, J. 2009. A tag recommendation system based on contents. In Proceedings of the ECML-PKDD Discovery Challenge Workshop.Google Scholar
- Zheng, Y. and Xie, X. 2011. Learning travel recommendations from user-generated gps traces. ACM Trans. Intell. Syst. Technol. 2, 2:1--2:29. Google ScholarDigital Library
Index Terms
- Factorization Machines with libFM
Recommendations
Co-manifold Matrix Factorization
ICCPR '20: Proceedings of the 2020 9th International Conference on Computing and Pattern RecognitionMatrix factorization plays a fundamental role in collaborative filtering. In collaborative filtering setting, the rating matrix R is very sparse. Thus, infinite number of matrices can fit the observed entries in the rating matrix. Without additional ...
Factorization Machines
ICDM '10: Proceedings of the 2010 IEEE International Conference on Data MiningIn this paper, we introduce Factorization Machines (FM) which are a new model class that combines the advantages of Support Vector Machines (SVM) with factorization models. Like SVMs, FMs are a general predictor working with any real valued feature ...
Separating-Plane Factorization Models: Scalable Recommendation from One-Class Implicit Feedback
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge ManagementWe study the video recommendation problem based on a large amount of user viewing logs instead of explicit ratings. As viewing records are implicitly suggest user preferences, existing matrix factorization methods fail to generate discriminative ...
Comments