Alan Said
Brijnesh J. Jain
ABSTRACT
In this work we describe an approach at multi-objective recommender system evaluation based on a previously introduced 3D benchmarking model. The benchmarking model takes user-centric, business-centric and technical constraints into consideration in order to provide a means of comparison of recommender algorithms in similar scenarios. We present a comparison of three recommendation algorithms deployed in a user study using this 3D model and compare to standard evaluation methods. The proposed approach simplifies benchmarking of recommender systems and allows for simple multi-objective comparisons.
- Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22, 1 (Jan. 2004). Google Scholar
Digital Library
- Jambor, T., and Wang, J. Optimizing multiple objectives in collaborative filtering. In Proceedings of the fourth ACM conference on Recommender systems, RecSys'10, ACM (New York, NY, USA, 2010), 55--62. Google ScholarDigital Library
- Said, A., Fields, B., and Jain, B. J. User-centric evaluation of a k-furthest neighbor collaborative filtering recommender algorithm. In Proceedings of the ACM 2013 conference on Computer Supported Cooperative Work, ACM (New York, NY, USA, 2013). Google ScholarDigital Library
- Said, A., Tikk, D., Shi, Y., Larson, M., Stumpf, K., and Cremonesi, P. Recommender systems evaluation: A 3d bench-mark. In Proceedings of the Workshop on Recommendation Utility Evaluation: Beyond RMSE (RUE 2012), RUE'12, CEUR-WS Vol. 910 (2012). Google ScholarDigital Library
Index Terms
- A 3D approach to recommender system evaluation
Recommendations
A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data MiningRecommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
Comparative recommender system evaluation: benchmarking recommendation frameworks
RecSys '14: Proceedings of the 8th ACM Conference on Recommender systemsRecommender systems research is often based on comparisons of predictive accuracy: the better the evaluation scores, the better the recommender. However, it is difficult to compare results from different recommender systems due to the many options in ...
Temporal diversity in recommender systems
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrievalCollaborative Filtering (CF) algorithms, used to build web-based recommender systems, are often evaluated in terms of how accurately they predict user ratings. However, current evaluation techniques disregard the fact that users continue to rate items ...
Comments