Skip to main content
Erschienen in: Journal of Combinatorial Optimization 2/2016

01.02.2016

Team selection for prediction tasks

verfasst von: MohammadAmin Fazli, Azin Ghazimatin, Jafar Habibi, Hamid Haghshenas

Erschienen in: Journal of Combinatorial Optimization | Ausgabe 2/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Given a random variable \(O \in \mathbb {R}\) and a set of experts \(E\), we describe a method for finding a subset of experts \(S \subseteq E\) whose aggregated opinion best predicts the outcome of \(O\). Therefore, the problem can be regarded as a team formation for performing a prediction task. We show that in case of aggregating experts’ opinions by simple averaging, finding the best team (the team with the lowest total error during past \(k\) rounds) can be modeled with an integer quadratic programming and we prove its NP-hardness whereas its relaxation is solvable in polynomial time. At the end, we do an experimental comparison between different rounding and greedy heuristics on artificial datasets which are generated based on calibration and informativeness of exprets’ information and show that our suggested tabu search works effectively.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Attarzadeh I, Ow SH (2011) Software development cost and time forecasting using a high performance artificial neural network model. In: Intelligent Computing and Information Science, pp 18–26. Springer, Berlin Attarzadeh I, Ow SH (2011) Software development cost and time forecasting using a high performance artificial neural network model. In: Intelligent Computing and Information Science, pp 18–26. Springer, Berlin
Zurück zum Zitat Boutilier C (2012) Eliciting forecasts from self-interested experts: scoring rules for decision makers. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, Vol 2, pp 737–744. International Foundation for Autonomous Agents and Multiagent Systems Boutilier C (2012) Eliciting forecasts from self-interested experts: scoring rules for decision makers. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, Vol 2, pp 737–744. International Foundation for Autonomous Agents and Multiagent Systems
Zurück zum Zitat Chen Y, Kash I (2011) Information elicitation for decision making. In The 10th International Conference on Autonomous Agents and Multiagent Systems, Vol 1, pp 175–182. International Foundation for Autonomous Agents and Multiagent Systems Chen Y, Kash I (2011) Information elicitation for decision making. In The 10th International Conference on Autonomous Agents and Multiagent Systems, Vol 1, pp 175–182. International Foundation for Autonomous Agents and Multiagent Systems
Zurück zum Zitat Chen Y, Chu C-H, Mullen T, Pennock DM (2005) Information markets vs. opinion pools: an empirical comparison. In: Proceedings of the 6th ACM Conference on Electronic Commerce, pp 58–67. ACM,Vancouver, Vancouver Chen Y, Chu C-H, Mullen T, Pennock DM (2005) Information markets vs. opinion pools: an empirical comparison. In: Proceedings of the 6th ACM Conference on Electronic Commerce, pp 58–67. ACM,Vancouver, Vancouver
Zurück zum Zitat Chen Y, Chu C-H, Mullen T (2006) Predicting uncertain outcomes using information markets: trader behavior and information aggregation. New Math Nat Comput 2(03):281–297MathSciNetCrossRefMATH Chen Y, Chu C-H, Mullen T (2006) Predicting uncertain outcomes using information markets: trader behavior and information aggregation. New Math Nat Comput 2(03):281–297MathSciNetCrossRefMATH
Zurück zum Zitat Chhabra M, Das S, Szymanski B (2013) Team formation in social networks. In: Computer and Information Sciences III, pp 291–299. Springer, Berlin Chhabra M, Das S, Szymanski B (2013) Team formation in social networks. In: Computer and Information Sciences III, pp 291–299. Springer, Berlin
Zurück zum Zitat Clemen RT, Winkler RL (1999) Combining probability distributions from experts in risk analysis. Risk Anal 19(2):187–203 Clemen RT, Winkler RL (1999) Combining probability distributions from experts in risk analysis. Risk Anal 19(2):187–203
Zurück zum Zitat Clemen RT, Winkler RL (2007) Aggregating probability distributions. Advances in Decision Analysis: From Foundations to Applications. Cambridge University Press, Cambridge, pp 154–176CrossRef Clemen RT, Winkler RL (2007) Aggregating probability distributions. Advances in Decision Analysis: From Foundations to Applications. Cambridge University Press, Cambridge, pp 154–176CrossRef
Zurück zum Zitat Cooke RM (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York, NY Cooke RM (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York, NY
Zurück zum Zitat Dani V, Madani O, Pennock DM, Sanghai S, Galebach B (2012) An empirical comparison of algorithms for aggregating expert predictions. arXiv preprint arXiv:1206.6814 Dani V, Madani O, Pennock DM, Sanghai S, Galebach B (2012) An empirical comparison of algorithms for aggregating expert predictions. arXiv preprint arXiv:​1206.​6814
Zurück zum Zitat Genest C, Zidek JV (1986) Combining probability distributions: a critique and an annotated bibliography. Stat Sci 1(1):114–135MathSciNetCrossRef Genest C, Zidek JV (1986) Combining probability distributions: a critique and an annotated bibliography. Stat Sci 1(1):114–135MathSciNetCrossRef
Zurück zum Zitat Graefe A, Scott AJ, Jones RJ Jr, Cuzán AG (2014) Combining forecasts: an application to elections. Int J Forecast 30(1):43–54CrossRef Graefe A, Scott AJ, Jones RJ Jr, Cuzán AG (2014) Combining forecasts: an application to elections. Int J Forecast 30(1):43–54CrossRef
Zurück zum Zitat Hammitt JK, Yifan Z (2013) Combining experts’ judgments: comparison of algorithmic methods using synthetic data. Risk Anal 33(1):109–120CrossRef Hammitt JK, Yifan Z (2013) Combining experts’ judgments: comparison of algorithmic methods using synthetic data. Risk Anal 33(1):109–120CrossRef
Zurück zum Zitat Hora SC (2007) Expert judgment. Encyclopedia of Quantitative Risk Analysis and Assessment Hora SC (2007) Expert judgment. Encyclopedia of Quantitative Risk Analysis and Assessment
Zurück zum Zitat Jacobs RA (1995) Methods for combining experts’ probability assessments. Neural Comput 7(5):867–888CrossRef Jacobs RA (1995) Methods for combining experts’ probability assessments. Neural Comput 7(5):867–888CrossRef
Zurück zum Zitat Jørgensen M (2007) Forecasting of software development work effort: evidence on expert judgement and formal models. Int J Forecast 23(3):449–462CrossRef Jørgensen M (2007) Forecasting of software development work effort: evidence on expert judgement and formal models. Int J Forecast 23(3):449–462CrossRef
Zurück zum Zitat Kallen MJ, Cooke RM (2002) Expert aggregation with dependence. In: Probabilistic Safety Assessment and Management, pp 1287–94. Elsevier, Oxford Kallen MJ, Cooke RM (2002) Expert aggregation with dependence. In: Probabilistic Safety Assessment and Management, pp 1287–94. Elsevier, Oxford
Zurück zum Zitat Kargar M, Zihayat M, An A (2013) Affordable and collaborative team formation in an expert network. Department of Computer Science and Engineering, York University, Technical Report CSE-2013, 1 Kargar M, Zihayat M, An A (2013) Affordable and collaborative team formation in an expert network. Department of Computer Science and Engineering, York University, Technical Report CSE-2013, 1
Zurück zum Zitat Lappas T, Liu K, Terzi E (2009) Finding a team of experts in social networks. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, pp 467–476. ACM, Vancouver Lappas T, Liu K, Terzi E (2009) Finding a team of experts in social networks. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, pp 467–476. ACM, Vancouver
Zurück zum Zitat Malhotra R, Jain A (2011) Software effort prediction using statistical and machine learning methods. Int J Adv Comput Sci Appl 2(1):1451–1521 Malhotra R, Jain A (2011) Software effort prediction using statistical and machine learning methods. Int J Adv Comput Sci Appl 2(1):1451–1521
Zurück zum Zitat Michaeli R, Simon L (2008) An illustration of Bayes’ theorem and its use as a decision-making aid for competitive intelligence and marketing analysts. Eur J Market 42(7–8):804–813 Michaeli R, Simon L (2008) An illustration of Bayes’ theorem and its use as a decision-making aid for competitive intelligence and marketing analysts. Eur J Market 42(7–8):804–813
Zurück zum Zitat Morris PA (1974) Decision analysis expert use. Manag Sci 20(9):1233–1241MATH Morris PA (1974) Decision analysis expert use. Manag Sci 20(9):1233–1241MATH
Zurück zum Zitat Mostaghimi M (1996) Combining ranked mean value forecasts. Eur J Oper Res 94(3):505–516CrossRefMATH Mostaghimi M (1996) Combining ranked mean value forecasts. Eur J Oper Res 94(3):505–516CrossRefMATH
Zurück zum Zitat Mostaghimi M (1997) Bayesian estimation of a decision using information theory. IEEE Trans Syst Man Cybern 27(4):506–517CrossRef Mostaghimi M (1997) Bayesian estimation of a decision using information theory. IEEE Trans Syst Man Cybern 27(4):506–517CrossRef
Zurück zum Zitat Othman A, Sandholm T (2010) Decision rules and decision markets. In: Proceedings of the 9th international conference on autonomous agents and multiagent systems, Vol 1, pp 625–632. International Foundation for Autonomous Agents and Multiagent Systems Othman A, Sandholm T (2010) Decision rules and decision markets. In: Proceedings of the 9th international conference on autonomous agents and multiagent systems, Vol 1, pp 625–632. International Foundation for Autonomous Agents and Multiagent Systems
Zurück zum Zitat Reggiani P, Weerts AH (2008) A Bayesian approach to decision-making under uncertainty: an application to real-time forecasting in the river rhine. J Hydrol 356(1):56–69CrossRef Reggiani P, Weerts AH (2008) A Bayesian approach to decision-making under uncertainty: an application to real-time forecasting in the river rhine. J Hydrol 356(1):56–69CrossRef
Zurück zum Zitat Sprenger T, Bolster P, Venkateswaran A (2012) Conditional prediction markets as corporate decision support systems–an experimental comparison with group deliberations. J Predict Market 1(3):189–208 Sprenger T, Bolster P, Venkateswaran A (2012) Conditional prediction markets as corporate decision support systems–an experimental comparison with group deliberations. J Predict Market 1(3):189–208
Zurück zum Zitat Wu Q, Hao J-K (2012a) Coloring large graphs based on independent set extraction. Comput Oper Res 39(2):283–290 Wu Q, Hao J-K (2012a) Coloring large graphs based on independent set extraction. Comput Oper Res 39(2):283–290
Zurück zum Zitat Wu Q, Hao J-K (2012b) An effective heuristic algorithm for sum coloring of graphs. Comput Oper Res 39(7):1593–1600 Wu Q, Hao J-K (2012b) An effective heuristic algorithm for sum coloring of graphs. Comput Oper Res 39(7):1593–1600
Zurück zum Zitat Wu Q, Hao J-K (2013) An adaptive multistart tabu search approach to solve the maximum clique problem. J Comb Optim 26(1):86–108 Wu Q, Hao J-K (2013) An adaptive multistart tabu search approach to solve the maximum clique problem. J Comb Optim 26(1):86–108
Zurück zum Zitat Zhang H, Horvitz E, Chen Y, Parkes DC (2012) Task routing for prediction tasks. In Proceedings of the 11th international conference on autonomous agents and multiagent systems, Vol 2, pp 889–896. International Foundation for Autonomous Agents and Multiagent Systems Zhang H, Horvitz E, Chen Y, Parkes DC (2012) Task routing for prediction tasks. In Proceedings of the 11th international conference on autonomous agents and multiagent systems, Vol 2, pp 889–896. International Foundation for Autonomous Agents and Multiagent Systems
Metadaten
Titel
Team selection for prediction tasks
verfasst von
MohammadAmin Fazli
Azin Ghazimatin
Jafar Habibi
Hamid Haghshenas
Publikationsdatum
01.02.2016
Verlag
Springer US
Erschienen in
Journal of Combinatorial Optimization / Ausgabe 2/2016
Print ISSN: 1382-6905
Elektronische ISSN: 1573-2886
DOI
https://doi.org/10.1007/s10878-014-9784-3

Weitere Artikel der Ausgabe 2/2016

Journal of Combinatorial Optimization 2/2016 Zur Ausgabe

Premium Partner