Robust Aggregation of Inconsistent Information: Concepts and Research Directions

Improving the reliability of information obtained from multiple sources in an automated way is today a critical need as decision-making processes and other applications heavily depend on such information. The sources can be sensors used in a wireless sensor network, multiple resources on the Internet or a large number of users of a web service or a social network. Examples of such a problem include finding a reliable rating of a product or a movie, the accurate current price of a stock or a reliable assessment of the prevailing market analysts’ sentiment about a stock. The problem arises from the fact that multiple sources often provide inconsistent information, due to differences of opinion, human or hardware errors, being out of date and, most importantly, the sources might even maliciously supply false information with an express intent to deceive. While it is clear that no aggregation procedure can strictly guarantee the accuracy of the output, in practice we must seek “the best” answer possible—the one that in the given circumstances minimizes the error or the likelihood of an error of the aggregate value. Such procedures are needed for both numerical and non-numerical data and, given the vast amount of data becoming available, should operate without a need for human judgment or an external “gold standard”. In this paper we provide a survey of solutions to this problem based on iterative filtering approaches that take into account not only the information but also the information sources and are able to assess the credibility of the information and the trustworthiness of the information sources. We also discuss future work and open research directions.