Reliable social sensing with physical constraints: analytic bounds and performance evaluation

Correctness guarantees are at the core of cyber-physical computing research. While prior research addressed correctness of timing behavior and correctness of program logic, this paper tackles the emerging topic of assessing correctness of input data. This topic is motivated by the desire to crowd-source sensing tasks, an act we henceforth call social sensing, in applications with humans in the loop. A key challenge in social sensing is that the reliability of sources is generally unknown, which makes it difficult to assess the correctness of collected observations. To address this challenge, we adopt a cyber-physical approach, where assessment of correctness of individual observations is aided by knowledge of physical constraints on sources and observed variables to compensate for the lack of information on source reliability. We cast the problem as one of maximum likelihood estimation. The goal is to jointly estimate both (i) the latent physical state of the observed environment, and (ii) the inferred reliability of individual sources such that they are maximally consistent with both provenance information (who reported what) and physical constraints. We also derive new analytic bounds that allow the social sensing applications to accurately quantify the estimation error of source reliability for given confidence levels. We evaluate the framework through both a real-world social sensing application and extensive simulation studies. The results demonstrate significant performance gains in estimation accuracy of the new algorithms and verify the correctness of the analytic bounds we derived.