Emiliano Miluzzo
ABSTRACT
This paper shows that the location of screen taps on modern smartphones and tablets can be identified from accelerometer and gyroscope readings. Our findings have serious implications, as we demonstrate that an attacker can launch a background process on commodity smartphones and tablets, and silently monitor the user's inputs, such as keyboard presses and icon taps. While precise tap detection is nontrivial, requiring machine learning algorithms to identify fingerprints of closely spaced keys, sensitive sensors on modern devices aid the process. We present TapPrints, a framework for inferring the location of taps on mobile device touch-screens using motion sensor data combined with machine learning analysis. By running tests on two different off-the-shelf smartphones and a tablet computer we show that identifying tap locations on the screen and inferring English letters could be done with up to 90% and 80% accuracy, respectively. By optimizing the core tap detection capability with additional information, such as contextual priors, we are able to further magnify the core threat.
- S. Agrawal, I. Constandache, S. Gaonkar, R. Roy Choudhury, K. Caves, and F. DeRuyter. Using Mobile Phones to Write in Air. In Proceedings of the 9th international conference on Mobile systems, applications, and services, pages 15--28. ACM, 2011. Google Scholar
Digital Library
- M. Azizyan, I. Constandache, and R. Roy Choudhury. Surroundsense: Mobile Phone Localization via Ambience Fingerprinting. In Proceedings of the 15th annual international conference on Mobile computing and networking, pages 261--272. ACM, 2009. Google ScholarDigital Library
- R. Becker, R. Cáceres, K. Hanson, J. Loh, S. Urbanek, A. Varshavsky, and C. Volinsky. A Tale of One City: Using Cellular Network Data for Urban Planning. IEEE Pervasive Computing, Vol. 10, No. 4, October-December 2011, 2011. Google ScholarDigital Library
- S. Block and A. Popescu. Device Orientation Event Specification. W3C, Draft 12 July 2011.Google Scholar
- L. Breiman. Random Forests. In Machine Learning, volume 45(1), 2001. Google ScholarDigital Library
- L. Cai and H. Chen. Touchlogger: Inferring Keystrokes on Touch Screen from Smartphone Motion. In Proceedings of the 6th USENIX conference on Hot topics in security (HotSec'11). USENIX Association, Berkeley, CA, USA, pages 9--9, 2011. Google ScholarDigital Library
- E. Owusu, J. Han, S. Das, A. Perrig and J. Zhang. ACCessory: Password Inference using Accelerometers on Smartphones. In Proceedings of the 13th Workshop on Mobile Computing Systems and Applications (HotMobile'12). San Diego, CA, USA, 20121. Google ScholarDigital Library
- L. Cai, S. Machiraju, and H. Chen. Defending Against Sensor-Sniffing Attacks on Mobile Phones. In Proceedings of the 1st ACM workshop on Networking, systems, and applications for mobile handhelds, 2009. Google ScholarDigital Library
- R. Caruana, A. Niculescu-Mizil, G. Crew, and A. Ksikes. Ensemble Selection from Libraries of Models. In Proceedings of the twenty-first international conference on Machine learning, ICML '04, pages 18--, New York, NY, USA, 2004. ACM. Google ScholarDigital Library
- T. G. Dietterich. Ensemble Methods in Machine Learning. In Multiple Classifier Systems, pages 1--15, 2000. Google ScholarCross Ref
- P. Domingos. Bayesian Averaging of Classifiers and the Overfitting Problem. In In Proceedings 17th International Conference on Machine Learning, pages 223--230. Morgan Kaufmann, 2000. Google ScholarDigital Library
- M. Egele, C. Kruegel, E. Kirda, and G. Vigna. Pios: Detecting Privacy Leaks in iOS Applications. In Proceedings of the Network and Distributed System Security Symposium, 2011.Google Scholar
- W. Enck, P. Gilbert, B. Chun, L. Cox, J. Jung, P. McDaniel, and A. Sheth. Taintdroid: an Information-Flow Tracking System for Realtime Privacy Monitoring on Smartphones. In Proceedings of the 9th USENIX conference on Operating systems design and implementation, pages 1--6. USENIX Association, 2010. Google ScholarDigital Library
- D. Foo Kune and Y. Kim. Timing Attacks on Pin Input Devices. In Proceedings of the 17th ACM conference on Computer and communications security (CCS '10), 2010. Google ScholarDigital Library
- M. Jahrer, A. Töscher, and R. Legenstein. Combining Predictions for Accurate Recommender Systems. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '10, pages 693--702, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- N. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. Campbell. A Survey of Mobile Phone Sensing. Communications Magazine, IEEE, 48(9):140--150, 2010. Google ScholarDigital Library
- H. Lu, J. Yang, Z. Liu, N. Lane, T. Choudhury, and A. Campbell. The Jigsaw Continuous Sensing Engine for Mobile Phone Applications. In Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems, pages 71--84. ACM, 2010. Google ScholarDigital Library
- V. M. and P. S. Compromising Electromagnetic Emanations of Wired and Wireless Keyboards. In Proceedings of the 18th conference on USENIX security symposium, 2009. Google ScholarDigital Library
- P. Marquardt, A. Verma, H. Carter, and P. Traynor. (sp)iphone: Decoding Vibrations from Nearby Keyboards Using Mobile Phone Accelerometers. In Proceedings of the 18th ACM conference on Computer and communications security, pages 551--562. ACM, 2011. Google ScholarDigital Library
- P. McCullagh and J. A. Nelder. Generalized Linear Models (Second edition). London: Chapman & Hall, 1989.Google Scholar
- S. McKinley and M. Levine. Cubic Spline Interpolation. College of the Redwoods, 1998.Google Scholar
- E. Miluzzo, N. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. Eisenman, X. Zheng, and A. Campbell. Sensing Meets Mobile Social Networks: the Design, Implementation and Evaluation of the CenceMe Application. In Proceedings of the 6th ACM conference on Embedded network sensor systems, pages 337--350. ACM, 2008. Google ScholarDigital Library
- B. Pinkas and T. Sander. Securing Passwords Against Dictionary Attacks. In Proceedings of the 9th ACM Conference on Computer and Communications Security, pages 161--170. ACM, 2002. Google ScholarDigital Library
- M. Poh, K. Kim, A. Goessling, N. Swenson, and R. Picard. Cardiovascular Monitoring Using Earphones and a Mobile Device. Pervasive Computing, IEEE, (99):1--1, 2011. Google ScholarDigital Library
- R. Schlegel, K. Zhang, X. Zhou, M. Intwala, A. Kapadia, and X. Wang. Soundcomber: a Stealthy and Context-Aware SoundTrojan for Smartphones. In Proceedings of the 18th Annual Network and Distributed System Security Symposium (NDSS '11), 2011.Google Scholar
- B. Schoelkopf, C. Burges, and A. Smola. Advances in Kernel Methods - Support Vector Learning. MIT Press, 1998. Google ScholarDigital Library
- L. Zhuang, F. Zhou, and J. D. Tygar. Keyboard Acoustic Emanations Revisited. ACM Trans. Inf. Syst. Secur., 2009. Google ScholarDigital Library
- K. Killourhy and R. Maxion. Comparing Anomaly-Detection Algorithms for Keystroke Dynamics. In Dependable Systems & Networks, 2009. DSN'09. IEEE/IFIP International Conference on, pages 125--134. IEEE, 2009.Google ScholarCross Ref
Index Terms
- Tapprints: your finger taps have fingerprints
Recommendations
Single-stroke language-agnostic keylogging using stereo-microphones and domain specific machine learning
WiSec '14: Proceedings of the 2014 ACM conference on Security and privacy in wireless & mobile networksMobile phones are equipped with an increasingly large number of precise and sophisticated sensors. This raises the risk of direct and indirect privacy breaches. In this paper, we investigate the feasibility of keystroke inference when user taps on a ...
Hobson's Choice: Security and Privacy Permissions in Android and iOS Devices
Proceedings of the Third International Conference on Human Aspects of Information Security, Privacy, and Trust - Volume 9190The use of smartphones and tablet devices has grown rapidly over recent years and the widespread availability of software, often from unknown developers, has led to security and privacy concerns. In order to prevent security compromises, these devices ...
A standard for developing secure mobile applications
The abundance of mobile software applications (apps) has created a security challenge. These apps are widely available across all platforms for little to no cost and are often created by small companies and less-experienced programmers. The lack of ...
Comments