Introduction
Related work
Implementations of eye and mouse tracking
- Eye tracking based emphatic software agent (ESA), an eye tracking software that captures the state of awareness of the learners and responds accordingly [24].
- Enhanced exploitation of eyes for effective e-learning (e5Learning) [25].
- Eye tracking based adaptive and personalized e-Learning Systems (AeLS) [26].
- Eye tracking based Eye tracking based programming tutoring system (Protus) [27].
- A mouse tracking web application developed by Zushi et al. [9] for their own specific learning management system (LMS).
- Open Gaze and Mouse Analyzer (OGAMA), an open-source software designed to analyze eye and mouse movements in slideshow study designs [33].
- Mousetrap, an integrated, open-source mouse-tracking package on OpenSesame for laboratory experiments [34].
- Known commercial applications: Lucky Orange, Hotjar, Crazy Egg, Fullstory, Ptengine, Heatmap.com, Smartlook, ContentSquare, SessionCam, Seevolution, Capturly, Inspectlet, MouseFlow, Clicktale, and Tamboo [35].
Mouse tracking in web development
- Mousedown: when either one of the mouse buttons are pressed (usually left, middle, or right button)
- Mouseup: when either pressed mouse buttons are released
- Mousemove: when the mouse cursor moves
- Mouseleave: when the mouse leaves an element (we only indicate when temporary leaving a webpage)
- Mouseenter: when the mouse enters an element (we only indicate when temporary entering a webpage)
- Beforeunload: when the webpage almost closes
- Scroll: when the webpage scrolls
- Touchstart: when a computer device screen is touching
- Touchend: when a touch from touchstart is removed
- Touchmove: when a touch is moving
- Touchcancel: when a touch is interrupted
- Resize: when the webpage is zoomed in or out
Default eye and mouse tracking generates big data
Reducing eye and mouse tracking data
Method and simulation
System overview
- NodeJS is browser JavaScript made into server side programming language. Also, NodeJS is written based on the criticism in 2009 about how Apache HTTP server handled huge concurrent user, sequential programming, and blocking functions [51] while NodeJS is asynchronous and is designed to build scalable network application. Additionally, its runtime is built on Chrome’s V8 Engine which implements C++ features such as hidden classes and inline caching to make JavaScript runs much faster [52]. The popular web framework for NodeJS is Express which is a fast, unopinionated, minimalist web framework for Node.js [53]. The choice for implementing mouse tracking code can either be using developed module available on Node Package Manager (NPM), use TypeScript, or the default option. TypeScript is a typed superset of JavaScript developed by Microsoft that compiles to plain JavaScript. The advantage for developers are defining interface between software components, and interactive static checking and code refactoring during development [54]. The default option is to call the scripts in the webpage layout which is usually written in Jade or Pug.
- Django is a web framework written in Python that uses model-view template (MVT) [55]. Like Python almost every module is available, Django prides itself as a batteries-included framework, meaning that it comes with many modules unlike other frameworks, it is not necessary for a developer to write a module from a scratch. Although it is powerful for building huge web applications, the difficulty in building huge application doesn’t change when building small applications. For mouse tracking, there is a choice to use Python modules but it is not yet known whether it can interact with the DOM elements in the webpage. Most documentation suggests to use vanilla JavaScript in Django.
- Rails is a model-view controller (MVC) web framework written in Ruby. Its philosophies are “convention over control” and “don’t repeat yourself” where in the 2000s introduced seamless database table creations, migrations, and scaffolding of views to enable rapid application development, even other web frameworks took ideas from Rails. There are few options other than JavaScript in implementing mouse tracking code which are coffee script (JavaScript coding made simpler) and jQuery which can be installed from Ruby’s package manager GEM. They are one of the first to introduce unobtrusive JavaScript where it should not be mixed in the HTML file [56].
- Laravel is also an MVC web framework but written in PHP and based on Symfony. Laravel values elegance, simplicity, and readability. The mouse tracking code can be written in JavaScript and placed in the asset directory. Laravel Mix is the tool for compiling those assets but the default method is also available [57].
- Spring is an application framework and inversion of control container for the Java platform. The framework’s core features can be used by any Java application, but there are extensions for building web applications on top of the Java Enterprise Edition platform. Java is one of the earliest programming language used to make an application and it’s still popular today. Java has its own bindings to connect to the DOM API.
- ReactJS is a JavaScript library for building UI which are maintained by Facebook and community. Unlike the previous back-end web framework, ReactJS is a front-end web framework. ReactJS have its own mouse event library which is to be injected on each UI [58].
- Angular is a complete rewrite to TypeScript based from the same team that built AngularJS. It is a web framework mainly maintained by Google and by a community of individuals and corporations to address many of the challenges encountered in developing single-page applications. It is one of the most popular framework to build web applications on mobile. The mouse events can be added on the components or templates [59].
Three techniques of mouse tracking
Events | Mouse tracking | Page tracking | ROI tracking |
---|---|---|---|
Duration |
\(\checkmark\)
|
\(\checkmark\)
|
\(\checkmark\)
|
Click Left |
\(\checkmark\)
|
\(\checkmark\)
|
\(\checkmark\)
|
Click Right |
\(\checkmark\)
|
\(\checkmark\)
|
\(\checkmark\)
|
Click Middle |
\(\checkmark\)
|
\(\checkmark\)
|
\(\checkmark\)
|
Mouse X |
\(\checkmark\)
| ✗ | Partial |
Mouse Y |
\(\checkmark\)
| ✗ | Partial |
Touch X |
\(\checkmark\)
| ✗ | Partial |
Touch Y |
\(\checkmark\)
| ✗ | Partial |
Keyboard Type |
\(\checkmark\)
|
\(\checkmark\)
|
\(\checkmark\)
|
Scroll X |
\(\checkmark\)
| ✗ | Partial |
Scroll Y |
\(\checkmark\)
| ✗ | Partial |
Other Events |
\(\checkmark\)
|
\(\checkmark\)
|
\(\checkmark\)
|
Events Amount | ✗ |
\(\checkmark\)
|
\(\checkmark\)
|
Simulation
- Chrome version 80.0.3987.132 that uses V8 Engine.
- Firefox version 74.069 that uses SpiderMonkey Engine.
- Microsoft Edge version 80.0.361.62 that uses Chakra.
ID | Name | Date 2019/3/01 | Duration | Left click | Right click | Middle click | Mouse X | Mouse Y | Scroll X | Scroll Y |
---|---|---|---|---|---|---|---|---|---|---|
1 | Student 1 | 11:06:39 | 13.674 | False | False | False | 0 | 0 | 0 | 0 |
2 | Student 1 | 11:06:39 | 0.002 | False | False | False | 1197 | 317 | 0 | 0 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
286511 | Student 23 | 14:44:09 | 0.002 | False | False | False | 1009 | 179 | 0 | 0 |
Statistics | Default Mouse Tracking Simulation | ROI Tracking Simulation | Whole Page Tracking Simulation | ||||||
---|---|---|---|---|---|---|---|---|---|
CPU (%) | RAM (MB) | Data Rate (kB) | CPU (%) | RAM (MB) | Data Rate (kB) | CPU (%) | RAM (MB) | Data Rate (kB) | |
Minimum | 0 | 2.88 | 0 | 0 | 1.75 | 0 | 0 | 1.58 | 0 |
Maximum | 86 | 3.66 | 228.45 | 12 | 1.92 | 46.8 | 26 | 2.12 | 2.07 |
Median | 3 | 3.29 | 5.62 | 0 | 1.86 | 0 | 0 | 2.07 | 0 |
Average | 21.34 | 3.25 | 28.23 | 0.87 | 1.85 | 2.28 | 0.05 | 2.06 | 0.01 |
Standard deviation | 29.09 | 0.25 | 36.8 | 1.24 | 0.06 | 5.05 | 0.46 | 0.04 | 0.08 |
Name | Date 2019/3/01 | Duration (seconds) | Left clicks | Right clicks | Middle clicks | Mouse moves | Scrolls |
---|---|---|---|---|---|---|---|
Student 1 | 14:12:29 | 41 | 3 | 0 | 0 | 629 | 114 |
Student 2 | 14:44:09 | 90 | 7 | 0 | 0 | 1176 | 137 |
... | ... | ... | ... | ... | ... | ... | ... |
Student 22 | 11:55:14 | 2188 | 157 | 5 | 0 | 20912 | 6626 |
Student 23 | 11:57:37 | 2236 | 323 | 0 | 0 | 17982 | 6930 |
ID | Name | Date 2019/3/01 | Duration (second) | Area (x1,x2,y1,y2) | Left clicks | Right clicks | Middle clicks | Mouse moves | Scrolls |
---|---|---|---|---|---|---|---|---|---|
1 | Student 1 | 11:06:39 | 14.148 | {“header”:[0,1920,0,64]} | 0 | 0 | 0 | 1 | 1 |
2 | Student 1 | 11:06:40 | 1.179 | {“quiz1”:[529,1900,291,570]} | 0 | 0 | 0 | 86 | 0 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
19062 | Student 23 | 14:44:09 | 0.002 | {“title”:[16,1904,150,270]} | 0 | 0 | 0 | 1 | 0 |