Introduction
-
We highlight the importance of machine learning classifiers in modeling and predicting individual’s smartphone usage for personalized context-aware mobile services and systems.
-
We do modeling and predicting personalized diverse activities with their phones in relevant multi-dimensional contexts using various machine learning classification techniques utilizing their own phone log data.
-
We have conducted a range of experiments on the real phone usage datasets of individual users to evaluate the effectiveness of each classifier based prediction model on unseen contextual test cases.
Machine learning classifiers: background and related work
ZeroR
Ripple Down Rule learner (RIDOR)
Repeated incremental pruning to produce error reduction (RIPPER)
K-nearest neighbors (KNN)
Naive Bayes (NB)
Logistic regression (LR)
Support Vector Machines (SVM)
Decision tree (DT)
Random forest (RF)
Adaptive boosting (AdaBoost)
Artificial neural network (ANN) and deep learning
Classifiers | Purposes | References |
---|---|---|
ZeroR | Instant messaging | Fetter et al. [22] |
RIDOR | Activity recognition, notification management, location prediction, interruptibility prediction | |
RIPPER | Location prediction | Anagnostopoulos et al. [23] |
KNN | Mobile search, recommender system, location prediction, activity recognition, interruptibility prediction | |
NB | Phone call prediction, location prediction, interruption management | |
LR | Activity recognition, user modeling, recommendation system, health analytics, interruptibility prediction | |
SVM | Instant messaging, transportation system, activity recognition, notification management, interruptibility prediction | |
DT | Context-aware system, mobile service, interruption management, interruptibility prediction, phone call prediction | |
RF | Call availability prediction, instant messaging, transportation system, activity recognition, interruptibility prediction | |
AdaBoost | Interruption management, interruptibility prediction, location prediction, recommeder system | |
ANN | Smartphone power modeling, mobile credit card payment, mobile commerce, mobile learning, smartphone characterization |
Contexts and context-aware mobile services
-
Temporal context Temporal context is one of the primary context having influence on phone call activities of an individual user. For instance, one’s mobile phone activities in the morning may not be similar in the evening or night [65]. As each user activity is associated with a particular timestamp (e.g., YYYY-MM-DD hh:mm:ss) in the real world, temporal context plays a primary role to model individuals’ mobile phone usage behavior. Halvey et al. [66] have shown through the analysis of a large sample of user data that time-of-the-week is an important factor in modeling mobile user behavior. For instance, the specific date (e.g., YYYY-MM-DD) of an activity, days-of-the-week (Monday, Tuesday, ..., Sunday) to occur that activity, the specific or exact time-of-the-day (e.g., hh:mm:ss) are considered as the temporal context. In addition to these primary temporal information, the additional temporal information such as public holidays, weekdays, weekends, might have the influence on mobile phone users to make decisions.
-
Spatial context Spatial context is another significant user’s context that can be used to model and predict individual’s mobile phone activities more effectively [67, 68]. The reason is that phone call activities of an individual can also be treated as a location based service. For instance, one’s activity at office may not be similar with her activities at home. Thus, understanding user mobility in their daily life is a fundamental issue for the context-aware applications that is able to provide location based mobile services for the benefit of individuals. With the increasing pervasiveness and context-awareness of smart mobile phones over the past few years, a number of emerging location based mobile applications are adopted by the users. The reasons that makes user current location or spatial context in mobile phone applications so popular are; location based mobile services rely on the knowledge about the user’s geographical availability to obtain relevant information on the area or spot, and the user behaves accordingly under that particular spatial context [69]. Thus, we take into account individual’s location as a spatial context in our analysis.
-
Social context In addition to the above spatio-temporal, social contexts have also an influence on individual mobile phone users to take decisions [70]. In the real world, individuals are engaged in various social activities such as professional meeting, seminar, lecture etc. People are well differ from each other in how they behave with their mobile phones during various events, which has been shown in Sarker et al. [71]. For instance, one individual mobile phone user may be happy to accept the incoming phone calls during a professional meeting, whereas another individual may not be wanted to do the similar activities during that event [72]. Even the activity of a particular individual may be different subject to what type of event is occurred [73]. For example, an individual’s phone call activity or response during a ‘professional meeting’ may be well different from her activity during a ‘lunch-break’ event. The social context, such as the relationships between individuals, e.g., family, friend, work or professional, romantic, or unknown, have a strong influence on individual mobile phone users to take such phone call handling decision in such events [70, 74]. For instance, an individual typically ‘declines’ an incoming phone call during an event official meeting, however, she ‘answers’ if the incoming call comes from her ‘boss’. Thus the social relational bonding between the caller and the callee has a strong influence to handling call response decision. We also use such social relational context to make the prediction model more effective.
Methodology: context-aware predictive modeling
Dataset exploration
Contextual data processing
-
Context discretization Real-world mobile phone data contains continuous time-series data. However, such data is not directly applicable to build the prediction model. Thus, in order to convert such continuous data into nominal values, we use our earlier behavior-oriented time-series segmentation (BOTS) approach [75] to process this temporal context. In this approach, we first generate initial time slices using a base period as the finest granularity required to distinguish day-to-day activities of an individual user. After that, we aggregate adjacent slices dynamically with the same dominant characteristics in order to generate behavior-oriented segments. We also measure the corresponding applicability that takes into account not only the support value but also the temporal coverage to finalize the optimal segments. Thus, by using this approach, in this work, we generate a number of optimal temporal segments based on the similar activities of each individual user at various time-of-the day, and days-of-the-week. We also take into account day-wise segmentation as individual’s activity may not similar in all days-of-the-week, may differ from day-to-day. For instance, one’s Monday’s activities may not be similar with her Friday’s activities because of various day-to-day situations in the real world life. The total number of time segments generated by this approach is dynamic, depends on the number of the similar activity patterns during 24-h a day. An example of such generated segments are Friday [09:00–11:15], Monday [12:30–13:30], Saturday [15:30–18:45] etc. In addition to this temporal context, we also use the nominal values of user locations that are available in the dataset [76]. An example of such spatial contexts are home, office, market, MIT, Harvard, John Street etc.
-
Data-centric social context generation In addition to above spatio-temporal context, we also use user individual’s unique phone number available in the dataset as a social relational context. The reason is that it is very difficult to make the device understandable about the semantic relational context, such as family, friend or others. Moreover, users activities may not be similar for all her friends or all the family members or others. For instance, one’s phone call response behavior may vary between ‘best friend’ and ‘close friend’, even though both persons represent same relationship category ‘friend’. Thus, it may differ from person to person in the real world. Therefore, we generate the data-centric social relational context from the mobile phone data to achieve our goal. The main principle to generate the value of data-centric social context is “each unique mobile phone number represents a particular one-to-one relationship” [10]. For instance, best friend’s phone number (03...034567) represents one relation (Rel01), while close friend’s phone number (03...044587) represents another relation (Rel02) etc. Similarly, boss’s phone number (02...034211) represents another different relation (Rel03), which is also unique comparing to others. Such data-centric context values can play a significant role for personalized context-aware mobile services.
Machine learning classifier-based predictive modeling
-
Smartphone usage or activity class label The standard methodology comprises in taking in a classifier from a named dataset, to anticipate the class of new examples [14]. In this work, we take into account users’ diverse phone call activities as different class labels to achieve our goal. For this, we take into account all the diverse phone call activities or usage, as different distinct class labels to model individual’s mobile phone activity. For instance, individual’s phone call activities, such as answering the call, decline the call, missed call, and making an outgoing call, are represented as distinct classes; Class 1, Class 2, Class 3, and Class 4 respectively.
-
Non-class attributes or contextual features These are independent in the data set, which are used to predict user’s such mobile phone activities. These are also known as contexts such as temporal context (e.g., day, time), spatial context (e.g., user location), and social context (e.g., social relationship), discussed earlier, in which the class levels are dependent.
-
Training dataset The training data set in machine learning is the actual dataset used to train the classification model for performing various predictive services relevant to the features involved. We use a major part of contextual mobile phone data discussed above for building the activity model. In order to build the model, we randomly partition the initial data into k mutually exclusive subsets or “folds”, \(d_1,d_2,...,d_k\), each of which has an approximately equal size. Then, in each iteration i, we use all the instances of all partitions except the partition \(d_i\) as the training dataset to build the activity model.
-
Testing dataset This is the data typically used to provide an unbiased evaluation of the activity prediction model that is learned using the training dataset mentioned above. Actually, testing data is used for testing the model whether it is predicting the class label appropriately or not, for a particular context. This is done by using a k fold cross validation. In k fold cross-validation, we use the partitioned data \(d_i\) as testing dataset in each iteration i.
Evaluation and experimental results
Smartphone datasets
Evaluation metric
-
Precision, recall/sensitivity and ROC Precision is a measure of exactness, which represents the ratio between the number of phone call activities that are correctly predicted and the total number of activities that are predicted (both correctly and incorrectly). On the other hand, recall is a measure of completeness, which represents the ratio between the number of phone call activities that are correctly predicted and the total number of activities that are relevant. Recall also represents the Sensitivity [26]. These are calculated using the values of true positive rate, false positive rate, and false negative rate in prediction results. receiver operating characteristic (ROC) [26] can be used as another metric that also features on true positive rate, and false positive rate, to evaluate the classifier output quality. If TP, FP, and FN denote true positives, and false positives, and false negatives, then the formal definition of precision and recall are [26]:$$\begin{aligned} Precision = \frac{TP}{TP + FP} \end{aligned}$$(1)$$\begin{aligned} Recall= \frac{TP}{TP + FN} \end{aligned}$$(2)
-
F-measure and kappa F-measure is a metric that combines precision and recall in a single score, which is the harmonic mean of precision and recall. Kappa is a metric that compares an observed accuracy with an expected accuracy (random chance). The formal definition of F-measure and Kappa are [26]:$$\begin{aligned} F\hbox {-}measure= 2 * \frac{Precision * Recall}{Precision + Recall} \end{aligned}$$(3)$$\begin{aligned} Kappa= \frac{(observed \; accuracy - expected \; accuracy)}{(1 - expected \; accuracy)} \end{aligned}$$(4)
-
MAE and RMSE mean absolute error (MAE) and Root Mean Squared Error (RMSE) are used to calculate the error rate of each classifier based model, which represents the success of prediction. If the predicted values on the test instances are \(p_1, p_2, ...,p_n\), and the actual values are \(a_1, a_2, ...,a_n\), for n data points, then MAE and RMSE are formally defined as below [26]:$$\begin{aligned} MAE= \frac{1}{n}\sum _{i=1}^n \left| p_i-a_i \right| \end{aligned}$$(5)$$\begin{aligned} RMSE= \sqrt{\frac{1}{n}\sum _{i=1}^n \left( p_i - a_i \right) ^{2}} \end{aligned}$$(6)
Experimental results and effectiveness analysis
Dataset | Classifier | Context-aware prediction results | ||||
---|---|---|---|---|---|---|
CCI (%) | ICI (%) | MAE | RMSE | ROC | ||
DS02 | ZeroR | 60.52 | 39.47 | 0.28 | 0.37 | 0.49 |
NB | 80.25 | 19.74 | 0.14 | 0.26 | 0.93 | |
AdaBoost | 62.12 | 37.87 | 0.34 | 0.40 | 0.58 | |
LR | 84.35 | 15.64 | 0.09 | 0.22 | 0.95 | |
DT | 85.13 | 14.86 | 0.08 | 0.21 | 0.96 | |
RF | 85.37 | 14.62 | 0.08 | 0.21 | 0.96 | |
RIPPER | 73.53 | 26.46 | 0.20 | 0.32 | 0.70 | |
SVM | 82.61 | 17.38 | 0.26 | 0.33 | 0.86 | |
KNN | 81.48 | 18.51 | 0.11 | 0.25 | 0.93 | |
RIDDOR | 67.87 | 32.12 | 0.16 | 0.40 | 0.70 |
Dataset | Classifier | Context-aware prediction results | ||||
---|---|---|---|---|---|---|
CCI (%) | ICI (%) | MAE | RMSE | ROC | ||
DS05 | ZeroR | 58.36 | 41.63 | 0.28 | 0.37 | 0.49 |
NB | 77.39 | 22.60 | 0.16 | 0.27 | 0.90 | |
AdaBoost | 58.36 | 41.63 | 0.26 | 0.36 | 0.61 | |
LR | 79.77 | 20.22 | 0.11 | 0.25 | 0.93 | |
DT | 82.28 | 17.71 | 0.10 | 0.23 | 0.94 | |
RF | 84.66 | 15.33 | 0.10 | 0.23 | 0.95 | |
RIPPER | 72.89 | 27.10 | 0.20 | 0.32 | 0.73 | |
SVM | 80.54 | 19.45 | 0.27 | 0.34 | 0.84 | |
KNN | 79.43 | 20.56 | 0.14 | 0.27 | 0.91 | |
RIDDOR | 60.38 | 39.61 | 0.19 | 0.44 | 0.67 |
Effectiveness comparison
Random forest vs. baseline classifier
Random forest vs. decision tree
Random forest vs. other ML classifiers
Effectiveness comparison with neural network classification model
Dataset | The number of epochs | Context-aware prediction results | ||||
---|---|---|---|---|---|---|
CCI (%) | ICI (%) | MAE | RMSE | ROC | ||
DS02 | 10 | 65.39 | 34.60 | 0.21 | 0.34 | 0.80 |
20 | 67.20 | 32.79 | 0.17 | 0.33 | 0.83 | |
30 | 63.88 | 36.11 | 0.18 | 0.38 | 0.76 | |
40 | 65.23 | 33.76 | 0.17 | 0.37 | 0.79 | |
50 | 64.45 | 35.54 | 0.18 | 0.41 | 0.72 | |
60 | 63.45 | 36.54 | 0.18 | 0.41 | 0.70 | |
70 | 62.24 | 37.75 | 0.19 | 0.42 | 0.67 | |
80 | 61.24 | 38.75 | 0.19 | 0.42 | 0.67 | |
90 | 60.24 | 39.75 | 0.20 | 0.43 | 0.62 | |
100 | 58.71 | 41.28 | 0.21 | 0.44 | 0.59 |
Dataset | The number of epochs | Context-aware prediction results | ||||
---|---|---|---|---|---|---|
CCI (%) | ICI (%) | MAE | RMSE | ROC | ||
DS05 | 10 | 63.97 | 36.02 | 0.22 | 0.34 | 0.77 |
20 | 65.25 | 34.74 | 0.20 | 0.33 | 0.80 | |
30 | 66.42 | 33.57 | 0.18 | 0.32 | 0.80 | |
40 | 65.41 | 34.58 | 0.18 | 0.37 | 0.78 | |
50 | 62.89 | 37.10 | 0.19 | 0.41 | 0.72 | |
60 | 60.04 | 39.95 | 0.20 | 0.43 | 0.65 | |
70 | 60.71 | 39.28 | 0.20 | 0.43 | 0.68 | |
80 | 62.71 | 37.28 | 0.19 | 0.42 | 0.69 | |
90 | 63.13 | 36.86 | 0.18 | 0.41 | 0.72 | |
100 | 59.24 | 40.75 | 0.21 | 0.43 | 0.67 |