Context pre-modeling: an empirical analysis for classification based user-centric context-aware predictive modeling

Context vectorization typically defines a good numerical measure to characterize a categorical context value. The vectors rather than direct contexts are used widely in machine learning because of the effectiveness and practicality of representing objects in a numerical way to help with many kinds of analyses. Thus, both the context transformation and context normalization are used to represent the contextual values into numerical values to feed into the machine learning technique.

Feature generation and extraction techniques typically provide a better understanding of the data, a way of improving prediction accuracy, as well as reducing computational cost or training time in a machine learning-based model or system [40]. Feature extraction typically is a process of combining existing features to produce a more useful one, also known as dimensionality reduction by which an initial set of raw data is reduced to more manageable groups for further processing [39]. For example, principal components analysis (PCA) can be used to extract a lower-dimensional space [12]. The key principle is that the extraction process takes into account creating brand new components based on the contextual information in the datasets. Several methods such as Principal Component Analysis (PCA), Singular Value Decomposition (SVD), and Linear Discriminant Analysis (LDA), etc. can be used for analyzing the significance of the contextual features [12, 39]. In our experiment, we take into account the PCA method, which is a popular and well-known feature extraction method used in the area of data science and machine learning. PCA method can produce new brand features or components by analyzing the characteristics of the contextual datasets [4]. Technically, PCA finds the eigenvectors of a covariance matrix with the highest eigenvalues and then uses those to project the data into a new subspace of equal or fewer dimensions. The below mathematical equations are relevant to the principal component analysis [4, 12].Where, the first principal component be a linear combination of X defined by coefficients or weights \(w = [w_1...w_n]\), and can be written in matrix form as \({\text {U}}_1 = {\text {w}}^T {\text {X}}\). Thus, in the scope of our study, we create principal components and calculate the variance of the components using this PCA method, considering the contexts and target behavioral activity of the users, to determine the significance of the contexts in a given dataset.

Contextual feature selection could be another way to resolve the issues of high dimensional contextual data, as well as to analyze the significance of the contextual features in a predictive model. It selects the most useful features to train on among existing features [39]. Feature selection techniques typically provide a better understanding of the data, a way of improving prediction accuracy, as well as reducing computational cost or training time in a machine learning-based model or system [40]. A contextual dataset may contain data with high dimensions, and some of them might not contain significant information for building a machine learning classification-based model. Moreover, further processing with all the given features or attributes using machine learning techniques might give poor prediction results because of the over-fitting problem [23, 41]. Thus, an optimal number of features by selecting contexts, is needed not only to reduce the computational cost but also to build a more effective context-aware model with a higher accuracy rate. For this, we can filter less significant, irrelevant or redundant context from the given dataset, by analyzing the data patterns and dependency. Several statistical methods such as chi-squared test, analysis of variance test, correlation coefficient analysis, etc. can be used for analyzing the significance of the given features [12, 40]. In our experiment, we take into account the correlation of the contexts, known as Pearson correlation coefficient [12], which is the most popular method for analyzing how each context is correlated with each other and with the target behavioral activity of the users. Correlation is a well-known similarity measures between two contextual features, and measures the strength between features and with behavioral activity class. The correlation-based feature selection is based on the following hypothesis: “Good feature subsets contain features highly correlated with the target class, yet uncorrelated or less correlated to each other”. If X and Y represent two random contextual variables, then the correlation coefficient between X and Y is defined as [12] In the field of statistics, the formula Eq. 8 is often used to determine how strong that relationship is between those two variables X and Y with values between \(-1\) and 1 considering both positive and negative correlation. Thus, in the scope of our study, we calculate the correlation coefficient values of each context and target behavioral activity of the users, to determine the significance of the contexts in a given dataset.

Naïve Bayesian (NB) [45] classification is one of the popular supervised learning techniques based on statistical probability. This classifier is widely used in various application areas because of its simplicity and easy to build. In many application areas, it gives significant prediction results with the test cases. A naive Bayesian contextual model is based on the most popular Bayes’ theorem in statistics, with the assumptions of independence between contextual features or predictors. Bayes’ theorem typically provides a way of calculating the posterior probability of an event and is stated mathematically as the following equation [12]:where, c is a class variable that represents user behavioral activity and P(c) is the class prior probability. \(X = \{x_1, x_2,..., x_n\}\) is a dependent feature vector of size n consists of contextual information, and P(X) is the prior probability of a contextual feature. P(c|X) represents the posterior probability of target class of the given contextual feature, while P(X|c) is the likelihood which is the probability of contextual feature of a given class.

Another classification algorithm, K-nearest neighbors (KNN) [46] is a simple and popular technique, in the area of machine learning, used in many application areas for analyzing the prediction problems. This is also called as a lazy learning algorithm because of its working procedure. This technique does not have a specialized training phase for building a model rather uses all the data instances while classification. It classifies new test cases based on a ‘feature similarity’ measure, such as a distance function, e.g., Euclidean distance [12]. A case is classified by a majority voting of its neighbors indicating as k values. Figure 1 shows an example to understand the concept of k in a KNN algorithm, considering different k values, such as \(k=3\) and \(k=6\). While building our context-aware model, we take into \(k=5\), as the number of neighbors.

Support vector machine (SVM) [47] classification is another popular supervised learning technique in machine learning, that can be used to build a context-aware predictive model. The computation concept behind this classifier is to find a hyperplane between the contextual data space, which best divides the dataset into two behavioral activity classes as shown in Fig. 2. The main principle to estimate the hyperplane is that it maximizes the margin between the two classes, and the vectors (cases) that define the hyperplane are the support vectors [12]. Overall, it works in two steps that include identifying the optimal hyperplane in contextual data space, and then to map the data instances according to the decision boundaries specified by the hyperplane for a given dataset.

In our context-aware model, we take into account several parameters including the kernel, while building the SVM based model. In the area of machine learning, kernel functions can be different types such as linear, poly, RBF, sigmoid, precomputed etc. [12, 39]. We use the Radial Basis Function (RBF), which is a popular kernel function used in various kernelized learning algorithms. In addition, the regularization parameter C, also known as the penalty parameter of the error term, which controls the trades-off the correct classification of training examples against the maximization of the decision function’s margin. The strength of the regularization is inversely proportional to C. Thus we use \(C=1.0\), considering the trade-off between achieving a low training error, and a low testing error in our SVM based context-aware model.

Decision tree (DT) [25] classification is a well-known supervised learning method, which is used widely for solving various prediction or classification problems in various application areas. A decision tree is a non-parametric supervised learning method that builds classification models in the form of a tree structure consisting of several nodes. It breaks down a contextual dataset into subsets and the associated tree is incrementally developed. It calculates entropy and information gain [25] while building the tree. In addition to entropy, “gini” function could be effective in a decision tree model [39]. An example of the decision tree nodes is shown in Fig. 3.

In our context-aware model, we take into account the best split at each node and use the “gini” function to measure the quality of a split in a decision tree model. In our decision tree model, nodes are expanded until all leaves are pure or until all leaves contain less than the minimum number of samples required to split an internal node, which is set as 2 in this experiment. We also set the minimum number of samples required to be at a leaf node as one. The maximum number of features are taken into account as the equal number of the contextual features in this experiment, to build our context-aware model.

The random forest (RF) [48] is an ensemble classification technique in machine learning consisting of multiple decision trees. It combines bootstrap aggregation (bagging) [49] and random feature selection [50] to construct a collection of decision trees exhibiting controlled variation. While training, each tree in a random forest contextual model learns from a random set of contextual data instances. The instances are drawn with replacement known as bootstrapping, which means some instances are used multiple times to build the tree. Overall, the entire random forest learner might have lower variance while building a model. After constructing the forest model, the prediction result is measured by taking into account the majority voting of the generated decision trees. Fig. 4 shows an example of a random forest structure considering several decision trees.

In our context-aware model, we take into account 100 trees while constructing the random forest. Although both the “gini” and “entropy” are the popular measures of impurity of a node [12, 39], we use “gini” function that represents the average gain of purity by splits of a given context to measure the quality of a split. We do not restrict the maximum depth while generating a tree, as we have not a huge number of contexts in this experiment. Thus nodes are expanded until all leaves are pure or until all leaves contain less than the minimum number of samples required to split an internal node, which is set as 2 in this experiment, mentioned in decision tree model as well. We also set the minimum number of samples required to be at a leaf node as one. The maximum number of features are taken into account as the square root of the total contextual features of this experiment, to build our context-aware model.

An artificial neural network (ANN) is mainly used for deep learning models. An ANN is comprised of a network of artificial neurons, also known as nodes of the network [12]. The nodes are connected by links and each link is associated with weight and they interact with each other in different layers. In this work, we consider a feed-forward artificial neural network consisting of several layers, such as the input layer, hidden layer, and output layer [12, 39]. In our ANN model, the size of the input layer has been chosen according to the number of selected contextual features, and the number of neurons in the output layer is equal to the number of classes. The three hidden layers with the neurons 100 have been carefully selected to build our contextual neural network model, as shown in Fig. 5.

In our context-aware model, we take into account three hidden layers with 100 neurons and compile the neural network with Adam optimizer [39]. While training the network, we use 1000 epochs with the batch size 200. We also use a small value of 0.001 as the learning rate as it allows the model to reach the global minimum. Regarding the activation function, we use the Rectified Linear Unit (ReLU) that overcomes the vanishing gradient problem, as well as allows the model to learn faster and perform better compared with other activation functions like the Sigmoid [39, 51]. We empirically set these hyperparameters to build our context-aware model using the artificial neural network.

We have collected individuals’ smartphone usage datasets consisting of different categories of smartphone apps such as social networking, instant messaging, mobile communications, entertainment, or other apps related to users’ daily life services in different contexts. The contexts are—temporal context such as time-of-the-day [24-h-a-day], days-of-the-week [7-days-a-week]; spatial context or user location such as at home, at the office, at the canteen, in the playground, on the way, etc; user work status such as workday or holiday; user diverse mood such as normal, happy, or sad; user device-related context such as battery level low, medium, or full; phone profile such as phone notification setting general, silent, or vibration; user Internet connectivity such as WiFi connectivity on or off. Different types of apps are Facebook, Gmail, LinkedIn, Instagram, Youtube video, Live Sport, Whatsapp, Internet browsing, watching movies, Skype, listening musics, reading news, playing games, etc. Datasets are collected from June 2018 to October 2018 from several participants for experimental purposes.

To evaluate the machine learning-based models, we employ the most popular K-fold cross-validation technique in machine learning [12]. In our evaluation, we use \(K = 10\) for generating train and test data to build a model, and measure the predicted accuracy that are defined as below:

As discussed above, the context values are category datatype that also considered as object type. Thus, we convert the context values into vectors to feed values into machine learning-based models. To achieve this goal, we first transform the context into numeric values. To do this, we use Label Encoder that transforms the context values into the desired numeric values. For instance, Internet connectivity on and off is transformed into 0 and 1. For more diverse values of contexts more numeric values are created. After performing encoding, we normalize the values using the Standard Scaler so that its distribution will have a mean value 0 and a standard deviation of 1. Table 1 shows the normalization value for some randomly selected instances of the dataset of user U1. Numerical values of different contexts shown in Table 1 are used to feed into machine learning-based models.

In this experiment, we show the outcome results of the principal component-based feature extraction model. For this, we first generate the principle components and their variance values. Figure 6 shows the cumulative graph considering all the principle components and their explained variances utilizing the datasets of user U1 and U2 respectively. The results are shown considering the variance with each component representing as 0, 1, 2, ....6 in the x-axis, where the variance value is between 0 and 1. If we observe Fig. 6, we see that, for each component, the variance graph increases linearly up to value 0.9. That means, all these generated components associated contain significant information to modeling.

In Tables 2 and 3, we have also shown the prediction results of the resultant context-aware models using various machine learning classification techniques utilizing the datasets of both the users U1 and U2. The techniques are random forest (RF), decision tree (DT), k-nearest neighbor (KNN), naive Bayes (NB), support vector machine (SVM), and artificial neural network (ANN) that are discussed briefly in earlier section. The results are shown by varying the variance threshold of v which is 90%, 70%, and 50%.

If we observe Tables 2 and 3, we can see that different threshold values give different results for each machine learning-based model. For 50%, it decreases the prediction results in terms of precision, recall, and f-score. The reason is that low cumulative variance value as a threshold, may lose the context information and consequently low prediction results. However, a larger value of threshold increases the number of components in the model, consequently, increases the computational complexity and decreases the prediction results in several cases depending on the dataset characteristics. Thus, we choose 90% as a threshold as the variance graph increases linearly up to 0.9, to output the prediction results. For another dataset, this value might be changed depending on their data characteristics and patterns. According to these results shown in Tables 2 and 3, we can conclude that the random forest classification based context-aware model performs comparatively better than other classification techniques for a particular chosen threshold of variance. Besides, to these prediction results, we also show the ROC values considering the random forest classification model utilizing the datasets of both the users U1 and U2 in Fig. 7. The reason for getting better results with component-based random forest model is that this model fits multiple decision trees with the generated feature components, and averages all the single tree results.

Overall, we can conclude from our experimental results is that feature extraction by generating new components based context-aware models are capable to determine the significance of the created components. As a result, an effective context-aware model can be built up with this feature extraction based approach that is also able to provide the significant prediction results with the trade-off the components and the prediction accuracy according to the preferred variance in a model.

In this experiment, we show the outcome results of the correlation-based feature selection approach for building context-aware models. For this, we first calculate the correlation of each context, and later with the target behavioral activity class as well. Tables 4 and 5 show the correlation values considering all the contexts utilizing the datasets of user U1 and U2 respectively. The results are shown considering the correlation with each context representing Con1, Con2, ..., Con7, where the value is between \(-1\) and \(+1\).

If we observe Tables 4 and 5, we see that, for each context, it generates a particular correlation value according to their relevance with other contexts. The negative correlation shown in Tables 4 and 5 is a relationship between two contextual variables in which one variable increases as the other decreases, and vice versa. The higher value represents highly correlated with each other and vice-versa. For instance, a perfect positive correlation is represented by the value +1 while a perfect negative correlation is represented by the value -1. For any 0 value, it indicates no correlation. A good feature subset contains features with less correlated or even uncorrelated to each other. The reason is that features with high correlation are more linearly dependent, and hence have almost the same effect on the dependent target class variable. In addition to correlation values between the contextual features, Table 6 also shows the correlation values for each context with the target class variable for the user U1 and U2 respectively, where the value is also between \(-1\) and \(+1\). In this case, a good feature subset contains features that are highly correlated with the target class variable for classification, as the target class is directly influenced by these features according to their correlation values.

To show the effect of feature subsets selection with their correlation values, we have also shown the prediction results of the resultant context-aware models using various machine learning classification techniques, such as random forest (RF), decision tree (DT), k-nearest neighbor (KNN), naive Bayes (NB), support vector machine (SVM), and artificial neural network (ANN), utilizing the datasets of both the users U1 and U2. The results are shown by varying the correlation threshold t with 90%, 70% and 10% in Tables 7 and 8 respectively. If we observe Tables 7 and 8, we can see that different threshold values give different results for each machine learning-based model. For 10%, it decreases the prediction results in terms of precision, recall, and f-score. The reason is that low correlation value as a threshold, may lose the context information and consequently low prediction results. However, a larger value of threshold increases the number of contexts in the model, consequently, increases the computational complexity and decreases the prediction results in several cases depending on the dataset characteristics. Thus, we choose 90% as a threshold as the correlation threshold, to output the prediction results. For another dataset, this value might be changed depending on their data characteristics and patterns. According to these results shown in Tables 7 and 8, we can conclude that the random forest classification based context-aware model performs comparatively better than other classification techniques for a particular chosen threshold of correlation. Besides these prediction results, we also show the ROC values in Fig. 8, considering the random forest classification model utilizing the datasets of both the users U1 and U2. The reason for getting better results with correlation-based random forest model is that this model fits multiple decision trees with the selected feature subsets, and averages all the single tree results.

Overall, we can conclude from our experimental results is that feature selection based context-aware models are capable to determine the significance of the selected features. As a result, an effective context-aware model can be built up with this feature selection based approach that is also able to provide the significant prediction results with the trade-off the selected features and the prediction accuracy according to the preferred correlation value in the model.

In this experiment, we compute and compare the effectiveness of both the principal component-based context-aware model, and correlation-based context-aware model, using above mentioned machine learning classification methods. These are random forest (RF), decision tree (DT), k-nearest neighbor (KNN), naive Bayes (NB), support vector machine (SVM), and artificial neural network (ANN) that are discussed briefly in earlier section. Figure 9a, b show the relative comparison of prediction results in terms of precision, recall, f-score utilizing a collection of apps usage datasets of ten individual participants.

If we observe Fig. 9a, b, we see that both the component-based models and the correlation-based models give significant prediction results for random forest learning comparing to other learners. To calculate these results, we consider 90% variance, and 90% correlation as the threshold value, to select the number of components, and contexts respectively. However, different values may give different results that are discussed earlier. The main difference between these two models are - in a component-based model the contexts are converted as principal components and are used in the model, and the number of components is chosen based on variance. On the other hand, a correlation-based method directly uses a subset of the contexts in the datasets, and the number of contexts are chosen according to their correlation. Overall, from Fig. 9a, b, we can conclude that for a particular chosen threshold of variance, or correlation, the random forest classification based context-aware model performs comparatively better than other classification techniques.