Multiplex network embedding for implicit sentiment analysis

Sentiment analysis is one of the most important tasks in the field of natural language processing. The existing text-based studies of sentiment classification can be mainly grouped into two categories: lexicon-based and corpus-based approaches. Lexicon-based approaches mainly use the sentiment polarity associated with the sentiment dictionary to calculate the sentiment polarity of each sentence or document. Corpus-based approaches [6‐8] take sentiment classification as a special case of text categorization problem, which utilize machine learning methods to extract reasonable features from texts and feed them into a classifier to predict sentiments. Nowadays, more people are used to express their attitudes on different entities in online social networks, forming user to entity sentiment links. These sentiment links imply positive or negative semantics. Most of current user sentiment analysis literature focuses on making a positive, neutral, or negative sentiment decision according to users’ text descriptions. And more and more attention has been paid to the task of user-entity sentiment analysis. For example, Li et al. [9] integrate both named entity information and sentiment level information together to form label sequences and adopt an approach based on graphical models. Gan et al. [10] propose a self-attention based hierarchical dilated convolutional neural network for multi-entity sentiment analysis. Ding et al. [11] design an entity-level sentiment analysis tool consisting of sentiment classification and entity recognition, which can classify issue comments into < sentiment, entity > tuples. However, these above entity sentiment analysis are mainly rely on text content, and all contain explicit words. Compared with explicit sentiment analysis, the lack of sentiment words in sentences makes the expression of sentiment more euphemistic and these traditional sentiment analysis methods often fail to retrieve users’ hidden real attitudes, making the implicit sentiment analysis more challenging.

Existing works of implicit sentiment analysis mainly focuses on the discrimination of implicit sentiment sentences. To capture implicit structures, recent studies show promising results by learning word embedding with neural language models, such as word2vec [41] model. The embedded word vector obtained by these methods has better representation and reasoning ability in the semantic space and can be used as the input of various deep learning models, such as LSTM, and CNN models. Kauter et al. [12] propose a new fine-grained method to identify explicit and implicit sentiment in financial news and reviews. Liao et al. [5] propose a multi-level semantic fusion method based on representation learning, which can learn and fuse three different levels (i.e. words, sentences and documents) of features to recognize factual implicit sentiment sentences, however, this model is limited by grammatical structure. Bi-directionally long short-term memory [13] model can effectively capture the semantics of long dependent sequences, and assign different weights to words in sentences automatically via introduction of attention mechanism [14, 15]. Wei et al. [16] propose an implicit sentiment classification model based on multi-polarization orthogonal attention mechanism, which can simulate the differences between vocabulary and specific sentiment polarity attention effectively, and improve the performance of implicit sentiment classification. Zuo et al. [17] propose a context-based heterogeneous graph convolution network model, which first regards the context of the whole document as a heterogeneous graph to maintain its dependency structure, and then employs graph convolution network to obtain the features of implicit sentiment sentences and context.

However, existing implicit sentiment analysis often faces three major challenges: (1) there is a lack of sentiment words; (2) the words are relatively objective and neutral; and (3) the words’ sequences, co-occurrence relationship are neglected. The traditional bag of words-based models, however, can neither accurately capture the meaning and text structure of words in the text, nor effectively represent the semantics of a sentence. In addition, since words are organized into text content by forming phrases, sentences and so on to express users' opinion and the different positions of words in a sentence will have a great impact for a detailed understanding of the text.

Graph-based text representation method is one of the effective methods to solve the above problems. In the graph, nodes represent features and edges represent the relationship between different nodes. Although there are various graph-based text representation models, such as word graphs and ngram-graphs [18], to better capture the inherent characteristics of text in social media, this paper uses word co-occurrence graph to represent the relationship between words in short text content.

Representing the text in a graph structure that considers all the information for sentiment classification has achieved excellent performance and has attracted considerable attention in recent years. For example, Bijari et al. [19] propose to represent sentences via graph scheme and Node2vec embedding is used as feature learning algorithms to extract the text features. Zhao et al. [20] design a bidirectional attention mechanism with position encoding to capture the aspect-specific representations. Gui et al. [21] propose to build a heterogeneous network by constructing the relationship among words, users and products.

On social networks, users with similar hobbies and concepts often form personalized social communities. For example, Xiao et al. [22] and Zhao et al. [23] both consider user structure and model the social relationship of users. In addition, part of the sentiment prediction work also considers user-product-aware information, for example, user product neural network (UPNN) [24], user product attention (UPA) [25], user product deep memory network (UPDMN) [26] and hierarchical user attention and product attention neural network (HUAPA) [27] have achieved excellent performance in sentiment analysis by considering internal and external connections of users and products.

These above researches demonstrate that both the representation of a text in a graph structure, user social structure and the connections between users and entities are effective for sentiment analysis and opinion mining. These studies inspire this work to take into account both aspects in the proposed framework.

The purpose of network embedding is to represent high-dimensional and sparse vector space with low dimensional and dense vector space. The learned features can be used in machine learning tasks such as classification, regression, clustering and so on. Network embedding methods mainly include matrix eigenvectors calculation-based methods and neural network-based methods. Locally linear embedding (LLE) [28] and Laplacian Eigenmaps (LE) [29] are conventional matrix eigenvectors calculation-based methods. Nowadays, neural network-based embedding methods, such as DeepWalk [30], which inspired by the neural network language model Word2vec [31] achieves excellent performance and has been widely used. LINE [32] proposes to establish the first- and second-order proximity between nodes, and can be used in large networks. Node2vec [33] designs a biased random walk procedure to learn a mapping of nodes that maximizes the likelihood of preserving network neighborhoods of nodes. SDNE [34] uses auto-encoder to capture network structures and learn user representations.

The above researches are network embedding models and methods. In addition, there are also some literatures focusing on embedding representation algorithms for different entities. For example, Zhang et al. [35] propose a new optimization model to capture the hidden relationship between item content features for cold-start and content-based recommendations. He et al. [36] propose the bipartite graph neural network (BGNN), which consists of two central operations as interdomain message passing and intra-domain alignment to learn the embedding representations of user and item nodes. Wang et al. [37] propose SHINE model, which utilizes deep neural network structure to learn the low-dimensional representation of nodes to predict sentiment links between users and celebrities. Yuan et al. [38] propose to fuse multi-networks information to learn the nodes’ low-dimensional embedding representation for user behavior classification.

The above studies on network embedding mainly focus on the field of recommender systems and user behavior analysis. The proposed scheme is different from these studies in two aspects. First, this work utilizes network embedding to address a different problem. In particular, this paper models the implicit sentiment analysis problem from special link prediction angle and explore user-entity modeling to find the implicit sentiment in social network comments. Second, this different problem raises new challenges that are not addressed in existing studies. In particular, implicit sentiment analysis requires the proposed embedding approach to be able to capture rich information from different networks, such as user social network, user-entity sentiment network, and word graph-based text level comments. These networks are highly heterogeneous. To address these challenges, a novel multiplex network embedding method called MEISP is proposed in this work. Specifically, deep neural network architecture is introduced for embedding these heterogeneous networks, and a multilayer perceptron structure-based early fusion method is adopted to fuse the text information and social network information. Furthermore, the proposed scheme can also extract nodes’ highly nonlinear representations while preserving the original networks structure.

The occurrence, ordering, and positioning of words and the relationship between different parts of a text play the most important role in determining the sentiment polarity of a sentence. Therefore, a first fundamental question is how to select an appropriate graph-based text representation models to reveal the innate and essential information of a text graph. As far as we know, word co-occurrence graph is an effective way to represent the relationship between one word and another in social media comments. This paper defines the text graph as \(G_{T} \,=\,\left( {W,E} \right)\), where each word in a sentence is considered as a node. In the graph \(G_{T}\), if the corresponding words appear simultaneously in the window of \(k\) words, it can be defined that there is a co-occurrence relationship between these words. Generally, 2 to 10 seem to be appropriate according to different experiments. Therefore, this paper uses a word graph with a window size of 5.

A user’s social relation connections can measure the local relationship of this user and has an indirect impact on his/her behaviors. Therefore, based on such connections, users can be categorized into different communities, which is of great significance for analyzing the dissemination of social network information. In this paper, the user structure graph is denoted as \(G_{R} \,=\,\left( {U,R} \right)\), where \({\text{U = }}\left\{ {u_{i} } \right\}_{i = 1}^{N}\) is the set of users; \(R = \left\{ {r_{ij} } \right\}_{i,j \in U}^{\left| R \right|}\) is user connections, \(r_{ij}\) can take the value of 1 or 0, which denote the user \(i\) follow the user \(j\) or not.

In social networks, a large number of users express their attitudes towards entities, and this topological relationship forms sentiment network. The sentiment network is denoted as \(G_{P} \,=\,\left( {U,V,P} \right)\), in particular, \(U\) and \(V\) represent users set and entities set contained in social media contents, respectively; \(P = \left\{ {p_{ij} } \right\}_{i \in U,j \in V}^{{}}\) is sentiment links among users and entities node pairs. Each \(p_{ij}\) can take the value of \(+ 1\), \(- 1\) or \(0\), which denote sentiment polarity between the user \(i\) and entity \(j\) are positive, negative or unobserved polarity relationship, respectively.

Output Taking \(G_{R}\), \(G_{P}\) and \(G_{T}\) as inputs, and through a well-designed multi network fusion method to learn the node low dimensional representation, and then to predict sentiment relation \(\hat{G}_{P}\) between uses and entities hidden in user comments.

The proposed method consists of two main modules: word graph-based text representation module and heterogeneous social graph network embedding module.

To capture the implicit text feature hidden in the user published content, this paper uses network embedding methods to learn the low dimensional potential representation of nodes in the network. For text representation, first, this paper processes text into graphs, and represent each text as a graph of vectors. Then, feature learning technology (representing the learning stage) is adopted to capture non-consecutive and long-distance semantics, and the internal features of the word graph-based text representation are determined.

In addition to text content, the topological relationship of users to entities forms the explicit sentiment network. To capture such information, rather than directly adopt the adjacency matrix of user-entity sentiment network, which is too long and sparse for further processing, this paper divides the original social network into user relationship structure network and user entity sentiment network, and learns their node embedding representations separately through two distinct autoencoders. Concatenation operation is then used to aggregate these above embeddings into final heterogeneous social network representation, since it can preserve more information out of the two types of heterogeneous information networks.

Once the word graph-based text embedding and heterogeneous social graph network embedding are obtained, multiple non-linear layers are stacked to learn better representations of data. In this way, the word graph-based text modeling can complement the learning of heterogeneous social network modeling, realize the seamless integration of these two modules. In the following subsections, this paper will introduce the proposed MEISP framework in detail. Figure 1 shows the work-flow of the proposed framework.

Node embedding representation is the process of vectorizing the nodes in a graph. In the process of learning representation, it can focus more on the important nodes and edges and ignore the unimportant nodes and edges. In the past decades, shallow networks have been used to learn network embedding representations and some high performance models have been proposed. At the same time, node embedding can be used to reveal the essential information of graph. Inspired by the recent success of deep learning, SDNE [34] a semi-supervised deep model, which has multi-layer nonlinear functions, has been proposed. The composition of multi-layer nonlinear functions can map data to highly nonlinear space, and by jointly optimizing first-order and second-order proximity in the semi-supervised depth model, this method can preserve the local and global network structure, and is robust to sparse networks. Figure 2 shows the deep neural network for text embedding. Next, this paper will discuss the workflow of feature learning for graph-based text contents.

Word co-occurrence is the most effective way to represent text as a word graph. For a given text \(T\), this paper defines the text as a graph \(G_{T} \,=\,\left( {W,E} \right)\), \(W = \left( {w_{1,} w_{2} \cdots w_{n} } \right)\) denote the node set of vertices, and \(w_{i}\) is the \(i - th\) vertex in the text. If two words appear in a context within a window of size \(k\), they are considered to have a word-word relation, and \(E = \left\{ {\left. {e_{ij} } \right|i \in W,j \in W} \right\}\) denotes the set of edges between vertices pairs \(\left( {w_{i} ,w_{j} } \right)\), and in this study, \(k\) is set as 5.

In this model, the auto-encoder is an unsupervised model, which consists of encoder and decoder. The encoder maps the input data to the representation space through multiple nonlinear operations, while the decoder maps the embeddings from the low dimensional potential space to the reconstruction space through multiple nonlinear functions.

For the \(i - {\text{th}}\) vertex, the adjacency matrix \(s_{i}\) is used as input, and each \(s_{i}\) contains the neighbor structure information of vertex \(i\), the representation of each hidden layer [34] is as follows:

After obtaining \(y_{i}^{(l)}\), the output \(\hat{x}_{i}^{{}}\) can get by reversing the operation process of the encoder.

The goal of the autoencoder is to minimize the reconstruction error of the output and input, so the reconstruction process can make the vertexes with similar structure have similar embedding representation vectors. Therefore, it is necessary to use a weighted loss function with a high penalty coefficient for non-zero elements. In other words, by reconstructing the second-order nearest neighbors between the vertices to preserve the global network structure. And then in order to make the two connected nodes very close in the mapping space, we also define the first-order similarity. The first order and the second order similarity are denoted as:where \(\odot\) is Hadamard product, \({\mathbf{b}}_{i} = \left\{ {b_{i,j} } \right\}_{j = 1}^{n}\). If \(s_{i,j} = 0\), then \(b_{i,j} = 1\), otherwise \(b_{i,j} = \beta > 1\). The parameter \(\beta\) controls the penalty term coefficient of non-zero elements.

The semi-supervised learning method ensures the first-order similarity and the second-order similarity respectively, the minimized objective function of joint optimization is:\(\alpha\) is the parameter controlling the first-order loss. The representations of word graph network can be denoted as \(h_{i}^{T} \,=\,y_{i}^{\left( L \right)}\). It is note that not all text words contribute equally important to the text sentiment, hence, this paper tries to learn unified embedding by assigning different importance weights to different words. The weight coefficients can be calculated as follows:where the word representation \(h_{i}^{T}\) obtained is used as input of multi-layer perceptron model, and then get the node representation \(z_{i}^{{\text{T}}}\),\(z_{w}\) is the context word matrix. The final representation of the vertex \(v_{i}\) is:

For the given sentiment relationship network \(G_{{\text{P}}} \,=\,\left( {U,V,P} \right)\), \(U\) and \(V\) are user and entity nodes set respectively, sentiment polarity adjacency vector can be denoted as \(x_{P} = \left\{ {\left. {p_{ij} } \right|i \in U,j \in V} \right\}\). The sentiment polarity adjacency vector \(x_{P}\) contains global sentiment information of user \(i\) to entity \(j\). However, in sentiment prediction task, due to the sparsity of adjacency vector \(x_{P}\), it cannot be used as sentiment relation directly. Recently, network embedding methods, which can learn rich network structure and semantic information, has attracted a lot of attention, and motivated by excellent performance of deep autoencoder in capturing highly nonlinear network structures via deep models, this paper introduces deep autoencoder structure to learn sentiment relationship network embedding. In the sentiment relationship autoencoder, encoder compresses inputs into potential space representation, decoder reconstructs outputs from potential space representation. Both encoder and decoder are neural network models and include multiple nonlinear layers. Figure 3 shows the architecture of autoencoder for user social relation and user-entity sentiment network embedding.

Given the adjacency matrix \(x_{{{\text{P}}i}}\) as input, the hidden representations of each layer \(l\) are:where \(W_{{\text{P}}}\) and \(b_{{\text{P}}}\) are the weight and bias in the sentiment deep architecture, respectively, \(\sigma \left( \cdot \right)\) is the nonlinear activation function, \(L_{{\text{P}}}\) is the number of layers of autoencoder. The output of \({\raise0.7ex\hbox{${L_{{\text{P}}} }$} \!\mathord{\left/ {\vphantom {{L_{{\text{P}}} } 2}}\right.\kern-\nulldelimiterspace} \!\lower0.7ex\hbox{$2$}}\) layer \({\mathbf{z}}_{i}^{{\text{P}}} \,=\,x_{{{\text{P}}i}}^{{{{L{\text{p}}} \mathord{\left/ {\vphantom {{L{\text{p}}} 2}} \right. \kern-\nulldelimiterspace} 2}}}\) denotes the sentiment polarity network embedding representation, and the output of \(L_{{\text{P}}}\) layer \(x_{{{\text{P}}i}}^{{L{\text{p}}}}\) denotes sentiment reconstruction vector \(x_{{{\text{P}}i}}\). In the process of sentiment polarity network embedded representation and data reconstruction, our basic goal is to minimize the reconstruction loss between input and output data. Therefore, the reconstruction loss term of sentiment polarity deep autoencoder is:where \(\left( {x_{{\text{P}}} - \hat{x}_{{\text{P}}} } \right)\) is used to measure the distance between \(x_{{\text{P}}}\) and \(\hat{x}_{{\text{P}}}\), \(\odot\) is the Hadamard product and \(l_{{{\text{P}}_{ij} }}\) is used to define reconstruction weight vector of sentiment link. In other words, if there is a sentiment connection between user node \(u_{i}\) and entity node \(v_{j}\), whether it's positive or negative, this paper defines the weight vector \(l_{{{\text{P}}_{ij} }} = \gamma > 1\), else \(l_{{{\text{P}}_{ij} }} = 1\). The meaning of the reconstruction weight vector lies in that we impose more penalty to the reconstruction error of the non-zero elements than that of zero elements in input \(x_{{P_{i} }}\).

For the given user social relation network \(G_{{\text{R}}} \,=\,\left( {U,R} \right)\), this paper collects the social relation among user pairs \(\left( {u_{i} ,\;u_{j} } \right)\). The relationship of each user pair can take a value of 1 or 0, indicating whether user \(i\) follows user \(j\). Similar to sentiment network embedding, this paper also employs deep neural network to embed user social relation. Given the adjacency matrix \(x_{{{\text{R}}i}}\) as input, the hidden representations of each layer \(l\) are:where \(W_{{\text{R}}}\) and \(b_{{\text{R}}}\) are the weight and bias in the social relation deep architecture, respectively. \(\sigma ( \cdot )\) is the nonlinear activation function. The output of \({\raise0.7ex\hbox{${L_{{\text{R}}} }$} \!\mathord{\left/ {\vphantom {{L_{{\text{R}}} } 2}}\right.\kern-\nulldelimiterspace} \!\lower0.7ex\hbox{$2$}}\) layer \({\mathbf{z}}_{i}^{{\text{R}}} \,=\,x_{{{\text{R}}i}}^{{{{L_{{\text{R}}} } \mathord{\left/ {\vphantom {{L_{{\text{R}}} } 2}} \right. \kern-\nulldelimiterspace} 2}}}\) denotes the user social relation network embedding representation, and the output of \(L_{{\text{R}}}\) layer \(x_{{{\text{R}}i}}^{{L_{{\text{R}}} }}\) denotes social relation reconstruction vector \(x_{{{\text{R}}i}}\).

The design of loss function of social relation embedding is the same reconstruction loss term of sentiment polarity network, and then the social relation loss is:where \(\left( {x_{{\text{R}}} - \hat{x}_{{\text{R}}} } \right)\) is used to measure the distance between \(x_{{\text{R}}}\) and \(\hat{x}_{{\text{P}}}\), \(\odot\) is the Hadamard product and \(l_{{{\text{R}}_{ij} }}\) is used to define reconstruction weight vector of user social relation link. That is if there is a link between \(u_{i}\) and \(u_{j}\), this paper defines the weight vector \(l_{{{\text{R}}_{ij} }} = \gamma > 1\), else \(l_{{{\text{R}}_{ij} }} = 1\).The meaning of the reconstruction weight vector lies in that we impose more penalty to the reconstruction error of the non-zero elements than that of zero elements in input \(x_{{R_{i} }}\).

Our work is to study heterogeneous social network-based and text-based multiplex network embedding for implicit sentiment mining and analysis. To combine the strength of both word graph-based text representation and heterogeneous social graph network modeling, an intuitive way is to choose concatenation to fuse the above information. However, the embedding representation obtained by concatenation is not effective for link prediction. Instead, early fusion model can optimizing all parameters simultaneously, so that the interaction between the word graph-based text modeling and heterogeneous social network modeling is more closely. Therefore, this paper uses the following methods to fuse the three kinds of network information.

First, for a given use/item node \(i\), both the social relation network embedding and the sentiment network embedding are equally important to the final representation. Therefore, the proposed method first aggregates these two embedding into a final node embedding \({\mathbf{z}}_{i}^{{\text{S}}}\) through a concatenation function, and the finally social network-based embedding can be denoted as:

Then, to fuse the text-based information and social network-based information, this paper integrates these two modules by early fusion model, that is \({\mathbf{z}}_{i}^{{\text{S}}}\) and \({\mathbf{z}}_{i}^{{\text{T}}}\) are used as the input of the multi-layer perceptron model, and then the representations of each hidden layer can be calculated by the following formulas:\(\delta_{k}\) is the activation function, \(L\) is the number of hidden layers. The final embedding representation can be denoted as \({\mathbf{z}}^{L}\). Through the above nonlinear fusion, heterogeneous social network modeling and word graph-based text modeling are combined. In particular, for a given user node \(i\) and an entity node \(j\), inner product operation is used to predict sentiment \(p_{ij}\), then, according to the reconstruction loss of three types deep autoencoders and supervised loss between predicted sentiment link polarity and ground truth, the objective function can be considered as:where \(\lambda_{1}\), \(\lambda_{2}\) \(\lambda_{3}\) and \(\lambda_{4}\) are balancing parameters. The first three terms in Eq. (16) are the reconstruction loss terms of sentiment autoencoder, user social relationship autoencoder, and text autoencoder, respectively. The fourth term is the supervised loss term for penalizing the divergence between predicted sentiment and ground truth. The fifth term is the regularization term that prevents over-fitting.

To verify the application performance of the proposed model in social media, this paper collected Weibo data for experiments. To remove noise, text with less than 10 words is firstly filtered out. Secondly, useless symbols such as spaces and emoticons contained in the text are removed, and only keep the words and numbers in the sentence. Then Jieba is used to segment the text. The stop words in the text are removed. Finally, the obtained text content is transformed into word graph network for feature learning. After the above processing the obtained data contains useful Weibo towards Movies dataset which contains users’ sentiment towards entities with 8735 texts, 1983 users, 1357 entities, 32,732 social links. And then, through the entity-level sentiment extraction method, this paper establishes the sentiment dataset with label information, which consists of users’ sentiment relation and social relation. The current amount of data is enough to extract valuable information from complex network data and complement text information for online opinion mine and sentiment analysis. Therefore, the experimental results are statistically significant.

For word graph-based text encoder, the reconstruction weight of non-zero elements \(\beta = 10\) and the weight of first-order term \(\alpha = 0.05\). Besides, this paper designs a 4-layer encode architecture for both social relationship autoencoder and sentiment autoencoder, trade-off parameter \(\lambda_{4} = 0.01\). Based on these parameters settings, this paper evaluates embedding dimensions, reconstruction weight \(\gamma\) of sentiment and social autoencoder, the balance parameters \(\lambda_{1}\), \(\lambda_{2}\), \(\lambda_{3}\) and different number of hidden layers of early fusion layer on the performance.

The purpose of multiplex network embedding is to learn the low-dimensional vector representation of different nodes, and then to reveal implicit sentiment polarity via similarity measurement function. In MEISP model, it contains word co-occurrence relations graph embedding (W–W), social relationship network embedding (U–U) and user-entity sentiment network embedding (U-E-P). In this section, this paper conducts ablation study to verify the effects of different models on sentiment analysis performance. Table 2 shows different model types.

MEISP_N is set as a baseline model, which only contains user-entity sentiment polarity network for sentiment prediction. On the basic of MEISP_N model, it adds word co-occurrence graph embedding and social relation graph embedding respectively, and record them as MEISP_W and MEISP_R model. Furthermore, in order to validate the effectiveness of word co-occurrence and user social relation on sentiment prediction, both the above two graph networks are embedded on top of the baseline model as MEISP.

First, we randomly hide 50% edges from the user-entity sentiment network, and construct a balanced test set in which the number of positive links equal to the number of negative links, and then the remaining network used to train the node representations via different models. The Precision, recall and F1value of experiment results are shown in Table 3.

From Table 3, we have the following observations. First, with the addition of U–U link information network and W–W information network to the MEISP_N model, the accuracy of sign link prediction has been gradually improved. Compared with the MEISP_N model, the MEISP_w model has increased the Precision, Recall, and F1 values by 7.63%, 2.81%, and 4.75%, respectively. Compared with the MEISP_N model, the MEISP_R model has increased the precision, Recall, and F1 values by 10.40%, 7.62%, and 8.95%, respectively. It is proved that the MEISP_w and MEISP_R models can improve the performance of sentiment link prediction, optimize the effect of sentiment analysis, and have universal applicability by introducing user relationships, word graph information. Second, for MEISP model, U-E-P sentiment polarity network, U–U link information network, and W–W graph information network are introduced to realize the embedding and fusion of multiple networks for sentiment link prediction. When compared with the MEISP_N model, MEISP has improved the precision, recall, and F1 values by 15.40%, 10.56%, and 11.67%, respectively. Third, comparing the MEIS_P model with the MEISP_W and MEISP_R models, the precision, recall, and F1 values have been improved significantly.

In summary, these comparative experiments fully verify that the MEISP model proposed in this paper has significantly improved the sentiment analysis of Weibo by adopting user relationships and word co-occurrence information.

In order to verify the embedding effect of MEISP model, we use link prediction as the downstream task, and compare it with baseline embedding methods LINE [32], Node2Vec [33] and SDNE [34].

For Weibo datasets, the training set and the testing set are given, and we randomly split the training set into actual training and validation data with a ratio of 9:1. The hyperparameters are tuned according to the performance on the validation set. Empirically, we set the learning rate as 0.01 with Adam [40] optimizer and the dropout rate as 0.5. We have done two groups of experiments. First, we randomly take 50% links of sentiment network out as the test set, and the remaining network is used to train network embedding. Then, the accuracy and AUROC are adopted as evaluation metrics. The experimental results are shown in Table 4.

It can be seen from Table 4 that our proposed MEISP model achieves Accuracy of 82.9% and AUROC of 81.6%. In term of accuracy, the MEISP method outperforms LINE, node2vec by 20.2%, 19.8%, respectively, and MEISP is about 14% higher than the network embedding model SDNE which also uses deep autoencoder. And achieves at least 14% gains on AUROC compared with the other three schemes.

Second, we randomly take 50–100% of the sentiment network links out as the test set to further verify the sentiment analysis results in the case of sparse network. The final results are shown in Fig. 5.

As can be seen from Fig. 5, with the increase of removed sentiment links from 50 to 100% in the test set, the AUROC values of all models decreased. For MEISP model, when the removed sentiment links are 50% and 100%, the AUROC values are 81.6% and 75.45%, respectively, which decreases by about 6%. However, for LINE, Ndode2vec and SDNE models, AUROC values decreased by 21%, 15% and 12%, respectively. It demonstrate the robustness of MEISP model is better than other models when the network is sparse.

The above two experiments show that compared with LINE, Node2vec and SDNE embedding methods, the proposed MEISP model can get better embedding representation results. Second, deep autoencoder structure-based multiplex network embedding is effective and more suitable for modeling sentiment analysis problem from link prediction angle.

It is difficult to judge the hidden sentiment relationship between new users and entities, when new user or entity nodes are just added in the network, which is the cold start problem. The most intuitive way is to use the information from the target network to learn new nodes embedding. For example, in the sentiment link prediction problem, the model only uses the information from the user-entity sentiment link network to solve the cold start problem. However, due to the little interaction between the new arrival node and the existing target network, the performance of sentiment link prediction is not satisfactory. The proposed scheme does not only consider the interaction between the new added node and the original network, but also makes full use of the side information between heterogeneous nodes.

In this section, we study the performance of MEISP when new nodes join the existing target network. We construct a test set of newly text, and the user social relations and user entity sentiment relationships contained in the new text are not used in the training process. The results for all and new added sentiment link prediction are shown in Table 5.

From the results, for MEISP, the Micro-F1 of new sentiment link prediction decreases by 1.6% when compared with all sentiment link prediction results, but for LINE, Node2Vec and SDNE, new sentiment link prediction results decrease by 11.9%, 10.7% and 6.7%, respectively. It can be seen that MEISP is still perform better in the cold start scenario, because the proposed multiplex network embedding method can mine the information from social network information and text information, which makes the obtained information for sentiment relationship analysis more complete.

At present, many sentiment analysis methods are proposed from the feature extraction angle and word embedding representation angle and most of them have achieved excellent performance. In this section, we compare MEISP model with a number of existing methods as listed below.

In this part of the experiment, balanced data is selected in order to reduce the bias to a certain category, which ultimately affect the sentiment analysis accuracy. And the optimal parameter setting is adopted for all models. The results are shown in Table 6.

As can be seen from Table 6, first, compared with the traditional sentiment analysis methods of Trigram and Textfeature, Word2vec + SVM and SSWE + SVM are more effective due to the deep learning-based word representation. Second, we notice that our method significantly outperforms UPNN model which is designed for sentiment classification specifically. The main difference between the MEISP and UPNN is the word representations derived. For UPNN model, it incorporate user and product level information into a convolutional neural networks, and only uses word embeddings trained with context information from text content only. In the MEISP model, it generates word representations from constructed multiplex networks embedding model, which inherently consider word emergence, ranking positioning and the relationship between different components of the text, as well as the heterogeneous social network information (user social relation network, user-entity sentiment network) during representation learning.

For the given user-entity sentiment relationship network \(G_{{\text{P}}} \,=\,\left( {U,V,P} \right)\), the sentiment polarity adjacency vector \(x_{P}\) contains global sentiment information of user \(i\) to entity \(j\). However, in sentiment prediction task, due to the sparsity of adjacency vector \(x_{P}\), it cannot be used as sentiment relation directly. The proposed MEISP model can capture word graph-based text information and heterogeneous social graph network information. We analyzed our predicted sentiment labels in section “Experiment” experiment part. In the confusion matrix, we observed that our model incorrectly linked the sentiment polarity relationship between users and entities. This is because (1) when the context is short, the text information is scarce, or (2) when the heterogeneous social networks are sparse, it is difficult to infer the sentiment of the user to entity. Therefore, the accuracy of each model is not high. As contextual information and heterogeneous social network information increase, the accuracy of all models is rising, indicating that text information and heterogeneous networks information are positively correlated with the accuracy of the model.