Edge-cloud computing oriented large-scale online music education mechanism driven by neural networks

The research motivation proposes developing a large-scale online music education system using an edge-cloud computing model. This refers to a distributed computing architecture combining edge computing and cloud computing. Edge computing provides low-latency and high-bandwidth services to end users by performing data processing at the edge of the network near the data source. For this music education system, edge servers closer to the learners can offer real-time interactive services and learning content while ensuring security and privacy of user data. Cloud computing provides flexible central storage and processing for large datasets. The cloud infrastructure allows aggregating and analyzing massive amounts of data from edge nodes to train machine learning recommendation models using deep learning algorithms. Together, the edge layer meets real-time interactive demands while the cloud enables robust analytics and personalization. This allows building an intelligent online education platform that can understand learners’ interests and adapt offerings accordingly. In summary, the edge-cloud computing model provides a scalable architecture to support large numbers of online learners distributed across geographies while also harnessing large-scale data to optimize the learning experience.

Music is a kind of regular audio, and the music played or processed in the computer has different formats. The audio signal is formed by the superposition of sine waves of various frequencies and phases, which can be drawn into a continuously changing curve. The range of 20 Hz-20 kHz is the frequency of the audio signal that can be received by the human ear. Audio analysis is to process and analyze digital audio signals, and extract some special properties of audio signals in time and frequency. Different research fields are interested in different signal frequency ranges [40]. Before using a computer to analyze audio, it is necessary to perform audio sampling, that is, the process of discretizing continuous time. According to the Nyquist sampling theorem, when the sampling frequency is greater than 40 kHz, the sampled sound can achieve lossless sound quality. This sampling frequency is defined as twice the maximum sound frequency acceptable to humans, and the frequency to which humans are sensitive is about 4 kHz, so the sampling frequency is set at 8 kHz.

Multi-source information fusion refers to the combination of multiple information sources according to specific standards to obtain a consistent description or interpretation of the measured object, so that the information system has better performance than the subsystems it contains. We perform feature extraction and deep feature fusion on audio information, release time, label information, and singer identification information of music from the edge and cloud to obtain an effective prediction model. The overall framework of recommendation methods based on multi-source information fusion can be disassembled into two stages. In the first stage, we use the WMF algorithm to decompose the user-music listening times matrix to obtain the latent factor vectors of users and music. In the second stage, we use the neural network model to realize the multi-source information fusion of music and predict the latent factor vector of music. Finally, the inner product of the music vector and the user vector is used as the user’s preference score for music and Top-N recommendations are made to solve the cold start problem of the music end.

In order to predict the user’s preference for a certain music, an effective method is to map the user and the music to the same latent semantic space to realize the quantitative measurement of user preference. Therefore, in this paper, we use the weighted matrix factorization (WMF) algorithm proposed in the literature [16] to construct the latent semantic space, and obtain the latent factor vectors of users and music. The objective function of the weighted matrix factorization algorithm is shown in the following formula.where \({p}_{ui}\) is a binary preference variable, indicating user’s preference for music i; \({\mathbf{x}}_{u}\) and \({\mathbf{y}}_{i}\) are latent factor vectors of user u and music i respectively; \({c}_{ui}\) is a confidence variable, indicating the possibility of user u liking music i; \(\lambda \left({\sum }_{u}\parallel {\mathbf{x}}_{u}{\parallel }^{2}+{\sum }_{i}\parallel {\mathbf{y}}_{i}{\parallel }^{2}\right)\) is the L2 regular term. The rating of music i for user u is represented by the product of their respective latent factor vectors \({\mathbf{x}}_{u}\) and \({\mathbf{y}}_{i}\). We implemented the above matrix factorization process using the implicit library and obtained latent factor vectors for users and music in the utilized dataset.

Although the latent representation of users and music is obtained, and the user’s preference for music is quantified, the cold start problem cannot be solved because the latent factor vectors obtained through WMF are all known music. To this end, in this paper, we construct a music feature system through multi-source information fusion, and carry out supervised learning of music features. However, the sequence processing ability of CNN is not as good as LSTM. Inspired by this, since audio is a sequence of data, the neural network we use in this paper combines the advantages of CNN and LSTM, and uses the Embedding layer to fuse other information of singers and music for music information mining and personalized recommendation. Through the neural network to capture the latent features of music in multi-source information, we fuse the latent features of music learned in each multi-source information together to achieve multi-source information fusion. The audio feature extraction part abstracts the audio spectrogram into a 100-dimensional feature vector. In the Embedding part, the music label, singer ID and release time information are respectively embedded for training, and the embedded representation of the three information is obtained, and feature fusion and selection are performed through a multi-layer perceptron (MLP). Finally, the high-order features obtained from the two parts are concatenated, and the feature selection is performed through the fully connected layer to obtain the music latent factor vector. The following will elaborate on the audio feature extraction part, embedding part and output layer.

In the music feature extraction part, in order to reduce the complexity of audio data learning, we convert the audio data in wav format into Mel spectrum through short-time Fourier transform and Mel filter bank, and use Mel spectrum as the input of neural network model. First, the short-time Fourier transform is used for time-frequency conversion of audio, and the sound wave image is converted into a spectrogram. The short-time Fourier transform is as shown in the formula below.where z(t) is signal source and g(t) is the window function. The time window width used in audio framing is 1024 audio frames, the jump size is 512 audio frames, and the Hamming window is used to extract audio amplitude information. Finally, we reduce the dimensionality of the spectrogram through a set of Mel filters to obtain the Mel spectrogram, and use the logarithmic energy output by each filter as the input of the audio part of the deep learning model. The Mel filter bank sets several band pass filters \({\text{H}}_{m}\left(\text{k}\right),1\le \text{m}\le \text{M}\) in the spectrum range, where M is the number of filters. In the normal frequency, the resolution of the Mel filter bank in the low frequency part is relatively high, which is in line with the auditory characteristics of the human ear. We use 128 Mel filters to get a 128-dimensional Mel spectral vector.

In the audio feature extraction model, we continuously use three convolutional layers and LSTM layers for feature extraction, and finally use a fully connected layer for feature fusion and selection to obtain a 100-dimensional audio feature vector. The dimensions of the audio feature vectors are set to be the same as those of the latent factor vectors. Since the spectrogram has the properties of both audio sequences and graphs, we combine the advantages of both CNN and LSTM to provide users with personalized music recommendations. LSTM uses three gates, namely the update gate, the forget gate and the output gate, which are used to control the memory state of the LSTM unit.

In the embedding part, word embedding is often utilized in the field of Natural Language Processing (NLP) for vocabulary representation. The Embedding layer maps words to a low-dimensional space, reducing the training difficulty of the NLP model. Inspired by this idea, we use an embedding layer to incorporate music tags, release times, artist IDs, and other available identifying information. The Embedding layer can make the model more scalable, and adding the identification data obtained in the future to the embedding training can build a more complete feature system. Before embedding training, we need to represent the feature values in one-hot encoding. One-hot encoding uses an N-bit state register to encode N states. Let the feature domain of the model be \(\text{F}=[{F}_{1};{F}_{2};\cdots \text{\hspace{0.17em}};{F}_{n}]\), and use one-hot encoding to represent the one-hot vector \({\varvec{o}}_{{F}_{i}}^{j}\) corresponding to the j-th eigenvalue of feature \({F}_{i}\). In the embedding layer, each feature corresponds to an embedding matrix. For the feature \({F}_{i}\), the feature mapping relationship in the embedding layer is as follows.

where \({e}_{{F}_{i}}^{j}\) is the embedding vector corresponding to the j-th eigenvalue in feature \({F}_{i}\); \({\text{W}}_{f}\in {\mathbb{R}}^{{F}_{i}\times \text{d}}\) is the weight matrix corresponding to feature \({F}_{i}\) in the embedding layer, and d is the vector dimension set by the embedding layer. Finally, the embedding vector of the music is learned by a multi-layer perceptron and spliced together to obtain the input of the next layer.

The output layer of the neural network model splices the hidden representations obtained from the audio feature extraction part and the embedding part, and performs feature fusion and selection through a fully connected layer, and outputs the music latent factor vector predicted by the model. The neurons in the fully connected layer all use the ReLU activation function, as shown below.

Suppose the output of the neuron a is \({O}_{a}\), and we can obtain the \({O}_{a}\) throughwhere \({X}_{a}\) is the input vector of the neuron a, \({\varvec{W}}_{a}^{\mathsf{T}}\) is the weights vector, and \({\varvec{b}}_{a}\) is the bias vector. The loss function used in our model is mean square error, which is calculated bywhere n is the total number of samples, \({\mathbf{y}}_{i}\) is the real latent factor vector corresponding to music i, and \({\mathbf{y}}_{i}^{{\prime }}\) is the latent factor vector of music i predicted by the model.

In the experiment, the music data information we use comes from the Million Songs Dataset (MSD) [41], in which the user listening data comes from the Echo Nest Taste Profile Subset, a subset of MSD. About 30 s of audio samples are obtained from music websites in the metadata collection. The training data used in this paper are all from a subset of the same dataset. In addition, we collected 10,000 pieces of music data based on edge devices to obtain online users’ music cleaning characteristics. First, we de-sparse the data, and extract 10,000 user listening records, and finally get a data set consisting of 10,000 users and 50,000 pieces of music. Due to the limitation of computing resources and time cost, we choose to extract a channel of random 3s wav format audio data during the experiment, and divide the audio data into a unified format.

In the weighted matrix decomposition stage, we use all the user listening records involved in 50,000 pieces of music to calculate the latent factor vectors of 50,000 pieces of music and 10,000 users. In the latent feature learning stage, we use 10,000 music pieces as the test set, and the remaining 40,000 pieces of music are used as the training set. When making Top-N recommendations, we use all users involved in the test set for calculation.

Similar to other literature studies, in this paper we mainly evaluate the model from two aspects, namely, the latent feature learning effect and the recommendation effect. The ability of a model to predict latent factor vectors is an important part affecting recommendation performance. Therefore, we use Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Squared Logarithmic Error (MSLE) evaluates the prediction effect of the neural network model. They are respectively defined as follows.where \({y}_{i}^{{\prime }}\) is the predicted value of music i latent factor vector; \({y}_{i}\) is the real value of music i latent factor vector. MSE and RMSE describe the error between the predicted value and the real value from the perspective of spatial distance. MAE is more robust to outliers. MSLE is more tolerant to the range of sample values. For the evaluation of recommendation effect, we use precision, recall and F1 value as evaluation indicators.

When evaluating the effect of the model, we use the same training set and test set to train each model and calculate the evaluation index to ensure the authenticity and validity of the evaluation results. The prediction effect of latent factor vector of neural network model directly affects the learning of latent features by the model, so we use MSE, RMSE, MAE, and MSLE to comprehensively evaluate the effect of the model. Table 1 compares the results achieved by four different models.

It can be seen from Table 1 that the autoencoder model has the lowest performance, which shows that the model is slightly less capable of extracting features in music data. After the CNN model is added to the LSTM structure, the prediction error decreases, which shows the effectiveness of the LSTM. This is due to the fact that the LSTM model is good at capturing temporal information, making up for the shortcomings of CNN. When using the embedding layer alone, the performance of the model is worse than that of the CNN model, and the four error indicators are all higher than CNN, which shows that the use of additional music information alone cannot surpass the performance of the neural network. Meanwhile, our proposed method achieves the best results. Since the audio information modeling is more complex, the effective information obtained from the audio feature learning module is insufficient, and it is necessary and effective to incorporate the release year, label and singer information.

Meanwhile, we validate the performance of the model on the recommendation task against these four models. Figure 2 depicts the comparative results of the four models on the three indicators. It can be seen from the experimental results that the accuracy, recall and F1 value of the method proposed in this paper are better than other models. The overall form shown in Fig. 2 basically matches that in Table 1, which also shows that the quality of feature extraction will also determine the classification performance of the model.

To investigate the impact of learning rate on model training and performance, we conducted experiments with four learning rates: 0.1, 0.01, and 0.001. Other hyperparameters were kept constant, along with the training and validation sets across experiments. Models were trained for 100 epochs while comparing the training loss, validation loss, and validation accuracy curves across learning rates. Figure 3 shows the model loss under different learning rates. With a learning rate of 0.1, the training loss decreased rapidly but overfitting occurred as evidenced by high validation loss. Setting the learning rate to 0.01 led to slower training loss reduction but lower validation loss. A learning rate of 0.001 resulted in steady training loss decrease and slightly improved validation performance over 0.01. However, a learning rate of 0.0001 caused slow training loss decline indicating underfitting. In summary, a learning rate of 0.001 achieved the optimal balance between training and validation loss, also yielding the highest validation accuracy. This learning rate ensures efficient model training while avoiding overfitting. Based on the comparative analysis, we set the learning rate to 0.001 for model training to guarantee both convergence speed and generalization capability. The experiment provides insights into selecting an appropriate learning rate schedule to optimize deep learning model performance.