1 Introduction
Score | Quality | Distortion | Class |
---|---|---|---|
1 |
Bad
| Very annoying but objectionable | 0 |
2 |
Poor
| Annoying, but not objectionable | |
3 |
Fair
| Perceptible and slightly annoying | 1 |
4 |
Good
| Just perceptible, but not annoying | |
5 |
Excellent
| Imperceptible |
2 Data description
Types | Features |
---|---|
QoS parameters | Average rate of playing phase (kbps) |
Video total download (DL) rate (kbps) | |
Video bitrate (kbps) | |
Initial max DL rate (kbps) | |
End-to-End (E2E) round-trip time (RTT) (ms) | |
Initial buffering latency (ms) | |
Video Initial buffer downloaded (byte) | |
Playing time (ms) | |
Playing total duration | |
Stalling times | |
Stalling ratio | |
Scoring factors | vMOS |
-
Heterogeneous data types. The preliminary results show that QoS parameters are in different types and different ranges. For example, the initial buffering latency is ranging from 500 ms (millisecond) to 30,000 ms while the average rate of play phase is ranging from 300 kbps (kilobit per second) to 16,000 kbps.
-
Positive/negative correlations of QoS parameters. As shown in the statistics results, we observe that there are positive or negative correlations between vMOS and other QoS parameters. For example, there is a positive correlation between vMOS and video total DL rate while there is a negative correlation between vMOS and initial buffering latency.
-
Dependence on QoS parameters. The preliminary statistics results also show that vMOS is essentially affected by multiple factors, such as average rate of playing phase, video total DL rate, end-to-end (E2E) round-trip time, and initial buffering latency.
3 QoE analysis framework
3.1 Data preprocessing
-
Positive values:$$ u_{ij}=\frac{x_{ij}-\min\left(x_{ij}\right)}{\max\left(x_{ij}\right)-\min\left(x_{ij}\right)}, $$(1)
-
Negative values:$$ u_{ij}=\frac{\max\left(x_{ij}\right)-x_{ij}}{\max\left(x_{ij}\right)-\min\left(x_{ij}\right)}, $$(2)
3.2 K-means + LR method
3.2.1 K-means clustering
3.2.2 Logistic regression
4 Empirical study
4.1 Experiment settings
Description | No. of samples | No. of features |
---|---|---|
Dataset 1 | 30,000 | 11 |
Dataset 2 | 30,000 | 11 |
Dataset 3 | 26,984 | 11 |
Types | Features | Variables |
---|---|---|
QoS parameters | Average rate of playing phase (kbps) |
x
1
|
Video total download (DL) rate (kbps) |
x
2
| |
Video bitrate (kbps) |
x
3
| |
Initial max DL rate (kbps) |
x
4
| |
End-to-End (E2E) round-trip time (RTT) (ms) |
x
5
| |
Initial buffering latency (ms) |
x
6
| |
Video Initial buffer downloaded (byte) |
x
7
| |
Playing time(ms) |
x
8
| |
Playing total duration |
x
9
| |
Stalling times |
x
10
| |
Stalling ratio |
x
11
|
4.2 K-means LR method
4.2.1 K-means Analysis
Dataset | Classification standard | QoE | No. of samples |
---|---|---|---|
Dataset 1 | 3.90 | Bad | 10,306 |
Good | 19,694 | ||
Dataset 2 | 3.90 | Bad | 19,833 |
Good | 10,167 | ||
Dataset 3 | 3.93 | Bad | 9,199 |
Good | 17,785 |
4.2.2 Logistics regression analysis
Coefficient | Dataset 1 | Dataset 2 | Dataset 3 | ||||||
---|---|---|---|---|---|---|---|---|---|
Value | Wal | Sig | Value | Wal | Sig | Value | Wal | Sig | |
β
1
| 34.43 | 0.005 | 0.943 | − 15.51 | 0.272 | 0.602 | − 11.094 | 0.214 | 0.643 |
β
2
| − 12.65 | 0.001 | 0.98 | 9.88 | 0.103 | 0.748 | 12.21 | 0.245 | 0.621 |
β
3
| 105.18 | 7.374 | 0.007 | 1.721 | 38.976 | 0 | 2.263 | 75.108 | 0 |
β
4
| − 18.403 | 0.27 | 0.599 | − 0.632 | 0.129 | 0.72 | − 0.736 | 0.152 | 0.697 |
β
5
| 15.89 | 0.41 | 0.522 | 0.313 | 0.074 | 0.785 | 0.943 | 0.585 | 0.444 |
β
6
| 112529 | 7.47 | 0.006 | 2156.26 | 404.857 | 0 | 2598.501 | 408.67 | 0 |
β
7
| − 14.49 | 0.013 | 0.909 | − 4.778 | 0.312 | 0.576 | 4.1 | 0.454 | 0.501 |
β
8
| 211762 | 2.37 | 0.123 | 3612.752 | 6.299 | 0.012 | 4204.75 | 5.961 | 0.015 |
β
9
| − 3252.45 | 2.359 | 0.125 | − 56.298 | 6.409 | 0.011 | − 64.055 | 5.813 | 0.016 |
β
10
| − 1911.44 | 0.265 | 0.607 | 19.609 | 0.062 | 0.803 | − 45.572 | 4.02 | 0.045 |
β
11
| − 207352 | 2.369 | 0.124 | − 3553.314 | 6.368 | 0.012 | − 4158.129 | 4.132 | 0.042 |
4.3 Performance comparison
Method | Precision | ||
---|---|---|---|
Dataset 1 | Dataset 2 | Dataset 3 | |
Multivariate linear regression | 72.55% | 72.51% | 71.51% |
Logistics regression | 74.93% | 77.55% | 83.55% |
SVM | 82.56% | 87.17% | 85.42% |
K-nearest-neighbor | 52.48% | 61.34% | 56.77% |
Naive Bayes | 88.12% | 83.47% | 91.09% |
K-means + LR (proposed) |
96.94%
|
97.13%
|
97.54%
|
4.4 Performance analysis of our method
4.4.1 Residual plots
4.4.2 Accuracy analysis
Features | Dataset 1 | Dataset 2 | Dataset 3 | |||
---|---|---|---|---|---|---|
D value | Ranking | D value | Ranking | D value | Ranking | |
Average rate of playing phase | 0.0605 | 7 | 0.0662 | 7 | 0.0667 | 6 |
Video total DL rate | 0.0596 | 8 | 0.0651 | 8 | 0.0653 | 8 |
Video bitrate | 0.1848 | 3 | 0.1620 | 3 | 0.2160 | 3 |
Initial max DL rate | 0.3008 | 2 | 0.2997 | 2 | 0.3155 | 2 |
E2E RTT | 0.5409 | 1 | 0.5186 | 1 | 0.5602 | 1 |
Initial buffering latency | 0.0222 | 11 | 0.0222 | 11 | 0.0199 | 11 |
Video initial buffer downloaded | 0.1020 | 4 | 0.1025 | 4 | 0.1033 | 4 |
Playing time | − 0.0668 | 6 | − 0.0667 | 6 | − 0.0657 | 7 |
Playing total duration | − 0.0814 | 5 | − 0.0741 | 5 | − 0.0695 | 5 |
Stalling times | − 0.0231 | 10 | − 0.0229 | 10 | − 0.0279 | 9 |
Stalling ratio | − 0.0232 | 9 | − 0.0231 | 9 | − 0.0224 | 10 |
5 Discussion and future work
-
Concentrating on optimizing several dominating QoS parameters in vMOS. For example, we may focus on optimizing the network resource to reduce initial buffering latency, stalling times, and stalling ratio so that we can significantly improve the video QoE while maintaining relatively low operating expenditure. However, it is not an easy task to achieve this goal because the enhancement of these QoS parameters is also involved with many other technologies, such as cross-layer optimization and distributed resource allocation [10, 25, 26].
-
Identifying QoS bottlenecks. Determining QoS bottlenecks can help to enhance system performance and consequently improving vMOS. However, it is also difficult to identify the QoS bottlenecks since they are often affected by many factors. For example, video stalling is essentially caused by many factors, such as network congestion, network failure, device mobility, and radio spectrum scarcity. There is a challenge in identifying the causality of stalling. In the future, we may apply data-driven approach to identify the reason behind video stalling according to different scenarios.
-
Distributing the videos appropriately to improve vMOS. For example, we can distribute the most popular videos at the servers close to users so that we can significantly reduce the initial buffering latency. However, to determine the popularity of video streaming is challenging since it requires the extensive efforts in analyzing the massive video data [27].