There are three pillars that characterize the new 5G revolution, namely, the use of heterogeneous wireless access technologies conforming an ultra-dense network, the software-driven flexibility of this network, and the simplified and user-centric operation and management of the system. This next-generation network operation and management shall be based on the usage of Big Data Analytics techniques to monitor the end-user quality of experience through direct measures of the network. This paper describes the Astellia approach towards this network revolution and presents some results on the performance of quality estimation techniques in current cellular networks. Thanks to the use of this approach, operators may fill the gap of knowledge between network key performance indicators and user experience. This way, they can operate in a proactive manner and have actual measurements of the users’ experience, which leads to a fairer judgement of the users’ complaints.
Hinweise
Competing interests
The authors declare that they have no competing interests.
1 Introduction
The arrival of the 5G is expected by the year 2020. Meanwhile, various organizations and projects around the world are embarked on the process of describing the main characteristics based on the society to come in the next decade [1]. However, there is consensus in the three main characteristics of the 5G [2]. Firstly, 5G will be a heterogeneous network in which several radio interfaces could be applied to different use cases. This network is expected to be from three to five times more dense, in terms of number of access nodes, than current deployments, i.e., long-term evolution (LTE) Release 8, which poses severe challenges in the proper interference management [3]. Secondly, 5G networks must be highly flexible and adapt to the dynamism of the traffic location and patterns. For this, radio access network (RAN) functionalities may run in large computer centers, able to dynamically assign more or less units of computation to the virtual cells distributed in the network. In this context, a virtual cell grabs the needed resources from a common pool and uses a set of remote radio head (RRH) units to configure the optimum cell to serve the user [4]. As a result, 5G may rely on this new concept of cloud RAN that combines collaborative radio and the real-time cloud infrastructure with a centralized general-purpose processing solution [5]. Finally, operation and management of this 5G network must be simplified, including a new approach in which all focus is on the user satisfaction [6]. This requires advanced data analytics and the use of Big Data techniques to monitor the users’ quality of experience (QoE) by using network metrics that combine network and behavioral data [7].
Current key performance indicators (KPIs) used by operators are average metrics that hide the real experience of users and are particular of a certain technology. In fact, KPIs today do not directly target the surveys of satisfaction. This is why the user-centric paradigm that will characterize the 5G requires a redefinition of network indicators with a single measure of the user experience, independently of the radio technology. This paper focuses on the definition of user satisfaction and presents, to the best of authors’ knowledge, the first real-field trial to estimate the user satisfaction using only network metrics extracted from an operative network. Results have been validated with actual surveys of the monitored clients, revealing a clear potential of this new philosophy of operation and maintenance (Fig. 1).
Fig. 1
Overall SatiX process
×
Anzeige
2 Evolution of system operation and management
The Nokia Siemens Networks 2013 Acquisition and Retention Study showed that in average almost 40 % of users are likely to churn within a year timeframe, being loyalty lower in mature markets [8]. It is relevant to see that service quality is becoming one of the key drivers for such consideration [8], something that did not happen in the past, when motivation was mainly to replace the mobile device. In consequence, service operation and management needs to be redefined to put customers at the heart of daily operations. In this sense, operators need a complete and real-time picture of service behavior and performance for each individual customer, in order to propose customers appropriate and tailor-made contracts. This has proven efficient to increase significantly users’ satisfaction and subsequent fidelity.
Given the increasing complexity of mobile generation, the natural trend in system operation and management is to automate these processes, for which KPIs are the main inputs for decision-making. KPIs are aggregate metrics that measure the average behavior of the different network elements and help operators define and evaluate how successful the network is. KPIs, however, are not capturing the actual perception of customers, as evidenced by the fact that, in a recurrent way, operators see that network KPIs are very satisfying but customer service is facing an increasing number of subscriber complaints experiencing poor quality of experience. Moreover, KPIs are different for each radio access technology and cannot be combined in cases where the user is camping on different technologies along the session. Therefore, better models are needed to capture the relationship between QoE and measurable network service and context parameters, with a single measure of the user experience that is independent of the radio technology.
Big data and data analytics have an important role to play in this new paradigm of system operation and management [9]. First, user location must be known in real time, which represents the analysis of enormous amount of data, including the receive power levels from several cells of millions of users and/or their GPS positions. Moreover, in order to derive the user QoE simultaneously, it is required to correlate this location with other context data, including bearer allocation, traffic pattern, day, hour, social behavior, contract details, etc.
A classical computational approach is unable to manage such a huge volume, velocity, and variety of data. Therefore, Big Data solutions together with Advanced Analytics must be put in place to transform these complex data into easily manageable data units.
Anzeige
3 SatiX and the estimation of users’ satisfaction
Astellia together with the Universitat Politècnica de València has developed SatiX, a powerful customer-centric monitoring solution that correlates radio network metrics and end-user perception information. Customer experience metrics such as blocked calls, dropped calls, coverage issues, voice, and data quality are correlated with technical radio measurements such as received pilot power, signal-to-noise ratio (SNR), throughput, BLER, and duration of the connection. It is worth noting that the tool operates with temporal and hence anonymous identifiers. Therefore, there are not privacy issues in the management of such information. Of course, the operator can easily translate this temporal identifier to obtain the actual identity of the customer, being these data subject to applicable regulations in terms of data security and privacy management.
The result is an advanced customer satisfaction index that not only provides objective customer perception indicators but also identifies radio parameters to adjust network conditions and enhance customer experience.
At user level a series of magnitudes extracted from call traces are aggregated in order to calculate different metrics, e.g., the experimented average SNR in a dedicated channel or the voice call drop rate. These metrics are the starting point of the SatiX process calculation.
3.1 Calculation of metrics
There are several metrics used as inputs for the calculation of attributes. For the sake of simplicity, we give only details for the UMTS technology (see Table 1; interested readers can refer to [10] for a detailed description of these metrics), although a similar approach can be followed for other technologies, like GSM or LTE.
Table 1
List of metrics used in UMTS
Metric
Description
Range/definition
RSCP
Mean value of the best server cell
From −109 to −70 dBm
Ec/N0
Mean value of the best server cell
From −17 to −6 dB
Handover ratio
Number of calls that end in another technology
From 0 to 100 %
Voice drops
Percentage of drops in relation with the total calls in a day
From 0 to 10 %
Voice blocks
Number of total blocks in a day
From 0 to 10
Throughput DL
Considering all the session duration
From 50 kbps to 2 Mbps
Throughput UL
Considering all the session duration
From 8 to 250 kbps
Data reliability
Average data connection reliability given by the data drop rate and data block rate
(%Data Blocks) + (%Data Drops) (1−(%Data Blocks))
For each metric, we derive five levels of quality, from 1, the lowest quality, to 5, the highest quality. As an example, for the RSCP metric, levels are calculated as follows:
Level 5 if −85 ≤ RSCP
Level 4 if −93 ≤ RSCP < −85 dBm
Level 3 if −100 ≤ RSCP < −93 dBm
Level 2 if −109 ≤ RSCP < −100 dBm
Level 1 if RSCP < −109 dBm
3.2 Calculation of attributes
From the calculated metrics and different combinations made from them, specific thresholds are used in order to obtain a list of attributes, which are the basic components of the final SatiX value. Some of those attributes are obtained directly from a metric, while others are a combination of metrics. The list of SatiX attributes for UMTS is listed in Table 2.
Table 2
List of attributes used in UMTS
Attribute
Acronym
Description
Voice drops
AVD
Reflects the ability of maintaining a successful connected voice call
Voice blocks
AVB
Means the ability of successfully initiating a voice call while being in a coverage zone
Coverage
ACov
User coverage per cell and day
Speech quality
ASQ
Average voice quality in voice calls
Data quality
ADQ
Quality experimented in a data calls
As an example, the level of the coverage attribute is calculated as the average of RSCP and Ec/N0 metrics, whereas data quality takes into account, throughput in both directions, reliability and handover rate. These attributes are to be properly weighted in an offline calibration process.
3.3 SatiX results
The final SatiX result, which corresponds with a measure of the mean opinion score of the customer concerning the service provided by the operator, is calculated as a function of the attributes defined for the technology, with certain deviations depending on expectation. For this function, we used a weighted sum of attributes as follows:
where AVD corresponds with voice drops attribute, AVB with voice blocks, ACov with coverage, ASQ with speech quality, and ADQ with data quality.
The optimum weights for the components of the SatiX attributes were calculated based on customer’s surveys on a measurement campaign performed during 3 months, with more than 2000 phone surveys. With the collected data, we minimized the mean squared error of the difference between the SatiX value and the actual surveys, provided that the summation of weights equals 1, and being all weights from 0 to 1. This convex problem was solved using the interior point method [11].
A different solution that permits a non-linear combination of attributes consists in defining a neural network that uses attributes as inputs and provides SatiX as output. Some trials have shown that a three hidden layer neural network performs well.
Finally, when analyzing the SatiX mean result for users depending on the mostly used services, we appreciated significant deviations, which motivated the inclusion of several rules in the SatiX calculation. As represented in Fig. 2, some services, especially non-real-time services, make the general opinion be more optimistic. On the contrary, users mainly consuming real-time services, like video streaming, that require extensive use of data connection expressed a worse opinion on the network performance.
Fig. 2
Impact of the service in the users’ satisfaction
×
Once the calculations are finalized, there are available data about metrics, attributes and SatiX for each user, which is complemented with other available data from other sources, like zones or administrative/political levels linked to each cell in order to obtain results associated to geographical data. Users’ expectations are also taken into account to bias the obtained technical SatiX value. The source information to derive these rules is multiple, ranging from the application layer information, the tariff, users mobility, and the variability of the calculated metrics. As outcome, different types of results are generated: dissatisfaction rankings, SatiX/attributes/metrics maps at cell and administrative area level, temporal evolution graphs showing how satisfaction, or metrics evolved after a certain update in the network, etc.
4 Results and discussion
The distribution of the different results of SatiX differs from the results of the surveys. The following figure shows both distributions. However, there are certain similarities between both distributions. For instance, the number of users with SatiX values from 0 to 4 is low. On the contrary, SatiX values between 5 and 10 are the most common (Fig. 3).
Fig. 3
Comparison of SatiX values with actual polls
×
The following figure shows the difference in results of SatiX algorithm with polls. Detractors are defined as those customers experiencing good KPIs but still judging low satisfaction with the operators’ service. Mathematically, we say that a user is a detractor if the difference between the SatiX level and the survey level is greater than or equal to 4 points. Conversely, a promoter is a customer that even experiencing bad quality still provides a positive feedback on the service. Similarly to the detractors, promoters are mathematically characterized by a difference greater than or equal to 4 points. In all polls, there are 6 % of detractors and 11 % of promoters. For detractors, there is no difference between being a prepaid or post-paid customer (in both cases 6 % of the group are detractors), whereas this fact is important for the promoters since 9 % of post-paid users are promoters, being this percentage increased up to 21 % for the prepaid users (Fig. 4).
Fig. 4
Error between SatiX and poll results
×
The calculation of correlation between SatiX results and the polls is made using three different methods. The first one is based on setting a level from which it is considered that the survey has been positive or negative. For this method, there is a correlation of 77 or 78 % when neglecting promoters and detractors. The second method uses a scale from 1 to 5 and assumes that both surveys and SatiX fit if difference is 0 or 1. In this case, correlation is 73 %. If we extract the detractors and promoters, the level of success increases considerably up to 83 %. The last correlation method takes into account a scale from 1 to 10 assuming a hit if the level is the same or differs only in 1 level. In this case, the success rate is 60 %, being 72 % without detractors and promoters.
On the other hand, for business decisions, Google Earth mappings can be generated using a custom application (e.g., Fig. 5). The map is useful for the operations department to detect dissatisfaction areas where actions should be taken and to correlate them with other maps obtained through direct extraction of KPI, like quality maps.
Fig. 5
Map with the representation of average SatiX per area
×
Finally, it is possible to map the SatiX data with estimated locations making use of a trace-based location service (see e.g., [12]). In the resulting maps, which could represent average satisfaction per geographical tile as shown in Fig. 6, the operator can find patterns of dissatisfaction, as for example due to the mobility between two specific cells. In Fig. 6, red circles highlight areas with special dissatisfaction, pointing out the need for a more detailed analysis of the network.
Fig. 6
Map with a detailed representation of SatiX linked with a certain position thanks to location services
×
5 Conclusions
This paper has described a new approach for the definition of KPI, which fits better with the future needs of the 5G in terms of network operation and maintenance.
In the future, some aspects need to be revisited. In particular, the generation of SatiX results through the aggregation of information contained in data records should be studied carefully from the averaging perspective. Depending on the aggregation level, the averaging should be different. This could lead to heavy queries with very long execution times, which requires the new approaches of Big Data.
Finally, current approach is still vendor-specific, since different vendors have intrinsic differences when comparing their metrics. This could generate differences in the SatiX calculations depending on the specific hardware, which should be avoided. To solve this esthetical problem, a separate calibration for each vendor shall be done.
Acknowledgements
The authors would like to thank the funding received from the Ministerio de Industria, Energía y Turismo TSI-100102-2013-106 funds.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Competing interests
The authors declare that they have no competing interests.
