Researchers in mobile technologies and services are currently requesting testbeds to validate their proposals in a realistic large-scale and controlled environment. Such demands are some times satisfied with private ad-hoc deployments in a given area of coverage. However, given national regulations related to the use of the spectrum and the cost of deployments, these solutions are not feasible for most researchers. In this paper, we introduce the architecture of the research infrastructure developed in the EuWireless project. EuWireless’ objective is to provide private networks that can be dynamically created as slices with different levels of configuration and control, as a cost-effective way to access large-scale testbeds with a licensed spectrum even in different countries. The paper focuses on the architecture of the Point of Presence, the key element in the EuWireless project, and how it will work in practice. We also present a proof of concept implementation that satisfies some of the design objectives and demonstrates the feasibility of the proposal.
Notes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
1 Introduction
This paper discusses how to implement private networks for research projects avoiding the investment in ad-hoc equipment. The motivation behind this objective is the increasing need of many researchers and small companies to have access to mobile networks with full control for a short period of time. For a few days, they need access with the capability to configure specific radio parameters, to change core network configurations or to deploy services in specific network locations. However, as licensed spectrum is usually only available for commercial Mobile Network Operators (MNOs), the commercial equipment needed to set up mobile networks is only offered to MNOs, and researchers do not even have the opportunity to pay for them. This option is only feasible in a few countries, where spectrum is offered for experimentation in specific areas, like university campuses. But even in such cases, the cost of deploying and operating a network is not negligible and is not the best solution for public research institutes or small companies when they only need to deploy such infrastructure for short time experimentation.
This problem has already been addressed in several research projects, where large research infrastructures are offered to external experimenters through the open calls approach. For instance, the ITIS Software1 research institute is operating the TRIANGLE and 5GENESIS [1‐3] platforms at University of Malaga and has supported more than 20 experiments of SMEs. From this experience, we have identified the following requirements: (1) larger outdoor areas of coverage, (2) more mobility options, (3) different spectrum bands, and (4) interconnection with the researcher’s premises or with commercial users. The section on related work expands current solutions [3‐9], where these gaps still exist.
Advertisement
The EuWireless project [10] is designing a new solution to fill these gaps based on two main ideas. First, the interconnection of existing research infrastructures for mobile networks in Europe using specific interconnection nodes called Points of Presence (PoP). Second, the creation of private networks tailored to run specific experiments, called slices, on top of this infrastructure. Both concepts imply addressing a number of technical challenges, such as the management of distributed resources to create the slices, the support of parallel slices sharing the resources, and security and access control aspects.
The main result of EuWireless is a distributed research infrastructure that is able to dynamically provide slices (EuW slices) as a Service. EuW slices can cover small or large geographical areas and can integrate different types of resources (e.g. virtual machines, transport links, spectrum, research testbeds, etc.) and domains. EuW slices can integrate EuWireless’ own infrastructure as well as MNOs’ and researchers’ resources. This way, EuW slices can support not only short term experiments but also long-term services. One of such services, for example, is the EuWireless virtual Mobile Network Operator (vMNO), a pan-European MNO that will provide cellular connectivity services to European researchers. Thus, three main user profiles have been identified to benefit from the EuWireless concept: (1) mobile network researchers who can run experiments in controlled and realistic networks, (2) service providers that can dynamically deploy temporary (or long-term) services, and (3) the EuW subscribers that use the services provided over the different EuW slices. In previous work [11], we discussed a preliminary architecture for EuWireless that relies on the GÉANT Testbed Service (GTS). This paper presents a more comprehensive view of the final EuWireless architecture, based on distributed PoPs that are independent of GTS. Additional details for the architecture as a reference document for implementation is publicly available in deliverable D2.2 [12]. In addition, the paper presents an entirely new proof of concept implementation and some experiments that support the feasibility of the proposal. In this case, to accelerate the implementation and experimentation phases, the proof of concept relies on a GTS node extended with EuWireless’ components. Due to the lack of 5G Core open source implementations available at the time of writing the paper, we have used an LTE Core in this proof of concept to demonstrate the feasibility of deploying slices with cellular network components.
The paper is organized as follows. First, Sect. 2 presents a background on mobile networks and how they work. Then, Sect. 3 introduces the user profiles considered in the project and the design principles. Section 4 provides an approach to the PoP architecture, detailing the layers and components of a PoP, and presenting in Message Sequence Charts (MSCs) the workflows to create and delete a slice. Section 5 explains how to integrate network resources in the architecture and a proof-of-concept of the architecture designed. Finally, Sect. 6 reviews related work on experimentation platforms for mobile networks, and Sect. 7 summarizes the conclusions and future work.
2 Background on Mobile Networks
This section presents some high-level background on Long Term Evolution (LTE), commonly referred to as 4G, and 5G mobile networks.
Advertisement
When LTE was presented in 2008 as the evolution of the Universal Mobile Telecommunications System (UMTS) technology [13], in addition to an increased bandwidth and data rate, it brought a profound architectural change of the operators’ infrastructure. The new standard was based on data packet switching and followed an all-IP approach for the communication between all the network’s entities, instead of the circuit-based nature of previous generations.
This new architecture is usually viewed as three separate domains: the User Equipment (UE) [14] used to connect to the network, the radio devices (antennas and base stations) [15] that provide the physical link for the UE to connect to, and the operator’s internal packet systems [16] that manage the traffic between the UE and other UEs or external data networks, such as the Internet. Figure 1 provides a high-level view of such architecture.
Fig. 1
4G network high-level architecture
×
The UE is the equipment used by the end-user to connect and communicate with the operator’s network. It has two main components: the actual physical device (i.e., a phone or a modem) or Mobile Equipment (ME), and one element to identify the user to the network, usually stored in a Subscriber Identity Module (SIM) card provided by the operator.
The Radio Access Network (RAN) consists of a mesh of base stations, the so-called Evolved NodeBs (eNBs), which connect the users with the operator’s core network. An intelligent deployment to maximize coverage area and the separation of the control plane from the data plane of the traffic allows reducing the time needed by the UE to jump between base stations, providing a nearly seamless communication when the user moves.
Finally, the Evolved Packet Core (EPC) constitutes the core of the 4G network and provides much of the intelligence needed to run it. Instead of using a monolithic architecture, an EPC is composed of several entities tasked with different functions and services. This distribution of tasks maximizes the performance of each component and enables their replication if the need arises. The primary entities inside the EPC are the following:
The Mobility Management Entity (MME) that controls the connection and the user’s mobility between base stations, and maintains a single session for a particular UE between all the entities.
The Home Subscriber Server (HSS) that maintains the users’ database and the security of the connections.
One or more Serving and Packet Data Network Gateways (SGW and PGW) that connect the RAN with the EPC and the EPC with external networks, respectively.
There are also entities, for instance, to control charging and billing, connect to older mobile networks, measure and optimize the handover between base stations, etc.
The Fifth Generation of mobile networks, the so-called 5G networks [17], is an evolution of the 4G architecture and doubles down on the effort of separating functionality into different entities, both at the core and the RAN. It is designed to support a massive number of users with heterogeneous devices, meeting not only a high traffic demand but also Quality of Service (QoS) requirements of new and complex services. Observe that the 5G network is composed of three main subsystems similar to LTE networks, which correspond to the evolution of the UE (5G UE), the 5G New Radio (5G NR) composed of new base stations called gNodeBs (gNBs), and the 5G core network, that comprises the 5G Network Functions (NFs).
A large part of the performance improvement in 5G is achieved by decoupling, even more, the functionality of the operator’s core network into new entities. For example, the MME of 4G is split into two different network functions: the Access and Mobility Management Function (AMF), which is responsible for only handling connection and mobility tasks, and the Session Management Function (SMF) to coordinate session synchronization between other NFs. The same happens with the HSS of 4G, as it is now divided into separate functions for user data management [the Unified Data Management (UDM)], authentication procedures [the Authentication Server Function (AUSF)], and a database with user profiles and encryption key management [the User Data Repository (UDR)]. This decoupling allows each entity to be used by new services independently without the need to parse the complete messages and protocols of the previous architecture.
To support the massive number of connections expected in this mobile generation, and meet the new QoS requirements, 5G networks introduce the concept of network slices, which are private and isolated virtual networks on top of a common shared physical infrastructure. The implementation of network slices relies mainly on three technologies: Network Function Virtualization (NFV), which allows the virtualization of core and network functions, Software Defined Networking (SDN), which supports the programmability of the network to separate data and control planes into distinct network topologies, and Mobile Edge Computing (MEC), which increases the flexibility of the network by moving part of the core close to the end user. Figure 2 shows the high-level architecture of a 5G network. Thus, a 5G network slice [18, 19] includes exactly the network functions required by a service to achieve the expected performance, using a common infrastructure that can be dynamically configured to put some NF near the end user. The EuWireless project aims at offering network slices, called EuW slices, to its users for experimentation and service provision.
Fig. 2
5G deployment over a SDN architecture
×
3 EuWireless User Profiles and Design Principles
EuWireless aims to create user-defined slices to run experiments for a limited period of time and permanent services over mobile networks. This section introduces the different user profiles that will benefit from the provision of the EuW slices, which aim at experimenting with wireless network components, but also at providing services. Finally, the section summarizes the design principles required to provide the EuW slices.
3.1 User Profiles
To facilitate the creation of slices, we have defined a generic slice template that includes the basic requirements, that is; the configurable attributes for every single resource, for the possible experiments and services supported by the EuW slices. Several sub-templates can be derived from the generic one just by establishing a set of values for some attributes. An EuW user can modify the values of some attributes to refine the slice to better suit the specific use case under test. All of this results in distinguishing user profiles according to the granularity of the slice description. Thus, we define two main user profiles, researcher and service provider, and differentiate specific cases for each of them, depending on the service or the level of ownership of the resources involved in the experiment, respectively. Additionally, there is a kind of user that benefits from the EuW slice itself by making use of the service running over it. We have called this type of user an EuW subscriber.
3.1.1 Researcher
The researcher profile corresponds to an EuW user that has a clear picture of the mobile network needed to run some experiments. The slice template must be adapted according to the part of the network under test, and also to the ownership of the resources included in the slice. For instance, if the EuW slice integrates resources owned by the researcher, the slice description has to include links that connect these external resources to the slice. An example of an EuW user that matches this profile is a researcher who studies the RAN subsystem. These researchers can provide details of the configuration of RAN elements, such as different eNBs, RAN controllers, and components to share and manage the spectrum according to the License Sharing Agreements (LSAs) with the MNO. However, the configuration of other subsystems can be delegated to EuWireless.
The researcher profile also fits industrial users. For example, an EPC vendor, in order to test a new EPC component, needs subscribers in a geographical area and the RAN subsystem, and has to detail the interfaces between the RAN and the EPC to deploy a fully functional mobile network. The EuW subscribers (real or emulated) would then make use of an emulated service through that EuW user’s core network, this being a specific hardware located in the vendor facilities, as shown in Fig. 3. Another example would be the core network researcher that does not own facilities, so the EPC must be deployed as software. In this case, EuWireless would also provide computational resources to run that software.
Fig. 3
Slice with EPC as researcher’s resource
×
3.1.2 Service Provider
The service provider profile is an EuW user that needs a mobile network to evaluate a service but does not care about the netowrk’s low-level configuration. As mentioned above, if the EuW slice integrates services or network resources owned by the EuW user, the slice description has to specify the links that connect these external resources to the slice. In the service provider profile, the description of the slice must include the service requirements [obtained from the Service Level Agreement (SLA)] and the connectivity to the service provider. The configuration of the underlying network (e.g. RAN and EPC components) will be automatically derived by EuWireless according to the provided service requirements.
The EuWireless vMNO is an example of a service provider, in which the EuW user is the EuWireless consortium itself, the subscribers are all real, and the resources needed are assigned long-term to that slice. As a result of having subscribers from commercial MNOs, the EuW coverage area is extended with the commercial one, so the EuW subscribers have access to the service regardless of their location. Bearing all of this in mind, the EuW vMNO would need the deployment of a complete mobile network to provide connectivity to the subscribers, including communication with the authentication server and the commercial MNOs.
3.1.3 EuWireless Subscribers
The EuW subscribers are the final users of the slice, and they can be real or emulated by creating several UE instances. In the case of real EuW subscribers, there are two types: on the one hand, EuW subscribers needed to run the experiments (e.g. stress and load tests) and obtain valuable results; and on the other hand, those subscribers who benefit from the service deployed on the slice, i.e. the case of the EuW vMNO providing connectivity to these EuW subscribers.
3.2 Design Principles
EuWireless identifies five basic design principles, as stated by Rios et al. [11]:
Support for isolated concurrent EuW users: slices from both researchers and service providers should be able to run in parallel, in a way that physical resources are shared among them without interfering with each other.
Pan-European deployment: since one of the objectives is to offer the same access opportunities to all European researchers regardless of their location, we propose a distributed infrastructure in which the coverage area is extended by deploying cross-country PoPs, and each EuW user accesses to the closest available PoP.
Scalable PoPs as the main core objects: the distribution of the architecture is possible thanks to the deployment of sets of resources in different locations across Europe, the so-called PoPs, increasing the infrastructure’s local performance and scalability. Hence, the project can be extended by deploying a PoP in a new location and interconnecting that PoP with the rest of the infrastructure to make its resources available for every EuW user.
Adaptable to integrate new mobile technologies: even though 5G technology is yet to be deployed on a large scale, the project aims at integrating novel technologies to be tested together with the current generation of mobile networks. Thus, we expect the project to allow an abstraction and extension such that new paradigms can be easily accommodated.
Fully automated: the process of slice provisioning should be automatic, enabling EuW users to define their slices and reserve the resources they need, whereas EuWireless applies fair user policies to prevent resource exhaustion from greedy experiments.
4 Architecture of EuWireless PoP
Three different architectural approaches were considered in [11] to meet these design principles. After studying the advantages and disadvantages of each option, the consortium decided in favour of the solution shown in Fig. 4, which abstracts the distributed nature of the research infrastructure and lets the user dynamically define slices based on slice needs and resource availability.
Fig. 4
EuWireless infrastructure overview
×
The EuWireless research infrastructure is supported by a set of distributed Points of Presence (PoPs) that transparently manages local and remote resources. EuW users access the EuWireless infrastructure through a web portal or an Application Programming Interface (API), where they can design slices by customizing or extending some of the available slice templates.
The PoP follows the layered architecture shown in Fig. 5, which groups the functional entities based on the scope of their tasks as follows:
EuW Portal and API: it is the entry point for EuW users to define and manage slices for experimentation.
Inter PoP: this layer has a global view of the EuWireless infrastructure and performs tasks involving multiple PoPs and slices. The PoP Directory is located in this layer and stores information on the rest of the PoPs comprising the infrastructure, including their local resources (not their availability) and their authorized users.
Intra Slice: this layer is in charge of managing a slice previously designed and deployed, which can span across one or multiple PoPs. The Slice + NF Catalogue entity is located in this layer, and stores the mapping between the slice’s abstract description and the actual resources. Since it is likely that a EuW user will employ a slice template or reuse the same slice description several times, the mapping information will be available to speed up the slice construction phase.
Intra PoP: this layer interacts with a PoP’s local resources. The local resources can have different administrative domains; that is, a PoP’s local resources can be owned by the EuWireless infrastructure, by a commercial MNO that shares its resources with EuWireless, or by a researcher that wants to include his/her resources into an EuW slice. This layer includes the Infrastructure Catalogue, which keeps information on the resources locally managed by the PoP and their availability.
Communication: this layer provides the connectivity services to the upper layers in order to effectively communicate with other PoPs or with the local resources.
Fig. 5
PoP architecture
×
The rest of the section presents the main entities of a PoP and some workflows to illustrate the interaction among them. To simplify the presentation in the workflows, we assume that the EuW user has been previously authenticated. The authentication procedure is delegated to the local PoP’s Security entity, which provides a reliable and secure service for both EuW users and other PoPs by maintaining the list of users and their access profiles for the local PoP. Given the heterogeneous types of EuW users (affiliated to different academic and research institutions) and the distributed nature of the infrastructure, the Security entity supports external Authentication, Authorization and Accounting (AAA) services such as RADIUS, LDAP, etc., so each institution can apply its policies on access control.
4.1 Slice Creation
The definition of a network slice relies on a modular virtualization system that supports an extensible catalogue of (abstract) resources that can be mapped to a single o a set of physical (or virtual) resources. Each resource exposes a set of attributes that can be tuned to fit the user’s needs, as well as the I/O communication ports in order to interconnect with others. These data paths that interconnect resource I/O ports define the network slice topology. In order to deploy a network slice in EuWireless, the EuW user has to define the set of resources integrated in the slice, configure their attributes, and define the underlying network slice topology.
Fig. 6
Slice creation using resources of multiple domains
×
Figure 6 describes the workflow to create a network slice that integrates resources distributed in different PoPs. The slice creation starts when the EuW user requests the creation of a new slice (configuration of resources and topology) using the EuW portal (or the API). In the multi-domain network slice scenario, different PoPs have to be coordinated in order to reserve and deploy the resources included in the slice. The Slice Orchestrator of the local PoP (the PoP associated with the EuW user) is the entity in charge of coordinating the whole process, as it is the PoP’s main administrative manager. It receives the request to create the network slice (including the slice description in terms of resources and topology) and coordinates the blocking of resources in the local and remote PoPs.
In the slice creation process, the Orchestrator delegates validation of the slice definition to the Validator. The Validator performs a syntax check to ensure that the slice definition conforms with the syntax of the slice description language, and a semantic validation to check parameter misconfiguration, such as exceeding the capacity of a link, inconsistencies in the deployed components or scenarios that can compromise the performance of the EuW infrastructure. It should be noted that the Validator only performs the syntactic and semantic correctness of the initial slice definition without considering the current resource availability.
Once the design is cleared, the Orchestrator proceeds to block local resources, so no other Orchestrator finds them available. In the case of remote resources, the Orchestrator sends the query to the Orchestrator of the corresponding remote PoP where the remote resources are connected. The Orchestrator maintains the database that accounts for infrastructure usage. Thus, every resource used in any slice is registered in its database alongside the slice identification and the user requesting it. This way, as it is also the main sink of information of the monitoring procedures, it commands the release of resources that are in an inconsistent state or that have been orphaned by a user disconnection, and informs any other remote Orchestrator of such events in order to act accordingly. At this stage, blocking a resource does not entail communication with the resource itself; it is just a change in its availability in the corresponding Infrastructure Catalogues.
Finally, when all the resources are blocked, the slice definition is sent to the Slice Constructor, which will configure and interconnect the resources following the topology defined by the slice definition. The Constructor carries out the reservation and configuration of the resources according to the slice definition and its mapping into the infrastructure. The Constructor obtains the mapping from the Slice+NF Catalogue, if it is a reused slice, or can request this task from a dedicated entity, called the Slice Designer. Given the resource mapping, the Constructor proceeds with the reservation and configuration of local and remote resources through the corresponding Resource Manager.
If a given slice definition does not match any of the existing templates or definitions in the Slice+NF Catalogue, the Designer maps the slice definition to virtual or physical infrastructure resources. To this end, the Designer first, takes the role of an architect that lays out the high-level topology of the slice definition, using abstract information of the resources of the PoPs involved. Second, it translates the previous topology into the infrastructure resources with specific requirements (e.g. a link with a specific bandwidth, a virtual machine with a number of cores and RAM, a chain of NFs, etc.). It is worth mentioning that the Designer does not ensure the resource availability; it just finds the appropriate resources to construct the slice and passes them to the Constructor.
In the multi-domain scenario depicted in Fig. 6, the Slice Constructor communicates with the local and remote Resource Managers, which are the entry points to configure and manage the resources associated with each PoP. In the slice lifecycle, each physical (or virtual) resource has to be reserved, configured, activated, and released. However, different types of resources need different configuration parameters, and, in general, they can present a different management interface. The role of the Resource Agents (RAs) is abstracting and unifying these common tasks, from the perspective of the Resource Manager. Thus, each resource in the EuWireless infrastructure has an associated RA that abstracts its actual management into a series of standardized primitives. In particular, the Resource Manager manages a resource through the specific RA based on six control primitives, all of them must be implemented in every RA: Reserve(), to allocate the physical resources and create the instances; Activate(), to configure the instances and put them in service; Reconfigure(), to change configuration parameters and attributes after the first installation; Query(), to obtain information about the resources’ state; Deactivate(), to stop the instances without releasing the resources; and Release(), to destroy the instances and return the allocated resources to the available pool. Thus, the RA is in charge of translating these primitives into the specific sequence of commands understood by the target resource.
When all resources are configured, each Resource Manager informs the Slice Constructor, who passes the management of the slice to a specific Instance Manager and informs the local Orchestrator, that finally, commands the slice activation and informs the EuW user of the successful creation of the slice. From that point onwards, the Instance Manager deals with the EuW user’s request related to his/her slice, translating the user request to specific requests or queries to the underlying resources that can be distributed in multiple PoPs. In addition, the Instance Manager takes part in the monitoring tasks in two different ways: it serves as a proxy for the monitoring requests sent by the EuW user but, more importantly, it is also responsible for the low-level monitoring of every resource, aggregating messages by components before sending them to the Monitoring Manager, and paying attention to any alarm or exception thrown by the resources used in the slice it manages.
Obviously, errors can occur during the creation of a multi-domain slice, from an invalid description of the slices, resources that are not available, or connectivity problems with external domain resources. In these cases, the solution is to release or unblock the resources (if any) and inform the EuW user of the problem.
4.2 Slice Release
When the EuW user does not need a slice anymore, he/she should destroy the slice. This way, the slice resources are available once again and can be used in other slices. Figure 7 shows the successful release of a slice. The corresponding Instance Manager receives the request and propagates the request to the local and remote Resource Managers that will send the deactivate and release primitives to each resource. When all the slice resources are released, the Instance Manager has nothing to supervise and passes the control to the local Orchestrator, which updates the Infrastructure Catalogues with the newly available resources, in collaboration with the remote Orchestrators.
Fig. 7
Slice release
×
During the lifecycle of a slice, the EuW user can temporarily deactivate a resource without releasing it, for instance, to run a particular experiment in which the resource is not required. This process is similar to the slice release. However, in this case, the deactivated resource remains reserved and cannot be used in other slices. Hence, the Instance Manager sends a request to pause the resource instead of releasing it, and then waits for user interaction to resume its operation through the EuW Portal. During this stage, the paused resource remains unavailable in the Infrastructure Catalogue. Observe that the temporary deactivation of resources modifies the configuration of the slice and could lead to unstable states. It is not the responsibility of the Instance Manager to ensure the correct operation of the slices after the initial deployment. However, the EuW user can use the monitoring capabilities to detect its performance. Further detail on this and other procedures is available in [12].
4.3 Other Entities
Some entities appear in Fig. 5 but not in the MSCs to avoid overloading the workflows with information. These entities are the MNO Broker and the monitoring components.
As mentioned in the introduction, one of the key features of EuWireless is the possibility to integrate resources from different administrative domains into a slice in a transparent way, especially resources shared by commercial MNOs, such as the radio spectrum. The MNO Broker acts as middleware, exposing on one side an interface as similar as possible as the one of a RA, and on the other side conforms to any requirement the operator may impose to access its resources. The reason for distinguishing this component from a RA is that all operators are going to impose their own rules and access restrictions to their resources. Hence, each PoP has an MNO Broker that conforms to the requirements of the MNO of its area.
Regarding the state of the elements composing any slice, the Resource Monitor supervises the RA health parameters using low-level primitives. In addition to the usual polling mechanism to read its value, the Resource Monitor can also be configured to trigger an alarm when a parameter of the resource that is being monitored reaches a certain threshold. To generate a monitoring report of a slice, the monitored information of all resources composing the slice have to be aggregated by the Instance Manager that handles the slice. Finally, at the Inter PoP level, the Monitoring Manager collects the monitoring information of all the slices associated to the PoP and generates meaningful reports for the EuW users.
5 Integration of Resources in the PoP and Proof-of-Concept
As described in Sect. 4, Resource Agents act as a middleware abstracting the underlying resources so the Resource Manager can handle them regardless of their nature or administrative domain. Hence, each kind of resource has an associated RA, which is able to control the resource’s lifecycle and trigger the transition among its different states, namely reserved, active and released. The technology that inspires this approach is GTS, a production service created by GÉANT in 2013. GTS is a tool for researchers to create isolated virtual testbeds within a shared infrastructure, offering highly flexible and efficient control of different networking components [20].
One of the direct advantages of this abstraction is the possibility of adding new resources in a flexible way, as long as there exists a specific RA to control these resources through the control primitives.
Each resource is defined through the requirements, ports, and attributes that the EuW user must specify before any deployment. Some of these attributes might be reconfigured after the resource has been activated and tested. The slice templates, which are available in the EuW Portal, facilitate the definition and initial configuration of the resources, since they assign default values to the attributes if the user does not specify any of them.
This section presents two examples of resources included in the EuWireless infrastructure, an EPC and a eNB, which are essential to deploy LTE network slices for research on mobile communications. Furthermore, following their integration, we present the management of these resources and how they can be used to deploy a slice for simple research, together with an example of the template used to define the characteristics of this slice.
Table 1 presents the definition of a virtualized EPC, that is, the set of requirements, attributes and ports that has to be properly configured in order to deploy an EPC in a slice. In addition, the table includes how the RA translates the control primitives into specific actions on the resource. In this case, the resource is a Virtual Machine (VM) running an open-source implementation of the EPC, the so-called NextEPC2, which corresponds to the LTE 3GPP Release 13. Thus, after the user specifies the value of the attributes, EuWireless automates the provisioning of resources, and the RA configures and activates the resources provisioned. In the table, the first row includes the requirements in terms of computing resources to prepare the scheduling and to reserve them. The second row corresponds to the ports, which define the slice topology as they indicate the data paths from and to each resource instance. The third row lists the specific attributes for an EPC type resource, which users can tune to their needs. Among these attributes, some information on the MNO is needed to register a subscriber, such as the Access Point Name (APN), the Tracking Area Code (TAC), the Public Land Mobile Network (PLMN), the authentication parameters, and the list of IP addresses and subscriber identities. The last row briefly introduces the lifecycle control primitives performed by the RAs.
Table 1
Example for the integration of a virtualized EPC
Requirements
CPU
1 processor
RAM
4 GB
HDD
25 GB
OS
Ubuntu 18.04 distribution with NextEPC software
Ports
S1-MME
Connects the EPC with one or multiple eNBs and transports control plane information
S1-U
Connects the EPC with one or multiple eNBs and transports data plane information
SGi
Connects the EPC with external IP networks
Attributes
Location
PoP selected to instantiate the VM with the EPC software
PLMN
Public Land Mobile Network, includes the Mobile Country Code (MCC) and Mobile Network Code (MNC)
TAC
Tracking area code
IP Pool
List of IP addresses available for the UEs
Subscribers
List of subscribers identified by their International Mobile Subscriber Identities (IMSIs)
APN
Access Point Name of the network providing internet access
Auth.
K code and OP/OPc authentication
param.
parameters
Primitives
Reservation
Locates the requirements needed to instantiate the VM in the PoP and the links available for definition of the ports
Activate
Installs the OS, configures the attributes specified by the EuW user, and initializes the entities composing the EPC
Deactivate
Stops the VM and the traffic flow, but maintains the existing connections and resources
Reconfigure
Changes configuration parameters and attributes after the first instantiation
Query
Provides information about the state of the resource without triggering a transition among states
Release
Deletes the VM and links associated to the ports, returning the resources to the pool available in the PoP
The second example, the eNB, abstracts a physical resource that radiates in a specific coverage area where the EuW user can allocate subscribers. In this case, the RA implements the primitives to control the eNB, and the resource has one port to communicate with the PoP where the rest of the slice is deployed. Table 2 shows the requirements, attributes and ports of the eNB type resource. Observe that being a physical resource, the only requirement to reserve the eNB is the availability of the physical resource itself, so there is no need to specify any other requirement in this case.
Table 2
Example for the integration of a physical eNB
Ports
S1
Connects the eNB with the EPC
Attributes
Location
PoP where the resource is placed
IP address
If the resource has no IP address fixed, IP address of the eNB to configure the EPC accordingly
Primitives
Reservation
Locates the physical device in the area requested and the availability of the specific links for the connection
Activate
Set the link to allow the traffic flow between the eNB and the EPC
Deactivate
Stops the traffic flow, but maintains the existing connection and the resource
Reconfigure
Changes the value of the IP address after the first instantiation
Query
Provides information about the state of the resource without triggering a transition among states
Release
Deletes the link associated to the port, returning the resource to the pool of resources available in the PoP
5.1 Proof-of-Concept
In order to show the feasibility of the EuWireless architecture, we have carried out a proof-of-concept that consists on the deployment of a single EuW PoP located at the University of Malaga premisses. We have carried out different experiments to demonstrate that the EuWireless PoP fulfils the following design objectives: support for isolated and concurrent EuWireless slices, guarantee remote access to EuWireless users, automated process for slice provisioning, and the possibility to integrate multi-domain resources as well as mobile technologies.
Figure 8 presents the proof-of-concept, including the PoP and other auxiliary equipment. The PoP relies on an isolated GTS point of deployment extended with two new types of resources, namely the ECP and the eNB. The extension relies on the implementation of the two RAs described above. Apart from these cellular resources, the PoP can instantiate resources such as virtual machines and virtual circuits. With all these resources, any EuW user can remotely deploy a full operative LTE slice on top of this EuW PoP. In addition to the PoP, we use a PC to design and manage slices remotely, and a pair of User Equipment (a PC with an LTE dongle and a mobile phone) with SIMs cards to test if the LTE slice works correctly. In the experiments, we design and deploy three different network slices, as shown in the upper part of Fig. 8.
Fig. 8
Experiment performed as Proof-of-Concept
×
The first step is to demonstrate that any EuW user can access a PoP remotely in order to design and manage a slice. This is achieved through the EuW Portal, which is a web portal running on top of the PoP that provides authenticated access. Currently, although the portal is publicly accessible3, only registered members of the Morse team have been granted access. Figure 9 shows the set of slices deployed that are running as part of the Demo project. Observe that there are three active slices, which correspond with the ones described in Fig. 8. The first slice runs an isolated virtual machine (provider id 483). The second slice (provider id 484) contains two virtual machines interconnected with a link. The third one (provider id 486) runs a fully operative LTE network with an EPC and an eNB, resources which are connected by a link.
Fig. 9
Proof of concept EuWireless portal over GTS
×
Besides the capability of remotely deploying different slices, another feature of EuWireless is to encompass multiple administrative domains. To prove this capability, we integrate the eNB resource in the PoP as an external domain resource. This integration implies that, although the eNB is interoperable across all the resources composing the slice, the low-level configuration, e.g., assigning the IP address, is external to EuWireless. Thus, the eNB resource in the LTE slice belongs to the EuW user, and EuWireless owns the EPC and the links. This characteristic is not explicit in Fig. 9, but it is reflected in the slice definition.
As commented in the previous section, the slice is described using the slice description language. Figure 10 shows the description of the LTE slice using a language based on GTS’ domain-specific language. In the slice description, the EuW user specifies the low-level configuration of EuWireless’ own resources. For example, in Fig. 10 the description includes different attributes of the EPC, such as the IP pool, which determines the set of IP addresses available to be assigned to the UEs. Each time a EuW subscriber registers in the network deployed in the slice, the subscriber’s UE obtains one of these IP addresses. In this case, the IP pool attribute is set to 45.45.0.0/16. Other parameters configured are the IP address of the MME for the interface S1, the list of IMSIs accepted in this EPC for the subscribers, the PLMN of the MNO, the TAC, and the authentication parameters. In contrast, the slice description only specifies the port to connect the eNB (external domain resource) to the slice. The low-level configuration of the eNB is done through its proprietary configuration mechanism. The EuW user can also specify the bandwidth of each link to be guaranteed as part of the SLA. The default value for the bandwidth, unless the EuW user states a more restrictive one, is 1 Gbps.
Fig. 10
LTE slice description
×
Thanks to GTS, slice provisioning and deployment is automatically performed. The following step is to check that the slices run properly. Since the first two slices only include legacy resources from GTS, we can rely on their correct isolated execution. Thus, we focus on checking that the third slice behaves as an LTE network. Inspired by the work presented in [2], we have designed a simple experiment that researchers can use to evaluate the performance of a service or application running in a mobile network. In particular, we have run a point-to-point video streaming session over the LTE slice using two mobile subscribers, as depicted in Fig. 8. Both subscribers are in the eNB coverage area and are thus connected to the same eNB and communicate through the virtualized EPC, which is deployed on top of the Malaga PoP.
The UEs used by the subscribers are a computer with an LTE modem and an Android mobile phone, each one using EuWireless SIM cards. Both subscribers use VLC software to perform the streaming session, where the computer acts as the streaming server, and the mobile phone acts as the client. This preliminary experiment includes the following steps. First, the EuW user accesses remotely to the EuW Portal and defines the slice with the resources involved and their attributes. Once the definition of the slice is validated, the reservation and the activation of the EPC, the eNB, and the link between them start. When all the resources are in an active state, the slice is ready to provide the network service, and the UEs are registered in the EPC to start the video streaming service. When the transmission finishes, the EuW user deactivates and releases the resources.
With this simple configuration, we perform multiple streaming sessions, which are monitored with the Wireshark packet sniffer running in the EPC. Figure 11 shows the attach procedure of one of the UEs. The first set of messages depicted are exchanged between the MME (one of the network functions included in the EPC), with IP address 10.102.81.35 (as defined in Fig. 10), and the eNB, with IP address 10.102.81.60, configured externally by the EuW user that owns this resource. The last three messages show the registration of the UE once the attach procedure is complete. In this registration, the UE is assigned with the IP address 45.45.0.10. The other UE, with IP address 45.45.0.9, was already registered in the network and acts as the video streaming server. When both UEs are registered in the network, we start the streaming session observing that the video is properly displayed on the client side, obtaining an average throughput of 14 Mbps for video transmission with 50 seconds of duration and a segment length of 1460 Bytes.
Fig. 11
Server UE attach
×
The last experiment demonstrates the isolation of the slices and the preservation of the performance capabilities regardless of the number of slices running in parallel. Two slices are deployed beside the LTE previously discussed, both with the network 45.45.0.0/16 configured to test the isolation. Even though the three slices are running on top of the same shared physical infrastructure, the UEs are not reachable from the nodes outside the LTE slice. Some transmissions were conducted to verify that sharing the physical infrastructure does not affect the performance of the network slices.
Figure 12 depicts the results of several TCP and UDP transmissions using iPerf3. Different tests were performed changing the bandwidth (from 10 to 15 Mbps) in UDP and the number of active slices (1 or 2) for both protocols to check the effect on the transmission. The bandwidth of the TCP transmissions is not fixed because it depends on the implementation of the iPerf tool used, which results in the fluctuation between 0 and 6 Mbps. In the UDP transmissions, however, we can observe the average bandwidth expected for each case. The three transmissions (TCP, UDP at 10 Mbps, and UDP at 15 Mbps) were repeated with another slice in parallel exchanging traffic at 1 Gbps to check its consequences on their performance. As shown in Fig. 12, the average bandwidth in these cases is not affected by the operation of the second slice.
Fig. 12
Bandwidth obtained in the TCP and UDP transmissions
×
In future work, we will evaluate the flexibility of slices to support more elaborated experiments, for example, to run experiments with different network configurations and scenarios, which is very valuable for researchers.
6 Related Work
Several initiatives have been created to provide the research community with access to commercial network equipment. This is the case of the program Future Internet Research and Experimentation (FIRE) [4], led by the European Commission, which counts on different technology laboratories distributed across Europe and equipped with state-of-the-art components. However, these laboratories only allow basic research on mobile networks since they are fixed and restricted, which logically limits research on wireless mobility. As a consequence, mobility has to be simulated by other means [21, 22]. On another note, the US National Science Foundation sponsored a virtual laboratory for research in networking and distributed systems, the so-called Global Environment for Networking Innovation (GENI) project [5, 23]. The GENI laboratory focuses on computer networks, although they have LTE base stations operating in the licensed Educational Broadband spectrum to deploy mobile edge testbeds outdoor at multiple campuses [24].
Regarding the projects oriented to wireless networks experimentation, the PAWR program in the US is currently supporting two projects: Cloud Enhanced Open Software Defined Mobile Wireless Testbed for City-Scale Deployment (COSMOS) and the Platform for Open Wireless Data-driven Experimental Research Reconfigurable Ecosystem for Next-gen End-to-end Wireless (Powder-RENEW). The project COSMOS [6] centres its architecture on edge cloud computing to achieve ultra-high bandwidth and low latency wireless communication to support city-scale real-world experiments. The platform makes use of the ORBIT Management Framework (OMF) [25] to perform control, measurement, and management tasks. Powder-RENEW [7] offers researchers a facility to perform 5G radio testbeds in a scaled, real-world environment. Regarding the architecture, the control layer has the Emulab [26] Control entity as the centralized system’s main component. In Europe, the project 5GinFIRE [27] presents an evolved version of the FIRE environment for experimentation in 5G vertical industries. The architecture of this approach adopts the recommendation of the ETSI NFV reference.
The 5G Infrastructure Public Private Partnership (5G PPP) [28] is another European initiative in which the European Commission is associated with the Information and Communications Technology (ICT) industry to provide a competitive European industry and compete in global innovative and technological markets.
In 2017, the 5G PPP started its Phase 2 program to fund 21 projects focused on validation of 5G technologies and their application to several vertical sectors. Among these projects, we should highlight 5Gtango, MATILDA, and SliceNet. The 5Gtango [29] project is oriented to the development and validation of specific network services and applications. For this purpose, they propose a centralized architecture with a Service Development Kit (SDK), a Service Platform, and a Validation and Verification Platform, the last two of which are in contact with the system’s orchestrator, normally OSM. The MATILDA project [30] is focused on the design, development, and orchestration of 5G-ready applications and network services. The project includes a multi-site virtualized infrastructure manager and a multi-site NFV Orchestrator (NFVO) to distribute the architecture and coordinate it by using a multi-site service conductor. However, as stated in [31], each Domain-Specific Orchestrator (DSO) will be handled by the Global Orchestrator (GO), and thus the orchestration remains centralized. The SliceNet [32] project is an end-to-end slice management framework for virtualized multi-domain and multi-tenant 5G networks. Hence, SliceNet presents a Cross-Plane Orchestration system to differentiate three planes: services, slices, and resources.
The Phase 3 program, which started in 2018, includes three infrastructure projects to provide 5G end-to-end facilities that can be used by several vertical industries to set up research trials meeting the 5G Key Performance Indicators (KPIs). The 5GENESIS [3] project, in which ITIS Software participates, stands for “5th Generation End-to-end Network, Experimentation, System Integration, and Showcasing”, and aims to validate KPIs in different use cases. The project is currently deploying across Europe five testing platforms that follow the same reference architecture in order to interoperate and expose APIs to verticals. The 5G-VINNI [8] project stands for “5G Verticals INNovation Infrastructure” and differentiates two kinds of facility sites: the main ones that support other projects defined by their SLAs, and the experimentation facility sites that support experiments whose results serve as input for the project implementation itself. The 5G EVE [9] project stands for “5G European Validation platform for Extensive trials”, and comprises four heterogeneous facilities located in different European countries to perform experimentation and validation of 5G pilots.
A common feature observed among these federation-based and 5G PPP approaches is the centralized orchestration of their architecture. However, centralized orchestration entails diverse constraints for networks, such as scalability issues and limitations in the automation of the network. In this context, EuWireless proposes to distribute not only the resources but also the orchestration into the different PoPs that compose the infrastructure to offer a research environment consisting of real resources spanned across Europe and coexisting with commercial operators thanks to the spectrum manager for Licensed Shared Access (LSA). The dynamic spectrum management falls beyond the scope of this paper, yet it is addressed in the EuWireless project, as stated in [33, 34]. In addition, the remarkable advantages of the EuWireless project comprise the seamless mobility due to Eduroam, the inclusion of commercial users in the experiments, and the extensibility of the resources available in the PoP with researchers’ components and beyond 5G elements that are orchestrated in a multi-administrative domain environment. All of these are shown in Table 3 in comparison with the related work presented in this section.
Table 3
Comparison of related work
Project
Coverage
Mobility
Spectrum
Extensible
Orchestration
EuWireless
Pan-European deployments
Eduroam, V2X
Commercial (shared LSA)
Yes
Multi-domain, distributed
FIRE
Fixed labs (Europe)
Per testbed
Per testbed
Per testbed
Per testbed
GENI
US universities
Per project
Educational broadband
Per project
Per project
COSMOS
US NY city sector
V2X
Commercial 5G
Yes
Centralized OMF
Powder
US Salt Lake city sector
V2X
Commercial 5G
Yes
Centralized Emulab
5Gtango
No
No
No
Yes
Centralized OSM
MATILDA
No
No
No
Yes
Multi-site, centralized
SliceNet
No
No
No
Yes
Multi-domain, cross-plane
5GENESIS
EU city sectors
Yes
Commercial 4G and 5G
Yes
Centralized, per platform
5G-VINNI
EU city sectors
Yes
Commercial 4G and 5G
Yes
Centralized, per platform
5G EVE
EU city sectors
Yes
Commercial 4G and 5G
Yes
Centralized, per platform
7 Conclusions
This paper provides a solution offering temporary research networks in the context of mobile networks with a low cost for the experimenters. The architecture presented in the paper is mature and detailed enough to proceed with the implementation of the EuWireless Point of Presence. Since there are several implementation options, the project envisions the integration of different technologies in a heterogeneous Point of Presence. The project has already deployed some proofs of concept that involve the implementation and integration of different RAs to create a complete E2E LTE network on top of the EuW PoP available in Malaga. This demonstrates the feasibility of the approach and is the basis for further implementation. The implementation of a large infrastructure like EuWireless at a regional, national, or global level would have a real impact on the manner in which researchers and small companies can validate their new technologies and services. More information at https://www.euwireless.eu/.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
