User level performance analysis of multi-hop in-band backhaul for 5G

The volume of global mobile data traffic is expected to increase exponentially in the coming years [1]. To enable this, disruptive solutions [2] are required for 5th generation mobile networks (5G). Network capacity can be increased by enhancing spectral efficiency, cell density, as well as by procuring new spectrum. Considering the expected demand, all three will be essential. Millimeter wavelength (millimeter-wave) (30 GHz and above) spectrum was always known to be ample and underutilized, but was previously not considered favorable for mobile communications due to its radio propagation properties. This was until propagation measurements [3] showed that despite the very little diffraction, millimeter-wave bands can be utilized for mobile access networks. While millimeter-waves [4] offer more bandwidth than all of the spectrum below 6 GHz combined, they require a substantially different approach. A particular issue is the maximum length of a millimeter-wave mobile access link; cell radii are estimated to be at most only hundreds of meters due to the unfavorable link budget. While this range would be prohibitively short in present day systems, many urban 5G cells will have to be very small regardless of whether millimeter-waves are used, in order to meet network capacity demands. Millimeter-waves will enable very dense cell layouts for extremely high network capacities. 5G could be a heterogeneous system that incorporates a higher capacity, higher frequency underlying small cell layer in urban areas, and complements them with an overlaying lower frequency macro cell layer which also covers rural areas.

Due to the very large number of 5G access points (AP), which can also be referred to as base stations, that have to be deployed in a given urban area, ease of deployment will be critical. It would be convenient to place APs on existing infrastructure, such as lamp posts, traffic signs, and the sides of buildings. The current and future wireline infrastructures will naturally be utilized in any available and feasible location, but bringing a fiber-optic cable or copper wire to every AP would be difficult, since it may require additional permissions or even expensive trenching, depending on the site. Conventional wireless point-to-point links would be far easier to deploy, but have their own challenges. Their antennas have to be precisely aligned, which increases deployment complexity. If they are accidentally slightly nudged, technicians have to realign them. They cannot be mounted on lamp posts or traffic signs that excessively sway and twist in the wind. A solution to all of these problems is the use of phased array antennas, which can perform beamforming to electronically steer their beams in the desired direction. This makes their deployment simpler than if conventional aperture antennas are used, as they only have to be roughly aligned. Furthermore, they tolerate mast swaying.

The cost of an AP and the installation could be decreased further if the backhaul links would use the same radio frequency (RF) devices and antennas as the mobile access links. This is possible because mobile access and backhaul links may use similar highly directional beamforming antennas [5] [6]. Such an AP would have the benefit of being able to establish a backhaul link in any direction in which it provides mobile coverage; this frees us from the task of aligning the AP antenna towards the direction of a backhaul link. If multi-hop routing on these wireless backhaul links is allowed, then only a fraction of the APs have to be connected to fiber-optic backhaul. This would enable a dense cellular network with minimal deployment cost. Furthermore, the mobile access and backhaul interfaces would use the same millimeter-wave frequencies; in other words, they would be in-band [7‐9]. Therefore, mobile access and backhaul have to share resources and can cause each other interference, but it is not necessary to reserve an additional frequency band for backhaul only [10].

The contributions of this paper are extensive simulations of user level performance of in-band networks. The novelty is that instead of considering a constant and equal traffic load at every AP, we consider multi-hop backhaul in combination with a TCP-based traffic model. Since in-band wireless backhaul implies interference issues, the interference sensitivity of such a network is quantized. The benefits of combining in-band and dedicated backhaul links is also shown. Furthermore, the possibility of the sudden loss of a link is also considered, in this case packets are rerouted on the mesh, and the corresponding system performance drop is analyzed.

Our results show that in millimeter-wave systems, since interference is mitigated by the use of directional antennas, the effect of interference on user level performance is far less than in current mobile networks, which use omnidirectional and sector antennas. When employing antennas with very high directivity, the system performance that users experience approaches the performance of a power limited network. We demonstrate that using in-band backhaul in combination with separate links dedicated for backhaul may even offer more performance gain than additional, high cost fiber-optic backhaul links. Our network also proves to be very resilient against occasional link outages.

The next section elaborates on the characteristics of millimeter-wave multi-hop networks. The third section provides details of the simulations. The following three sections describe the results of the investigations of interference, dedicated backhaul links, and occasional link outages, respectively. Finally, conclusions are drawn which underline the feasibility of such networks.

It is true that millimeter-waves undergo greater free space attenuation in the first meter of propagation than the much lower cellular frequencies that are currently in use, but only if isotropic or dipole antennas are considered. This is because the effective aperture area of an isotropic or dipole antenna is proportionally smaller at higher frequencies; for fixed sized aperture antennas free space attenuation is in fact less at higher frequencies due to the higher antenna gain. From this, it can be seen that high gain directional antennas are necessary if the link budget is to be compensated. Beamforming antennas [6, 11, 12], which are phase controlled antenna arrays, can direct a narrow beam towards the mobile device despite its movement. Due to the very small effective antenna sizes, the physical size of array antennas is small at millimeter wavelengths. Therefore, they can even be integrated into handheld mobile devices [13]. The maximum number of antenna elements will also be limited by cost and power consumption issues. According to [14], measurements have shown that the distance dependency of the millimeter-wave path loss is not inordinately different than that which is described in currently existing lower frequency channel models. Due to these factors, the maximum radius of a millimeter-wave cell will unfortunately be shorter than in fourth generation mobile networks; [15] estimates it will be 220 m, and [16] expects it to be 200 m, in urban environments. Rain attenuation is an issue at these frequencies, but it is fortunately less significant at short distances [17]. Millimeter-wave links are far more resistant to fog than free space optics. Providing indoor coverage with outdoor APs will be challenging if at all possible since millimeter-waves suffer from much higher penetration loss than lower frequencies [18‐22]. Indoor coverage can be provided with indoor APs [23‐27], but an AP can provide coverage only for one floor since multi-floor propagation is impossible based on the measurements of [28].

Due to the very little diffraction at millimeter wavelengths, there will be a high probability of a mobile device experiencing outage [14], especially at the edge of the cell. While it cannot always be avoided that the path to the closest AP may be obstructed, it may also be that there is an unobstructed propagation path to a neighboring AP. Therefore, millimeter-wave networks should be designed ensuring that cells overlap somewhat. While in fourth generation mobile networks this caused interference issues, directional antennas mitigate this problem. This is yet another factor that makes dense deployments more favorable.

While the dense deployment of small cells in urban environments offers high network capacity and good coverage, it magnifies the issue of deployment cost per cell. Each millimeter-wave access point should be a low cost device, easy to deploy, and have low cost backhaul access. Wireless backhaul [29] is far easier to deploy than cables, but if the access points are not placed above the rooftop level, then line-of-sight (LOS) backhaul links can be established only within visual range, which implies short distances. Non-line-of-sight (NLOS) backhaul links would have limited range due to the higher path losses. Since the inter-site distance is short, it would be convenient to allow the APs to connect to each other [30]; they would automatically discover and establish the backhaul links. If multiple hops [29, 31] on wireless AP to AP links are possible, only a fraction of the APs have to be connected to the core network via a fiber-optic cable, the rest of the APs would act as relay [32] nodes, thereby greatly reducing deployment cost. A multi-hop backhaul network can be used both outdoors and indoors. This solution would differ from repeater nodes by the decoding and error correction before forwarding, thereby signal quality does not degrade with every hop. A drawback of multi-hop wireless backhaul is that each additional hop adds latency; another is that the wireless backhaul also needs radio resources. Therefore, it essentially offers a tradeoff of radio resources and latency for low cost coverage expansion. This is a good match for millimeter-waves, which offer huge bandwidths, but suffer from limited cell size. Latency remains an issue, which imposes a limit on the maximum number of hops.

Placing millimeter-wave access points at lower heights may make deployment easier, but it also increases the chance that a moving object blocks a wireless backhaul link [18, 33]. This is likely to cause complete link failure, since millimeter-waves do not diffract around or penetrate obstacles easily, and their first Fresnel zones are narrow. To counter this, APs can be organized into a partial mesh topology with redundant wireless backhaul links, enabling traffic to be diverted to the remaining links in case of a blockage. This topology also enables efficient AP to AP communication within the mesh for fast handovers within the mesh; handovers are expected to be frequent in millimeter-wave small cells [34].

The channel quality of mobile access links greatly fluctuates due to small scale fading [35] and severe shadow fading [36‐38]. In contrast, since the APs are stationary, the channel quality of wireless backhaul links is steadier. It may change though due to the movement of other objects or people which cause blockages and dynamic shadowing, mast movement, and rain fading. Millimeter-wave wireless backhaul links have similar channel properties as microwave point-to-point links with aperture antennas. The difference is that for 5G it would be beneficial to be able to place them on existing structures such as unstable poles. The swaying of lamp posts in windy weather limits the directionality of aperture antennas placed there. Therefore, if high gains are necessary, APs with beamforming antennas can be deployed much more freely. Furthermore, they do not need to be precisely aligned.

As an alternative to beamforming antennas, beam steering lens antennas could also be used [39‐42]. These antennas would also have an array of antenna elements, but only one of them is selected for transmission/reception depending on the desired direction. The millimeter-waves are focused by a lens. They only require one RF chain and no phase shifters, thus costing less than beamforming antennas. While they can easily achieve high gain with a larger lens, they are limited in the angle range in which they can steer their beam; therefore, more of them would be needed to cover all directions. They are well suited for backhaul connections, which do not require very wide angle ranges. However, [43] describes a lens-array antenna concept and envisions the combination of a lens and a phased array antenna for both high gain and wide beam sweeping angles.

If the same RF devices and antennas are used by an AP for access and backhaul, then the backhaul links are expected to have a longer range than the access links. Less antenna elements can be integrated into a mobile device than into an AP, which implies less antenna gain. Mobile devices may not be able to transmit with as much power as APs due to battery constraints or human radiation exposure limits [44‐46]. Therefore, it is likely that the inter-site distances in millimeter-wave mesh networks will be limited by the maximum cell sizes and not range of the backhaul links. Furthermore, backhaul links may operate at higher signal to noise ratios (SNR) than access links, and thereby higher data rates.

Spatial multiplexing can be used in combination with beamforming for the necessary range and higher data rates [47]. However, this implies further multiplying the number of antenna elements. Another technique is to employ dual-polarized antenna arrays, thereby enabling the use of two orthogonal frequencies and two data streams [47]. This is possible if the cross-polarization discrimination is high, as measured for millimeter-waves in [28, 48].

Obvious cost benefits are offered by multiplexing wireless backhaul and mobile access on the same frequency band, also known as in-band backhaul. Its benefit is two-fold: it offers both hardware and frequency reuse. As high gain beamforming antennas are well suited for both millimeter-wave access and backhaul, it becomes possible to eliminate separate RF devices and antennas for backhaul. For the purposes of mobile access, these antennas are required to be able to steer their beam in a wide angle, in order to enable that only a few sectors could together provide omnidirectional coverage. From this, it also follows that the backhaul links can be established at any angle where mobile coverage is provided. Therefore, if such a sufficient number of APs would be placed in an area, then the APs could establish a mesh of backhaul links by themselves, even without precise backhaul mesh network planning in advance. This further simplifies deployment. It naturally requires the implementation of the necessary neighbor discovery protocols.

The obvious drawback of frequency reuse in the form of in-band backhaul is that resources have to be divided among access and backhaul links. A possible approach is to divide these resources according to a time division multiplexing (TDM) scheme [7, 8], which has the advantage that it does not require the AP antennas to be able to transmit in multiple directions simultaneously. A balance in the division of radio resources has to be found not only between downlink (DL) and uplink (UL), but also among backhaul and access. It is straightforward to statically allocate each a certain share of the radio resources in advance, which is in the case of TDM a set of time slots, but this will not necessarily be optimal. On the other hand, a dynamic resource allocation [9] has to adapt to quickly changing traffic conditions. As 5G networks are expected to offer very high data rates, download/upload times are expected to be very short for most use cases. This may imply that the traffic load on the wireless mesh will fluctuate rapidly, thus presenting challenges to dynamically adapting resource allocations to the traffic conditions.

Another important issue with in-band backhaul is that backhaul and access links will cause each other interference. Though high directivity beamforming antennas are necessary to extend the short link ranges, they have the additional benefit of decreasing the interference in the network. The more antenna elements the beamforming antennas are comprised of, the lower the general level of interference will be. However, the number of antenna elements per device will be limited by cost, power consumption, size and other issues. Atmospheric attenuation, which is significant at millimeter wavelengths, also can mitigate interference, as interferers from other cells are more likely to be further away and thus more affected. This article describes simulation studies on interference in in-band networks in the fourth section.

Networks characterized by very short inter-site distances are referred to as ultra-dense networks [49]; millimeter-wave networks will have to be ultra-dense due to the short mobile link ranges. A side effect of having smaller cells is that since there will be less users per cell, the variance of the traffic load of individual cells will be higher, as the law of large numbers is less applicable. In ultra-dense networks, it should be expected that while some cells may experience transient congestions, other cells may not be fully loaded. To ensure that 5G systems can deliver the promise of short transfer times and high quality of service, such an ultra-dense network should be designed in such a way that continuous cell congestion should be limited. Therefore, if in a multi-hop network a backhaul link carries the traffic of multiple cells, then for only a small fraction of the time will its traffic load be the sum of the capacity of all the cells it serves. In exceptional cases where very high demand is expected, such as in a stadium, fiber-optic connectivity should be considered for a higher number of APs.

In contrast, short, transient congestions in a network serving transmission control protocol (TCP) traffic is difficult to avoid due to the characteristic varying data rate. It is easier to guarantee low latency for delay sensitive applications if differentiated service queueing is used on the access and backhaul links. In a scenario where delay sensitive traffic is scheduled with strictly higher priority than best effort traffic, and the data rate of the delay sensitive traffic is far below the link capacities, the two traffic types only slightly deteriorate each other’s quality of service.

To investigate the performance of a multi-hop mesh network, a packet-based simulator was implemented on an ns-3 [50] simulator platform. Instead of investigating coverage or capacity, the goal of the simulations was to study the ways in which an in-band multi-hop backhaul mesh affects the user level performance of data traffic, assuming a certain set of hardware limitations, in the case of a more realistic traffic model. The traffic model we have chosen is TCP based, which is currently a very common internet traffic type.

The backhaul links and mobile access links are assumed to be in-band, and the AP-UE links use the same radio resources, RF circuits and antennas. Additionally, both have the same radio interface and time slot structure. The links are simulated with 1 GHz bandwidth and a transmission time interval (TTI) of 100 μs [51, 52]. From each 100 μs time slot, 20 μs is reserved for the control plane, and the rest for the user plane. The links are simulated to be time division duplex (TDD). In the simulations a topology with 20 APs is used. The APs are placed in an 800 × 800 m area, as shown in Fig. 1. The APs have four independent sectors, each covering a 90° angle. The backhaul links that comprise the multi-hop wireless mesh network are also shown in Fig. 1. It is based on an urban environment where buildings prohibit links between many AP pairs. A high capacity fiber-optic link connects one of the APs of the wireless mesh to a remote server; this link is simulated with different constant delays. The case with a 0.5 ms delay represents a scenario where the server is geographically close to the mesh, for example, in the same city; the case with a 5 ms delay represents a scenario where the server is, such as in the same country; and a 50 ms case represents a scenario where, for instance, the server is on a different continent. The assumption is that the network is ultra-dense, with very many small cells that cover a small area; therefore, the number of users will be comparable to the number of APs. In each simulation run, 20 user equipment (UEs) are deployed randomly in the area, and they are connected to the closest AP. The AP is considered to be in range if it is within 200 m [16] of the UE, if there happens to be no AP within range then the position of the UE is re-randomized. The quality of the UE’s channel depends on the distance between the UE and AP. Adaptive modulation and coding is used, with the highest data rate modulation and coding combination allowed being 16 QAM (quadrature amplitude modulation) [53] with a 5/6 error coding rate. It is assumed that at 200 m, the signal to noise ratio (SNR) of the channel is just enough to allow the use of the minimum data rate modulation and coding, which is −2 dB for a quaternary phase shift keying (QPSK) with a 1/6 code rate. The SNR of users closer to their serving AP than the maximum distance is calculated with a path loss exponent of 3.728, the 73 GHz, non-line-of-sight (NLOS), single beam value from [16]. If a UE happens to be very close to an AP, then it is also considered that the transmission power is capped to a value that corresponds to an SNR of 13 dB, which is sufficient for the maximum data rate modulation and coding. In contrast, the transmission power of backhaul links is always capped at a value that corresponds to an SNR of 13 dB, since the APs are assumed to have more maximum transmission power than the UEs, and the backhaul links have better channel quality than the access links. A physical layer automatic retransmission request (ARQ) mechanism has been implemented that retransmits all erroneously received physical layer frames, for every hop individually. This guarantees a very low drop rate on the wireless links, but does introduce some packet delay jitter.

The simulator considers a fixed TDD allocation in the user plane for the time slots as shown in Fig. 2. For AP sectors that serve only mobile users, the direction of data transfer alternates between DL and UL; thus they serve each direction in half of the time slots. Since the backhaul is in-band, AP sectors that serve both a backhaul link and mobile users have to also alternate between serving the backhaul link and the mobile users. An AP sector that serves one backhaul link, thus devotes a quarter of its time slots to one direction of the backhaul link; if it serves two backhaul links then one-sixth. While this fixed allocation is not optimal from a capacity point of view, it offers low latency and does not require traffic estimations and signaling. The allocation shown in Fig. 2 aims to minimize self-interference of the APs, where a transmitting sector interferes with a receiving sector of the same AP; this can also be mitigated with interference cancellation [54].

In the initial phases of the simulations, several TCP flavors were experimented with; both with and without selective acknowledgements (SACK) [55]. An early conclusion was that SACK is necessary for efficient network performance. The TCP flavor chosen for all of the results presented in this paper is TCP CUBIC [56], which is both a common flavor and performed well, with SACK enabled. At every node there was an IP buffer that was set to use random early detection (RED) [57, 58].

In the simulations, every UE downloaded one file with file transfer protocol (FTP). The downloads were initiated at a random point of time individually selected from the beginning of the simulation till 3 s later. Simulations were run where the size of the downloaded files was set to 10 or 100 MB. All simulations were run ten times and all values shown are averages.

One objective of the simulations was to investigate the effect of interference on user level traffic in an in-band multi-hop backhaul network. The interference model implemented in the simulator is two dimensional and takes into consideration amongst other things the topology, the radiation patterns of the antennas, the in-band nature of the network, and the traffic load on the links in every time slot. The antennas of the APs and UEs were considered to be phase controlled arrays. The radiation pattern of the antenna elements themselves was proportional to the cos2 of the angle of radiation, it was also assumed that the antenna elements do not radiate backwards. The number of elements in the antenna array was simulated with different values: 8 × 8 at the APs and 4 × 4 at the UEs, or 4 × 4 at the APs and 2 × 2 at the UEs, or 2 × 2 at the APs and single elements at the UEs. This corresponds to antenna gains ranging from 7.8 to 21 dB. An interference free theoretical case was included for reference. The antenna element spacing was half a wavelength. Every antenna element transmitted with equal amplitude. While the beamforming antennas always aligned their main beam precisely towards the antenna on the opposite end of the link, they did not consider the directions of interferers.

The model also takes into account that the in-band wireless backhaul network operates in TDD mode; every antenna has to switch among serving backhaul links or mobile access links, in addition to switching between transmitting and receiving. Therefore, depending on the time slot in question, it will either generate or suffer from interference, and its beam may change its orientation between time slots. An essential feature of this simulator is that it also considers the number and size of the packets transmitted in every time slot on every link. When a link is not used at its full capacity, the interference it generates is proportionally less, due to the lower transmission energy requirement.

Based on the received signal quality, in this case the interference level of previous receptions, a link adaptation algorithm estimates the signal to interference and noise ratio (SINR) of the next reception. Based on this, it always selects the appropriate modulation (16 QAM or QPSK) and error coding rate. The interference is estimated to be the maximum interference level observed for that link in the last 36 time slots.

Simulation results of file download times are shown in Figs. 3 and 4, while download goodputs are shown in Figs. 5 and 6. Upon examining the results, the first conclusion that can be drawn is that interference appears to be less critical an issue than in previous generation cellular networks. This is mainly because in this scenario, all the antennas are directional beamforming antennas instead of omni or sector antennas, which greatly decrease the interference in the system. By selecting antenna arrays with more elements, the performance of the system approaches the performance of the interference free reference case. The average interference level of the system is shown in Fig. 7. The interference observed on mobile links is higher than for backhaul links, because the UEs were assumed to have antennas with less directivity than AP antennas. It has to be noted that these results were achieved with a sophisticated link adaptation algorithm; without one or with a less efficient algorithm, the performance of the system would be much worse.

The core network is connected to some APs by fiber-optic links, these APs are the wireless mesh network gateways. If there are only a few such wireless mesh network gateways, then the bottleneck links will be the backhaul links that connect these gateways to other APs. The simulations revealed that interference issues are critical at the bottleneck links. The total throughput of the network is limited by the capacity of these bottleneck links, as they are the links that may become congested. When the link adaptation algorithm decreases the data rate of a congested link due to interference, or a retransmission is necessary, then the total network throughput is decreased. On the other hand, most of the links on the mesh network will never operate at their maximum capacity, therefore, a decrease in their data rate or ARQ retransmissions have little effect on total network capacity. However, if interference prevents the successful reception of a frame even after the maximum number of ARQ retransmissions (simulated as 3), then the resulting packet losses may decrease the performance of the system. This is one of the effects seen in Figs. 3 and 5; in these cases, the downloaded files are small (10 MB), due to which, the wireless mesh network does not suffer much from congestion. Instead, some TCP connections may remain in a slow start phase until the download is finished, this is the underlying reason for the base delay to the remote server having such a significant effect. This is in contrast to the cases shown in Figs. 4 and 6, where the larger downloads (100 MB) congest the mesh network and only the bottleneck links affect the performance of the network. The detrimental effect of this congestion can be seen in the increased number of timeouts, shown in Fig. 8. In addition, the area further away from the congested links tends to have underutilized links, which results in less interference locally as the links have to transmit less.

Since bottleneck links are crucial for performance of the wireless mesh network, a series of simulations was carried out to measure the effect it would have if the bottleneck links would be implemented with higher capacity wireless links. This capacity increase is considered to be a replacement of the in-band backhaul links with links that have dedicated RF devices and antennas. Therefore, they do not need to split their resources among backhaul and access links. If more in-band links would be connected to the wireless mesh network gateways, then due to the in-band concept the same radio resources would have to be split among more links, which would not offer improvement. The dedicated backhaul links can use low cost, high gain lens antennas. Furthermore, an option was investigated in which the wireless mesh was connected to the core network with another fiber-optic link in addition to the original connection.

In the simulations, the dedicated backhaul links were assumed to have two streams, one horizontally and one vertically polarized stream. It was considered that this doubled their link capacity. They were always simulated with very high gain antennas. Otherwise, they had the same parameters and interface as the in-band links; they operated in TDD mode, but only had to keep switching between the two directions of a single backhaul link. A simulated topology with 4 dedicated links shown in Fig. 9 and another with 12 dedicated links in Fig. 10. For comparison, a topology without dedicated wireless links, but with two fiber-optic links to the core network was also investigated, see Fig. 11. In addition, two topologies with two fiber-optic connections to the core network was defined, one with 7, the other with 18 dedicated backhaul links, as shown in Figs. 12 and 13. Note that the links closest to the wireless mesh network gateways were chosen to be higher capacity dedicated backhaul links.

Comparisons of the user level performance of the different topologies can be seen in Figs. 14 and 15. The size of the downloaded files was 100 MB, the APs had 8 × 8 element antennas arrays, and the UEs had 4 × 4 element antenna arrays. The results show that as expected, replacing the bottleneck in-band backhaul links to higher capacity dedicated links has a significant impact on user level performance. Similarly, an additional fiber-optic connection also has a performance benefit. However, it is interesting to note that upgrading four wireless links has a much larger impact than adding an extra fiber link. This is because the throughput of the fiber links is capped by the lower capacity wireless links connected to it. By upgrading a few wireless links, the bottlenecks of the network are shifted deeper into the mesh where there are much more wireless links to carry the traffic load. Additionally, the performance of the different UEs becomes more even with higher capacity topologies, as shown in Fig. 16, where the fairness among users is measured with Jain’s fairness index [59].

Adding an additional fiber link decreases the maximum number of wireless hops from four to three, similarly the average number of wireless hops decreases from 2.05 to 1.6. On the one hand, the number is hops in the wireless mesh network is an important factor, as every additional hop increases latency. If the air interface latency can be at most a few milliseconds for delay critical traffic, then only a few hops may be allowed, and delay critical traffic has to be scheduled with high priority. On the other hand, the simulation results prove that the number of hops does not have any visible effect on user level performance. The reason for this is that the queuing delays of the best effort packets can even be hundreds of milliseconds, which is several orders of magnitude more than the TTI which is 0.1 ms.

The overall conclusion is that the most cost effective solution would be the combined use of both in-band and dedicated links. The network should be proportioned by examining the routing tree topology within the mesh, and selecting the links near the root of this tree topology to upgrade them to higher capacities. This is to be carried out where few parallel links carry the traffic load, in other words, close to the wireless mesh network gateways, which have a fiber connection to the core network.

Simulations were also carried out to investigate the effect of the blockage of a wireless backhaul link on user level performance. In the simulations, if no link is blocked, then packets are routed on pre-calculated primary paths which have the minimum number of hops. Additionally, maximally disjoint secondary paths are also calculated in advance. If a primary path is blocked, then according to a path reselection algorithm [60], the corresponding secondary path is used. However, when the path is reselected, there may be some packets in transit on the wireless mesh. In the simulations, two alternatives are compared; in one, these packets are simply dropped when they arrive at the blocked link. In the other, the in-transit packets are rerouted according to a fast local reroute algorithm [60]; they are rerouted according to the secondary path of the AP with the failed link.

If a link is continuously unavailable, then logically, the network will perform as if there was just one link less in the mesh. Therefore, the studies focused on the issues caused by the appearance and loss of a link, in other words, the confusion in the routing on the mesh. A worst case scenario was defined, where a link is repeatedly available for one second and then experiences an outage for the next second. While this case is not necessarily realistic, as it represents more frequent outages than expected, it is useful as a reference worst case scenario. The topology with the link that was simulated to experience an occasional outage is shown in Fig. 17. This link was selected to be a link with heavy traffic, as the outage of a less utilized link would have a less noticeable effect.

The simulation cases were defined with only in-band wireless links, the AP antennas were arrays of 8 × 8 elements, and the UE antennas had 4 × 4 elements. As reference cases, simulations were also run where all the links were reliable, and where the link in question was continuously experiencing an outage, meaning the mesh simply has one link less. The simulated goodputs of only those connections that have to be rerouted on the mesh in case of an outage are shown in Figs. 18 and 19. The results show that the path reselection and fast local reroute algorithms work as planned, and the remaining wireless mesh links handle the rerouted traffic well; the user level performance drop is close to the ratio of lost link capacity. These results prove that the occasional loss and appearance of a link does not cause serious problems, except for the decrease of the network capacity; assuming that every AP remains connected. This justifies connecting the APs into a partial mesh topology in order to ensure robustness. However, if multiple links fail simultaneously, some APs may be disconnected. The only ways to avoid this is to increase the connectivity of the partial mesh topology or deploy more reliable links. However, it is less likely that multiple links are blocked at the same time, unless the link failures are due to heavy precipitation [17]. These results also imply that investigating a worst case scenario may be justified as the performance drop even in this scenario is not critical, the user level performance drop in a scenario with less frequent outages would be even closer to the ratio of lost capacity.

It also can be seen that due to the very frequent outages, in the cases where the packets that arrived at the blocked link were simply dropped, the user level performance may be even worse than if the link would be continuously blocked. This can be seen even more clearly in Figs. 20 and 21 where the average number of timeouts per file download is shown. The dropping of packets will cause the TCP connections to time out, and this decreases the overall system performance. However, if the packets are rerouted with the fast local reroute algorithm instead of being dropped, then the system in some cases performs better than if the link were always unavailable.

The objective of our performed simulations was not to investigate coverage or capacity, as these have already been investigated in literature. While there have been many simulation studies evaluating millimeter-wave networks, the focus of most of them is not on the traffic model. A detailed channel model is presented by Akdeniz et al. [35] and according to it the capacity of a millimeter-wave cellular network is calculated assuming a full buffer traffic model. Bai et al. [61] estimates user and cell throughput rates without specifying the traffic model. Bai et al. [62] give a more detailed description of their model in, but a traffic model is not included. Akdeniz et al. [63] evaluate millimeter-wave systems also with a full buffer traffic model. In contrast to the previously mentioned publications, Hui and Axnas [9] and Baldemair et al. [49] consider self-backhauled multi-hop networks. Routing and resource allocation for a wireless mesh is analyzed by Hui and Axnas [9], by employing a simple traffic model where a certain number of UEs are randomly selected to transmit and receive data. Baldemair et al. [49] define the traffic load simply as the number of UEs per AP. Garcia-Rois et al. [64] investigate scheduling on a dynamic duplex multi-hop mesh network, and also consider interference; however, the authors consider packets to be generated according to a random stochastic process.

The distinguishing novelty of our paper is that multi-hop in-band millimeter-wave mesh networks are simulated and evaluated in conjunction with a realistic traffic model. Additionally, link outages and rerouting on the mesh is considered.

A different ns-3 simulator module [65] for millimeter wave access has been developed at the same time as the work presented in this paper. However, Mezzavilla et al. [65] state that relay devices and TCP performance is a topic for their future work, and therefore the module was not applicable for this study.

In this paper, we have described and studied a multi-hop, millimeter-wave wireless backhaul network concept for 5G. A packet-based simulator was implemented on the NS-3 simulator platform with which simulations were carried out to investigate user level performance in the case of a realistic traffic model. The distinguishing feature of our work with the simulator is the in-band interference calculation, which takes into account the momentary number of packets transported on every in-band link. Our paper offers evidence that supports the feasibility of the multi-hop in-band backhaul concept for 5G, where not only the spectrum, but also the RF devices are shared among access and backhaul. We can confirm that the offered download rates exceed the data rates of previous generation networks.

The simulations quantized the detrimental effect of interference on user level performance. The remarkable result is that even in the case of in-band backhaul, the effect of interference is not severe, provided that a sophisticated link adaptation is used. Interference issues can be minimized with higher directivity antennas, such as antenna arrays with more elements. Interference will often be below the noise level. Results have also shown that the often overlooked factor in interference calculations, the momentary traffic on individual links, cannot be ignored. Only a few bottleneck links can be fully loaded, and the interference elsewhere in the network has less impact on user level performance. Therefore, it is important to minimize the bottlenecks in the multi-hop network, while ensuring that the bottleneck links do not excessively interfere with each other. In contrast, this effort can be saved for links that carry less traffic. While the complexity of any simulation model is limited, unless future networks will have significantly different approaches and parameters from those envisioned in this paper, it can be stated that these results confirm that interference will be less of an issue in millimeter-wave networks than in previous generation systems.

While in-band backhaul links are low cost and are very easy to deploy, dedicated RF devices for backhaul links offer higher capacities. The results presented in this paper show the benefits of using a combination of the two techniques. System capacity and performance can be increased by using dedicated devices at the bottleneck links; these are usually the links close to the fiber-optic links to the core network. On the other hand, device and deployment cost can be greatly decreased by using in-band devices for other links. A gradual tapering of capacity is recommended from the fiber-optic link to the edges of the mesh.

Due to the higher probabilities of a link being obstructed, simulations also covered cases in which a critical link experiences occasional outages. Results show that even in a worst case scenario, the effect on TCP traffic is tolerable, provided that efficient rerouting algorithms are used. However, this is only true as long as all APs have a remaining path to the core network, which is the reason for considering partial mesh topologies.

In the future we intend to consider different traffic types, such as delay and loss sensitive machine-type traffic. Furthermore, we are planning to investigate the routing of packets in the wireless mesh when there are frequent handovers.