Performance analysis for optimum transmission and comparison with maximal ratio transmission for MIMO systems with cochannel interference

The most adverse effect mobile radio systems suffer from is mainly multipath fading and cochannel interference (CCI), which ultimately limit the quality of service offered to the users. Space diversity combining with a single antenna at the transmitter and multiple antennas at the receiver (SIMO) provides an attractive means to combat multipath fading of the desired signal and reduces the relative power of cochannel interfering signals. A practical and simple diversity combining technique is maximal ratio combining (MRC), which is only optimal in the presence of spatially white Gaussian noise. MRC mitigates fading and maximizes signal-to-noise (SNR), but ignores CCI; however, it provides CCI with uncoherent addition and, therefore, results in an effective CCI reduction. Optimum combining (OC), in which the combiner weights need to be adjusted to maximize the output signal-to-interference-plus-noise ratio (SINR), can resolve both problems of multipath fading of the desired signal and the presence of CCI, thus increasing the performance of mobile radio systems.

The performance of OC was studied for both nonfading [1] and fading [2‐12] communication systems in the presence of a single or multiple cochannel interferers. Performance analysis of OC and comparison with MRC were studied in [6]. The emphasis is on obtaining closed-form expressions. Whereas publications in the area dealt with SIMO, applications in more recent years have become increasingly sophisticated, thereby relying on the more general multiple-input/multiple-output (MIMO) antenna systems which promise significant increases in system performance and capacity. With no CCI, the performance of MIMO systems based on maximum ratio transmission (MRT) in a Rayleigh fading channel was studied in [13‐15]. In the presence of CCI, the outage performances based on MRT [16] and optimum transmission (OT) [17, 18] were studied. In general, the analyses of the above SIMO and MIMO systems adopt the following assumptions: (1) The number of interferers exceeds the number of antenna elements, and the antenna array is unable to cancel every interfering signal [5, 6, 18]. At this point, the interference is approximated by Gaussian noise. (2) The phase of each interferer relative to the desired signal for each diversity branch is neglected, and thus phase tracking and symbol synchronization are not only perfect for the desired signal, but also for CCI [3‐8, 12‐17]. (3) Average powers of interferers are assumed to be equal, which is valid in the case that these interferers are approximately at the same distance from the receiver [3‐5, 18]. (4) The effect of thermal noise is neglected, which is reasonable for interference limited systems [5, 7, 8, 18]. Based on the above assumptions, the SINR distribution is derived and enables simpler and faster analytical computation of performance measures such as outage probability and average error probability.

Multiple interference meets the conditions of the central limit theorem; hence, it can be assumed Gaussian (nonfading case). The noise approximation model is simplistic, but was shown to be inaccurate for the case of a few dominant interferers. In some cases, it is pessimistic; in some others, it is optimistic; and in certain cases, it is even very close to the actual performance. For the accurate estimation of the performance degradation caused by interfering signals, their statistical and modulation characteristics have to be taken into account in the analysis. All of the early studies mentioned above did not consider Nyquist pulse shaping and the modulation characteristics of the CCI. The effects of cross-channel intersymbol interference (ISI) produced by CCI due to symbol timing offset were neglected. In [9‐11], the bit error rate (BER) of PSK operating in several different flat fading environments in the presence of CCI was analyzed using the precise CCI model, but no diversity schemes were considered in [9]. The performances of dual-branch equal gain combining (EGC) and dual-branch selection combining (SC) were investigated in [10, 11]. However, the performances of MIMO systems using MRT and OT schemes have not been studied to the best of our knowledge.

This article studies the average BER of quadrature amplitude modulation (QAM) with OT and provides the comparison with MRT using the precise CCI model when the desired signal and interferers are subject to nonfading and Rayleigh fading for Nyquist pulse shaping. QAM has widely been applied in future generation wireless systems (e.g., 3GPP LTE standard). We are dedicated to a precise analysis of CCI including the effects of ISI produced by the CCI and the effects of random symbol and carrier timing offsets. The focus of this study is on the analysis of the schemes rather than on the implementation aspects. The analyses are not limited to a single interferer case, but rather assume the presence of multiple independent interferers. With the multiple ISI-like CCI sources, the simulation is expected to be very tedious and time-exhausting in MIMO systems. Therefore, the error probability for each fading channel is estimated fast and accurately using Gauss quadrature rules (GQR) which can approximate the pdf of ISI-like CCI. We also derive new expressions that approximate the BER of the MRT-based MIMO system using Gaussian models and its accuracy is assessed. Simulation results show the use of precise CCI model and GQR offers significant improvement in the performance analysis and comparison for MIMO systems.

The rest of this article is organized as follows. The system models of MIMO based on MRT and OT schemes in the presence of CCI and noise are introduced in Section 2. The error probability evaluation using GQR in the presence of ISI-like CCI is discussed in Section 3. Simulation results and comparison are presented in Section 4. Conclusions are summarized in Section 5.

We consider a MIMO system equipped with T antenna elements at the transmitter and R antenna elements at the receiver as shown in Figure 1. It is assumed that there exists totally L cochannel interferers from the neighboring cells. The system is modeled, assuming that the desired signal and cochannel sources transmitting QAM signal over a flat fading channel. The transmitted QAM baseband signal from the desired signal can be expressed in the form

where w k t represents the transmit weight on the k th antenna (k = 1,..., T) and T _s is the symbol interval. Since the CCI transmit weights are not controlled by the desired receiver, the transmit weights of CCI can be neglected. Thus, the i th transmit CCI can be combined as

where c _i,n = a _i,n + jb _i,n is the sequence of complex data symbols. The data symbols a _i,n and b _i,n on the in-phase and quadrature paths define the signal constellation of the QAM signal with M points. In the constellation, we take a i , n , b i , n = 1 , 3 , … , M + 1. The transmitter filter gives a pulse g _t (t) having the square-root raised-cosine spectrum with a rolloff factor β. Nyquist pulse shaping with an excess bandwidth of β = 0.5 is a good compromise between spectrum efficiency and detectability [9]. The desired symbol sequence is indexed by i = 0, and CCI sources by i > 0 (i = 1,..., L for CCI).

The channel is spatially independent flat Rayleigh fading, which is a valid assumption when the antenna spacing is sufficiently large and the delay spread is small. Unlike [19‐21], the fading experience by CCI is independent of the fading experienced by the desired signal. The complex channel gain between the k th transmit antenna and the m th receive antenna for the desired signal can be represented by h D , k , m = λ k , m e j θ k m, where λ _k,m is the envelope with Rayleigh distribution having variance σ D 2 = E [ λ k , m 2 ] for all paths. The complex channel gain between the i th CCI source and the m th receive antenna can be represented by h I , i , m = λ i , m e J θ i , m with variance σ i 2 = E [ λ i , m 2 ]. Phases θ _k,m and θ _i,m have a uniform distribution in [0, 2π]. In a nonfading environment, the channel gains λ _k,m and λ _i,m are constants. With zero-mean information symbols, the average power of the i th cochannel interferer received by each antenna is derived as σ i 2 σ c 2 ∕ T s, where σ c 2 = E c i , n 2 represents the data symbol variance for all cochannel sources. For an M-QAM system, σ c 2 = 2 ( M - 1 ) ∕ 3. The input noise n _m (t) is a zero-mean AWGN with two-sided power spectral density of N₀ W/Hz. Thus, the noise power measured in the Nyquist band is N₀/T _s . Due to constant total transmitted power constraint, the average value of the SNR on each receive antenna is, therefore, defined by σ D 2 σ c 2 ∕ N 0. Equal average power is assumed for all the received interferers, and therefore, we set σ i 2 = σ I 2 for i = 1,..., L. The SIR per diversity branch can be denoted by S I R = σ D 2 ∕ L σ I 2.

At the receiver, we assume that the frequency and symbol synchronization are perfect for the desired signal. In a precise interference model, after matching and sampling at t = lT _s , the signal received at the m th antenna is given by

where the random variable τ _i is uniformly distributed in [0, T _s ] and it represents a possible symbol timing offset between the desired signal and the i th interferer; pulse response g(t) having the raised-cosine spectrum is the combined transmitter filter g _t (t) and receiver filter g _r (t) which have the same response; the filtered noise is v _m (t) = n _m (t) ⊗ g _r (t) and the power is calculated as σ v 2 = N 0 where ⊗ denotes the convolution operation. Since noise is wide-sense stationary (WSS) and the power is independent of sampling instance, we have E [ v m 2 ( l T s ) ] = σ v 2 = N 0. The signal from the m th receive branch is weighted by a complex weight w m r. The output of the combiner has the form

For convenience, the MIMO signal can be expressed in a matrix form. The channel gain for the desired user can be defined as a R × T matrix

and the L cochannel interferers can be written in a R × L matrix form as

The T × 1 weight vector at the transmitter and the R × 1 weight vector at the receiver are defined as w t = [ w 1 t , w 2 t , w 3 t , … , w T t ] T with ||w ^t ||² = 1 (i.e., average transmit power is restricted to be constant) and w r = [ w 1 r , w 2 r , w 3 r , … , w R r ] T, respectively, where (·) ^T is the transpose operator and || · ||² is the Euclidean norm. The output defined in Equation 4 can then be written in a matrix form as

where cg = [c₁g₁, c₂g₂ ,..., c _L g _L ] ^T , a L × 1 vector, represents ISI produced by all interferers with g _i = [g(NT _s - τ _i ), g((N - 1)T _s - τ _i ),..., g(-τ _i ),..., g(-(N - 1)T _s - τ _i ), g(-NT _s - τ _i )] ^T , which are the 2N + 1 truncated samples of the raised-cosine pulse due to the delay offset τ _i from the i th interferer, and c _i = [c _i (-NT _s ), c _i (-(N - 1)T _s ),..., c _i (0),..., c _i ((N - 1)T _s ), c _i (NT _s )], which is the symbol sequence of the i th interferer. The vector v = [v₁(lT _s ), v₂(lT _s ),..., v _R (lT _s )] ^T represents R discrete filtered noise sources at the receiver. The vectors w _t and w _r are determined using MRT and OT methods in this study.

Since CCI is not Gaussian distributed, maximizing SINR cannot guarantee the minimum error probability. The calculation of the exact error probability for MIMO systems in the presence of CCI will be discussed in this section. To complete this, we begin by the combined signal in Equation 7 as

where sampling time is at t = lT _s and g s = w r T H D w t which is equal to Ω_max is the largest eigenvalue of the matrix H D H H D for MRT and H D H R - 1 H D for OT. With defining g _i,n = g(nT _s + τ _i ), the combined ISI in the in-phase rail due to total CCI can be denoted by

where we define the sampled pulse response of the i th CCI source as

with h I , i , m = λ i , m e j θ i , m = λ i , m ( cos θ i , m + i sin θ i , m ) and w m r = w I , m r + j w Q , m r. The ISI corresponding to the quadrature channel is denoted by η. As sampling time is set at t = 0, with a slight change in indexing the signal, we denote above pulse responses in the in-phase and quadrature channels, respectively, as

The m th weighted discrete-time noise is expressed as ω m , l = w m r v m ( l T s ). The power spectra of the filtered noise v _m (t) is N₀G(f) and hence resulting in the output power (variance) σ ω m 2 = [ ( w I , m r ) 2 + ( w Q , m r ) 2 ] N 0, where G(f) has a raised-cosine spectral characteristic. Since the noise is uncorrelated between diversity paths, the variance of the combined output noise, w _I , is expressed as

We define ω, = ω _I,l + ω _Q,l where ω _I,l and ω _Q,l have equal power (variance), σ 2 = σ ω 2 ∕ 2. Since the distribution density functions of quantities ξ and η are symmetric to zero and are identical, it has been shown that the average symbol error probability P _M can be bounded tightly by [22, 23]

Because ξ is a random variable whose distribution is not known explicitly, the evaluation of E[g(ξ)] is performed by computing the conditional error probability of each of all possible sequences of CCI, and then averaging over all those sequences [22, 24]. For (18), g(·) is given by erfc (·).

This fast semi-analytical technique in (18) is computationally very efficient compared to the Monte-Carlo method. However, this approach is cumbersome and may be computationally infeasible if a large number of cross-channel ISI symbols (e.g., with high order of modulation) are included or/and more than one interferer are present, especially when dealing with low error rates. Thus, such a method becomes extremely time-consuming when we consider MIMO systems. Some techniques can be used for evaluation of numerical approximations to the average E[g(ξ)]. One efficient approach called the GQR approximation will be addressed for the numerical evaluation of (18), which depends on knowing the moments of, up to an order that depends on the accuracy required.

Using the Gaussian quadrature rule, the averaging operation in (18) can be approximated by

a linear combination of values of the function g(·), where f _ξ (x) denotes the probability function of the random variable ξ with range [a, b]. The weights (or coefficients) w _i , and the abscissas x _i , i = 1,2,..., N, can be calculated from the knowledge of the first 2N + 1 moments of ξ. We compute the average in (19) by means of the classic GQR's suggested in [22]. The precise BER results are obtained using a combination of analysis and simulation under fading conditions.

For the ISI ξ in (16), we can assume that there are N₁ terms in the first summation and N₂ terms in the second for each interferer. We assume N _s = L(N₁ + N₂). The random variable ξ is the sum of N _s ISI terms for the multiple CCI case. The ISI ξ can be rewritten as

where I _j represents a discrete random variable, a _i,n or b _i,n , whose moments are given and x _j is a sequence of known constants p _i,n or q _i,n . It is suggested that we reorder the sequence y _i 's so that max |y _i | ≥ max |y_i+1|, i.e., |x _i | ≥ |x_i+1|, 1 ≤ i ≤ N _s - 1. This reordering lets the moments of the dominant terms be computed first and rolloff error be minimized. A recursive algorithm which can be used to determine the moments of all order of ξ was discussed in [22].

We only exhibit the simulation results of 4-QAM with Nyquist pulse shaping with an excess bandwidth of the rolloff factor β = 0.5 which is a good compromise between spectrum efficiency and detectability. Average error rate due to fading can be evaluated by averaging the error rate over all possible varying channel parameters, including the timing offset. A single dominant CCI and six strongest interferers are considered individually. We make the assumption of equal-power interferers for the case of six interferers. Due to this assumption, the results are pessimistic with respect to the case of unequal-power. The average BER P _b = P _M /2 for 4-QAM. Because the objective of carrying out the simulations is to evaluate the performance, it is assumed that perfect knowledge of channel fading coefficients is available to both transmitting and receiving stations. We consider the MIMO systems with several different combinations of antennas. The average value of SIR is set to 10 dB for simulation. The performances of MIMO systems based on both MRT and OT schemes are investigated and compared, when the signal and interferers are subject to nonfading and Rayleigh fading. We only consider the MIMO system with the order up to three transmit antennas or three receive antennas. This is often the case in mobile radio systems. The quantity T + R is the total number of antennas used, and is a measure of the system cost. An increase in system cost results in improved error performance. Therefore, one of our major objectives is to determine the distribution of the number of antenna elements between the transmitter and the receiver for minimum average BER given a total number of transmitter and receiver antenna elements.

We first consider the performance of MRT, when the precise CCI and Gaussian noise-like CCI models are employed. In general, for a given average SNR, the transmit power in each of antennas is smaller for T > R, whereas the total combined noise power at the receiver is higher for T < R. Therefore, the effects of these two factors compensate for each other which makes the performance on BER is symmetric in T and R in the absence of CCI. For example, the BER for (T, R) = (3,1) or (3, 2) will be the same as that for (T, R) = (1,3) or (2, 3). In the presence of CCI, Figures 2 and 3 show plots of BER versus average SNR, when all channels are unfaded, but the random carrier phase and symbol timing offsets of CCI are included. It is observed that the results obtained using precise interference model are considerably better than that obtained by using the Gaussian model. Those curves appear different for L = 1, but they become close when L = 6. Based on the central limit theorem, by increasing the number of interference and number of receiver antennas, the Gaussian CCI model can approach to the precise CCI model (without fading). Unlike the Gaussian CCI case, the performance is not symmetric in T and R using the precise CCI model. We can see that the performance is slightly better for T > R in a high order MIMO system, for example (T, R) = (3,2). This is attributed to the fact that more interfering signals received by antennas can approach to Gaussian distributed CCI which may cause a higher degradation.

When T + R ≥ 5, the error probability becomes small and then all curves are very close in our simulation range for L = 1 and L = 6 either using the precise CCI model or the Gaussian CCI model. It is expected that those curves will appear different at lower BER. The irreducible error floor is caused by the residual CCI. Next, we intend to explore the effect of a fixed number of antenna elements (T + R = 4) between the transmitter and the receiver when the precise CCI model is used. In theory, neglecting the phase of the channel results in a lowest BER with (T, R) = (2, 2). Interestingly, Figure 4 shows that all curves are very close for SIR = 10 dB. The average BER with (T, R) = (2, 2) is the slightly worse, particularly for the case of L = 6, because the power of the received desired signal may be degraded by the variation phase of the channel. However, decreasing the value of SIR to 5 dB, Figure 5 shows that the performance with (T, R) = (2, 2) becomes the best. In other words, the receiver with (T, R) = (2, 2) has better ability to combat interference and can compensate for the reduced signal power when the interference becomes dominant. The performance with (T, R) = (3, 1) is better than that with (1, 3) because T > R. For the L = 6 case, all curves are very close since the combined interfering signals can approach Gaussian CCI, as discussed above.

When both the desired signal and CCI are subject to fading, the simulation results are exhibited in Figures 6 and 7. The average BER becomes very high due to the fading effect on the desired signal. The high average irreducible error floor is due to the fact that fading effects increase the chance of taking on a lower instantaneous SIR. The Gaussian model slightly underestimates the average error probability without diversity, similar to the result given in [9]. The curves of the Gaussian CCI and the precise CCI appear different with the increase of the transmitter and receiver antenna elements, since the fading effect of the desired signal is reduced and then results in a similar behavior to the nonfading case. In general, the Gaussian interference model predicts that the BER floor can be increased by three orders of magnitude in going from T + R = 3 to T + R = 5 MIMO systems. The Gaussian model always overestimates the performance for this case. It is noted that the performance with L = 6 becomes better than that with L = 1 for this fading CCI case. The possible explanation is that when the total interference power is equally distributed among six interferers, the probability that at least one of the interferers is strongly faded is greater in the case of multiple interferers, thus leading to a smaller error rate. Unlike the nonfading case, the performance is not symmetric in T and R when Gaussian CCI model is used due to the effect of fading. The BER is better for T < R, since more fading interferers received by antennas results in a small BER performance, particularly in the case of L = 1. However, when the precise CCI model is used, the BER with (T, R) = (3, 2) is slightly better than that with (T, R) = (2, 3), whereas the BER with (T, R) = (2, 4) is better than that with (T, R) = (4, 2) in our test because of the fading effect of the multiple interferers. Unlike the nonfading case, the BER with (T, R) = (2, 2) is the lowest given T + R = 4. This is due to the fact that the probability of low instantaneous SIR is high under fading conditions. The receiver with (T, R) = (2, 2) has better performance at high value of SIR, as discussed above. This result is similar to that discussed in [14], where no CCI was considered. Hence, | T - R | must be as smaller as possible for this fading case, assuming that | T + R | has to be kept fixed.

Next, we consider the OT scheme and compare its results with the MRT scheme in MIMO systems. The number of receiver antennas must be greater than two in order to cancel CCI. For the nonfading case, Figure 8 shows that OT cannot outperform MRT with R = 2 for the case of L = 1 due to significant noise enhancement under certain channel conditions (phase offset for each diversity branch) of CCI. When channels of the desired signal and CCI are very similar, cancellation of CCI might cause severe noise amplification. In out test, we find that OT with two receiver antennas is unable to show the superiority over MRT for higher value of SIR (e.g., SIR > 7 dB) for the case with (T, R) = (1, 2); however, interference cancellation can compensate for noise enhancement effect for low value of SIR. It is seen that the use of R = 3 avoids this worse CCI situation and then improve the raised curve of BER, as shown in Figure 9. In fact, the maximum SINR is unable to guarantee the minimum BER, if the interference is not Gaussian distributed. The joint antenna weights, derived for SINR maximization, are capable of minimizing the total power of interference and noise, while the power of CCI is reduced and the power of noise is enlarged. As a result, the BER becomes relatively high, since CCI causes much less impairment than the Gaussian noise-like CCI given the same power as discussed in Figure 2. On the contrary, the MRT scheme mitigates the effect of CCI well and achieves satisfied performance in this nonfading case.

When the desired signal and CCI are subject to fading, the probability of low instantaneous SIR is considerably increased, and then OT can demonstrate its superiority in cancelling CCI in the range of SNR. Figures 10 and 11 show that OT can improve the performance significantly for L = 1 at high SNR. Due to the noise enhancement effect, the performance of OT with two antennas is much worse than that with three antennas, given the a fixed number of antenna elements (e,g., T + R = 4) between the transmitter and receiver. Unlike the MRT case, the performance of OT with (T, R) = (2, 2) is worse than that with (T, R) = (1, 3) in the case of L = 1 due to noise enhancement discussed in the nonfading case. However, the performance with (T, R) = (2, 2) is still better in the case of L = 6, since CCI cannot be eliminated and has a similar behavior to the MRT case. Similarly, the case of (T, R) = (3, 2) is worse than the case (T = 2, R = 3). It is noted that OT has worse performance than MRT in the case of (T, R) = (3, 2) when SNR < 20 dB since the noise enhancement effect cannot compensate for the gain of CCI interference cancellation. The OT scheme has a similar behavior to the MRT scheme for L = 6, having an error floor because L > T + R. The results are similar to that presented in [18], which shows that the OT scheme with T = 5, R = 1 (or T = 4, R = 2) is always worse than the one with (T, R) = (1, 5) or (T, R) = (2, 4) for L = 6 in a Rayleigh-Rayleigh fading channel in absence of noise, assuming that L > T + R. Similar to the MRT case, it is preferable to distribute the number of antenna elements evenly between the transmitter and the receiver for an optimum performance using OT when L = 6 (e.g., T = 3, R = 3 and T = 2, R = 2). For the (T, R) = (3, 3) case, the fading effect is largely reduced and thus all curves are very close in our simulation range.

Usually, it is impossible to generalize the performance of the noise model with reference to the interference model, in all cases. Unlike the results of EGC presented in [10], we show that the Gaussian interference approach always overestimates the effect of interference with the MRT-based MIMO system in nonfading and fading cases. Also, it is not easy to generalize the performance of the OT scheme with reference to the MRT scheme in all MIMO cases, depending on the combination of the antennas at the transmitter and the receiver, number of interference and the statistical characteristics of the channel. In general, the optimum scheme of choosing weights that maximize the output SINR would provide little performance gain at the cost of increased complexity when the number of interferers is large. To implement the OT scheme, the knowledge of the desired user's channel as well as interfering channels is needed at the receiver and transmitter. In such a case, MRT is usually preferred because of its implementation simplicity. Another advantage is that MRT does not require the mobiles to have full knowledge of uplink channel to determine the transmit weights. Only the largest right singular vector is required, which can easily be sent through a feedback channel [12].

In this article, we have analyzed the performance of OT and MRT-based MIMO systems subject to CCI operating over nonfading and fading channels. The use of precise CCI model provides significant improvement in the performance analysis. To the author's knowledge, the precise analysis of OT and comparison with MRT with applications to MIMO systems were not investigated. The results of this study are expected to lead to a better understanding of the effects of interference, and then to optimize spectrum reuse and coverage in MIMO systems. The main important contributions are summarized as follows: