Symbol Error-Rate Analysis of OSTB Codes and Linear Precoder Design for MIMO Correlated Keyhole Channels

The benefits of using multiple transmit and receive antennas and space-time coding for wireless communication are well known [1]. In principle, the capacity of such multiple-input multiple-output (MIMO) channel can be much higher than a single-input, single-output (SISO) channel. The capacity of a MIMO channel is maximum when each transmit and receive antenna pair forms a statistically independent transmission path, a condition which is nearly met in a rich scattering environment. Space-time codes (STC) can be used to exploit the spatial diversity available in such a channel to realize high-rate data transmission [1]. However, in realistic situations, the channel capacity is degraded by spatial correlations at the transmitting and/or receiving antenna arrays, due to, for example, insufficient spacing of antenna elements in the arrays, the existence of few dominant scatters, or small AOA spreading [2]. Furthermore, recent studies, both theoretical and experimental, have shown that, in addition to antenna correlation, the MIMO channel capacity can also be decreased by so-called keyhole effects in wave propagation [3, 4]. In particular, a keyhole effect reduces the rank of the MIMO channel matrix to one, even in the absence of any spatial correlation in the channel [4]. In a keyhole MIMO channel, the signals from the transmit antenna array are constrained by the physical structure of the propagation environment to pass through a keyhole before reaching the receive antenna array (Figure 1(a)). Such keyhole effects have been observed in both indoor and outdoor propagation environments [5, 6].

In previous work, the capacity of MIMO keyhole channels is studied in [3, 7‐9]. In [10], expressions for the exact symbol error rate (SER) of orthogonal space-time block codes (OSTBC) over a double Rayleigh fading keyhole channel are derived for -PSK and -QAM modulation schemes. In a subsequent work [7], similar analysis is also carried out for keyhole channels with Nakagami- fading. In another related work, [11] derives expressions for the pairwise error probability (PEP) of MIMO systems with keyhole channels. However, in [7, 10, 11], the channel is assumed to be spatially uncorrelated. Similar analysis for MIMO keyhole channels with spatial correlations has not been reported so far. It is well known that if the channel correlation matrix is known at the transmitter (e.g., as channel side information (CSI)), a linear precoder matched to the channel correlation structure can be used at the MIMO channel input to improve the system performance (see, e.g., [12‐16]). However, the aforementioned work assumes Rayleigh or Ricean fading and is not applicable to double Rayleigh fading as in the case of channels with keyholes. Formally, a keyhole MIMO channel can be modeled by a matrix which is the outer product of two complex Gaussian vectors [4]. This implies that each coefficient in the channel matrix is a product of two independent complex Gaussian variables (and hence the name double Rayleigh fading). Since keyhole effects lead to the degeneration of the channel matrix [3, 4], it is particularly important to exploit CSI available at the transmitter using a properly designed precoder to enhance the MIMO system performance.

In contrast to previous work, we consider in this paper the SER analysis and the design of linear precoders which minimize the average SER of double Rayleigh fading MIMO channels with spatial correlations. More specifically, we derive easy, to evaluate, exact analytical expressions for the average SER of systems with -PSK, -PAM, and -QAM signaling and maximum likelihood (ML) detection. These closed-form SER expressions are useful in performance analysis of such systems and can be easily evaluated with computing software, using readily available functions. Numerical results are presented in this paper to confirm that our analytical expressions agree with those obtained by simulation of the OSTBC-MIMO system. Based on the analytical expressions for SER, we then present a design of linear precoders which minimize the average SER by exploiting the knowledge of the channel correlation matrix (available at the transmitter either as prior knowledge or via a feedback channel from the receiver), under a constraint on the total average transmitted power. In our formulation, the antenna correlations are allowed to be complex valued [17]. In the general case, we consider it is difficult to find a closed-form solution for the minimum SER (MSER) precoder. As such, the MSER precoder matrix is found by a constrained gradient descent minimization method, based on the derivatives of the average SER. Experimental results are presented to demonstrate the performance improvements achieved by the proposed MSER precoder. In these experiments, MIMO keyhole channels with both real and complex correlation matrices have been considered.

The expressions and the optimal precoder design derived in this paper are novel in the following aspects. Different to [10], the SER expression derived in this paper applies to MIMO keyhole channels with spatial correlations at both transmit and receive antennas. While the SER expressions in [10], which apply only to uncorrelated channels, do not generalize to our case, we show that in the absence of spatial correlations the SER expressions we derive reduce to those in [10]. Note that, for the uncorrelated channel considered in [10], the precoding problem we consider is not relevant. Also, different to [14, 15], the SER expressions and the optimal precoders derived in this paper account for the keyhole effect, that is, each channel coefficient is a product of two complex Gaussian random variables. While there is no direct relationship between these expressions, they correspond to two extreme cases of a double scattering channel with scatterers [4]. In particular, when , the channel exhibits the keyhole effect considered in this paper. When , the MIMO channel is the single-scattering case considered in [14, 15]. It should be noted that neither the results in this paper nor those in [14, 15] readily generalize to the case .

The rest of this paper is organized as follows. Section 2 describes the keyhole channel model and the OSTBC-MIMO system model under consideration. Section 3 derives the SER expressions for this system, where -PSK, -PAM, and -QAM modulation schemes are considered. Section 4 then presents an algorithm for designing an MSER linear precoder for the system. In Section 5, numerical results are presented to demonstrate the accuracy of the given SER expressions and the performance gains achievable with the proposed precoder designs. Finally, concluding remarks are given in Section 6. Some proofs are provided in the appendices.

Notation

is matrix stack operator, is matrix transpose, is Hermitian operation, is complex conjugate, denotes the trace of a matrix, is Frobenius norm, ( is vector norm), denotes the Kronecker product, denotes identity matrix, and defines new symbols.

Channel Model

The gain matrix of a double Rayleigh fading keyhole MIMO channel with transmit antennas and receive antennas can be represented by the outer product of two independent complex Gaussian vectors (size and (size representing transmit and receive Rayleigh fading, respectively, [4], that is, . However, each vector is assumed to be correlated so that and , where is the transmit antenna correlation matrix, is the receive antenna correlation matrix, and and are independent identically distributed (iid) complex Gaussian vectors of sizes and respectively, (here, and are the unique square roots of Hermitian positive semidefinite matrices and , resp., [18]). Note that each element of is a product of two independent Gaussian variables. Note also that the rank of is necessarily one.

System Model

The communication system considered in this paper is shown in Figure 1(b). An orthogonal space-time block (OSTB) encoder operates on a block of input symbols where and is the complex valued modulation signal set with . In this paper, -PSK, -PAM, and -QAM signal sets will be considered. Let the output of the OSTB encoder be a matrix, where is the space dimension and is the time dimension. The elements of are linear combinations of and their complex conjugates. The OSTB codeword is precoded using a matrix to produce the transmitted codeword . Thus, the transmission rate is . The average power of a transmitted codeword is . From the orthogonality property of the OSTBC [21], it follows that

where is the average power of modulation signal set, and is a constant that depends on the particular OSTBC in use. For example, for [21], and for in [19].

The codeword is transmitted through the MIMO keyhole channel whose output is a matrix , where is the complex channel noise matrix. The elements of are independent complex Gaussian variables with iid real and imaginary parts of variance . The receiver decodes channel output using a ML detector. It is assumed that the channel matrix remains constant for the duration of a codeword (quasi-static channel), and that the receiver has the knowledge of and to be used in ML decoding. While is not known to the transmitter, the antenna correlation matrices and are assumed to be known.

We use the moment generating function-(MGF) based approach [23] to find the average output SER of the above-described system, assuming that the precoder matrix is given. Our main goal in this section is to find an easy to evaluate expression for average SER. To this end, we first find an expression for the MGF of the output signal to noise ratio (SNR). When the space-time block code is orthogonal, the MIMO system is equivalent to a SISO system, in which the SNR at the input of the ML detector is given by [10]:

where with being the SNR per receive antenna. Noting that , consider

where is an Hermitian positive semidefinite matrix. Using eigenvalue decomposition, let , where is the unitary matrix of eigen vectors and is the diagonal matrix of whose diagonal elements are eigenvalues , of . Then, (5) can be expressed as

where is also an iid complex Gaussian vector of size . Hence, it follows that

where denotes the th element of . In a similar manner,

where , are the eigenvalues of the matrix and is an iid complex Gaussian vector of size . Note that if the channel is uncorrelated and , then and are chi-square distributed random variables with and degrees of freedom, respectively. This special case was considered in [10]. On the other hand, in general, the MGF of is given by [24, equation (14)]:

A similar expression exists for . In this paper, we mainly focus on the case in which the eigenvalues of both and are distinct. However, the following derivation may be generalized to the case of eigenvalues with multiplicity, by using the appropriate partial fraction expansion, as discussed at the end of this section. For the case of distinct eigenvalues, the partial fraction expansion yields

where

Let the corresponding set of coefficients for be , . Hence, the pdf of can be found by inverse Laplace transform as

The pdf of (where and are independent) is given by [25, page 141]

Then, it follows that the MGF of is

where and is the exponential integral [26, (8.211)]

This integral is readily available as a built-in function in software packages such as Matlab and Mathematica. Given this result, the MGF of the output SNR, can be obtained as .

The average SER, , of several modulation schemes over the correlated keyhole channel can now be derived using the MGF of the random variable [23]. For example, the conditional SER of -PSK given the instantaneous SNR is [23, (8.23)]

where . For convenience, define

Then, the average SER of -PSK signaling can be expressed as

where . Similar expression can be obtained for -PAM and -QAM using [23, (8.5) and (8.12)] which are summarized below:

where , , and .

Note

When and/or have repeated eigenvalues, the partial fraction expansions of and have to be appropriately modified. We now outline the solution for this case. Suppose has distinct eigenvalues with repeating times, and has distinct eigenvalues with repeating times. For this general case, the same procedure as above can be used to show that the MGF of Y is given by (see Appendix A)

where is the confluent hypergeometric function [26, (9.211-4)], is the gamma function, and , are partial fraction coefficients (procedures for finding which are well known). This expression can now be used to generalize the SER expressions (21)–(23). It can be shown that when for all ( ) and for all ( ), (24) reduces to (16). Other interesting cases are when either transmit antennas or receive antennas are uncorrelated, in which case either or is the identity matrix. Also, when both and are identity matrices (no channel correlation), then it is easy to show that (24) reduces to [10, (6)].

Given and , our goal is to find the optimal precoder matrix which minimizes the average SER , subject to a constraint on the average transmitted power in (1). This constrained optimization problem can be solved by minimizing the Lagrangian [27]:

where is the Lagrange multiplier and . It is generally difficult to find directly the minimum of this function, by setting the partial derivatives to zero. Hence, we resort to gradient descent minimization to find locally optimal values for and , starting from an initial value [27]. In general, the solution of a constrained minimization problem such as (25) by gradient descent requires gradient projection, [27, Chapter 11]. However, the specific nature of the constraint (1) allows a simpler approach. To this end, let be the vector of partial derivatives of with respect to the elements of , where denotes the matrix derivative operator[28, Definition 2]. We can perform gradient decent according to

where and is chosen so that the updated solution satisfies the power constraint, by solving

By defining and , (27) can be written as

This equation yields two positive values for for which the updated precoder in (26) satisfies the power constraint, provided that the step size is chosen sufficiently small (see Appendix B). The choice of the smaller value for out of the two possible solutions results in the proper convergence of the gradient descent algorithm (see Figure 8). In each iteration , we first find by solving (28) and then using (26). The convergence of the solution can be decided when becomes less than a prescribed threshold. In the following, we derive closed-form expressions for for different modulation schemes.

While the SER expressions derived in the previous section are simple to evaluate for a given , the elements of the precoder matrix appear only indirectly (in the eigenvalues of ) in these expressions. Consequently, they are not easily differentiable with respect to the precoder matrix. However, a slight reformulation of the given expressions readily eliminates this difficulty as follows. First, we express (14) as . Then, from (18) it follows that

where . Now using [24, (14)], (9) can be rewritten as

Hence,

where and we have obtained SER of -PSK as a function (and hence as an explicit function of ). However, note that the evaluation of this expression requires the explicit evaluation of a double integral, which is avoided in (21) by using the exponential integral. Similar expressions can be derived for -PAM and -QAM using [23, (8.5) and (8.12)] and the same procedure as above, which are summarized below:

where , , and . It can now be seen that computing for (31)–(33) simply involves computing the derivative vector of the form (see Appendix C):

where . Then, by defining

for -PSK, -PAM, and -QAM can expressed, respectively, as

In this section, we present numerical results to verify the analytical SER expressions derived in Section 3 and to demonstrate the benefit of the proposed MSER precoders. In order to confirm the analytical SER expressions derived in Section 3, we compare them with SER estimated by simulating the underlying MIMO system. While the expressions we have derived are valid for a general channel correlation matrix, we here present experimental results obtained for the case of exponential correlation model [29] widely used in the literature. In this case, the transmit and receive antenna correlation matrices are given by , and , where , . In our simulations, we have used . We have considered a number of different OSTBC-MIMO systems to verify the SER expressions. A typical set of examples are presented in Figsure 2 and 3 which confirm that the analytical SER computed with (21), (22), and (23) agrees very well with the values estimated by system simulation.

The results in Figure 2 are for a MIMO system based on the OSTBC from [19] (Alamouti code) for which . In this case, , , that is, rate is one. Figure 3 shows the results for a MIMO system based on the OSTBC from [20] for which and , , , that is, rate is 3/4. Note that no precoding has been used in these cases ( set to a scaled identity matrix). Since spatial correlation in the channel degrades the SER performance, it is also of interest to compare the SER of an OSTBC over a correlated keyhole channel with that over a keyhole channel with no spatial correlation. Such a comparison for the MIMO system is shown in Figure 4 which shows that the degradation of performance due to spatial correlation in the given keyhole channel can be quite significant at high SNR (exponential antenna correlation model is used). Also included here is the SER over an6 Rayleigh fading channel (no keyhole effect and no antenna correlation). In particular, these curves clearly highlight the loss of diversity (as indicated by the slope of the curves [1]) in the MIMO system due to the combined effect of the keyhole propagation and the antenna correlation.

Next, we investigate the performance achievable with MSER precoders found using the proposed design algorithm. Figure 5 shows the performance of the above-described MIMO system with a linear precoder as function of SNR (only 8-PSK and 16-QAM performance is shown, but similar improvements were also observed for 8-PAM). As before, the results in Figure 5 are obtained for the case of exponential correlation at both transmit and receive antennas so that both and have real elements. Since our derivation is valid for a complex Hermitian and positive semidefinite correlation matrix, we also carried out precoder designs for a MIMO system based on from [21], in which the transmit antennas have the exponential correlation matrix while the receive antennas have the complex correlation matrix [22]:

where , , and .

The relevant results are presented in Figure 6. The results in both Figures 5 and 6 clearly demonstrate the effectiveness of the MSER precoder designed by the proposed algorithm. The effect of the precoder is to shift the SER versus SNR curve downwards, without changing its slope at high SNR. Thus, the precoder does not change the diversity order of the system. (The slope of SER versus SNR curve at high SNR determines the diversity order of the system. This asymptotic slope depends on the rank of the channel correlation matrix.) The fact that precoding is less useful at high SNRs is generally known [13]. In our particular case, it can be shown that in the limit , the MSER precoder is proportional to the identity matrix (see Appendix D), that is,

However, at low to moderate channel SNR, the optimal precoder is seen to improve the system performance, the improvement being dependent on the degree of spatial correlation in the channel coefficients. It was also observed that at low SNR, the precoder matrix obtained by the design algorithm had nearly identical elements in all the cases we considered. If the MSER precoder matrix has equal elements, it can be shown that there exists an equivalent solution (which is also optimal in the MSER sense) of the form (see Appendix E):

where is an vector and is a vector with all elements equal to zero. Then, from the discussion in [13, Section IV- A], it follows that the resulting precoding scheme is also equivalent to beamforming in the direction of . Note that if the MSER precoder matrix is known to have equal elements, it can be directly determined by using the power constraint.

Note that the result in Appendix E shows the existence of multiple local minima. Since the given descent algorithm may converge to a local minimum, one can choose the best design among multiple designs obtained with random initializations for the precoder. In our experiments, the best solution obtained in this manner was comparable to the one obtained with the diagonal initialization which is the scaled (to satisfy the power constraint) identity matrix. It is also possible to use a relaxation-type algorithm in which the SNR is reduced from a higher value (for which the diagonal precoder is nearly optimal) to the desired value in steps, and to progressively optimize the precoder to each SNR value. In all our designs, the gradient descent minimization algorithm converged rapidly to a stable solution. A typical example is shown in Figure 7 which shows the convergence of both the precoder matrix and the Lagrange multiplier. Thus, this algorithm can be used to adaptively update the precoder matrix based on, for example, the estimates of the channel correlation matrix obtained via measurements. Typically, the channel correlation matrix changes much slower than the channel matrix itself and hence can be estimated periodically at the receiver and be fed back to the transmitter for adapting the precoder matrix.

Exact analytical expressions were derived for SER of -PSK, -PAM, and -QAM modulated OSTBCs over a MIMO spatially correlated keyhole channel. A general complex correlation matrix has been assumed in the derivations. These expressions are easy to compute using numerical software and have been verified by Monte Carlo simulations. The given analytical expressions have been used to design MSER linear precoders based on the knowledge of channel correlation matrix available as CSI at the transmitter. Using simulation experiments, it has been demonstrated that the proposed MSER precoder can significantly reduce the error probability of a MIMO system operating on a fading channel degraded by a keyhole effect.

An important extension to this work includes precoder design for more general multiple scattering fading channels [5]. In particular, an interesting case is the double scattering model [4] (of which the keyhole channel is a special case).