Exact Symbol Error Probability of Cross-QAM in AWGN and Fading Channels

Quadrature amplitude modulation (QAM) has been widely used in digital communication systems due to its high bandwidth efficiency. When the number of bits per symbol is even, transmission can be implemented easily by using square QAM. However, if there is a requirement for the transmission of an odd number of bits per symbol, the rectangular QAM is not a good choice in terms of power efficiency. The issue was overcome by Smith who proposed cross-QAM constellation which is obtained from a square constellation by removing some outer point in each corner and is given the shape of a cross [1]. Smith shows that both the peak and average power can be reduced by using a cross-QAM constellation, and there is at least a 1-dB gain in the average signal-to-noise ratio.

Recently, cross-QAM has been found to be useful in adaptive modulation schemes wherein the constellation size is adjusted depending on the channel quality [2‐6]. As the channel quality improves, the constellation size is expected to be increased by incrementing to . If one were to use just square QAM, the increments should be from to (for instance, we need to go from 16 to 64 to 256-QAM ). Using cross-QAM, however, the increment is smoother (16-QAM to cross 32-QAM to 64-QAM ). The steps between consecutive squared constellations are too big, especially for small constellations [7]. An intermediate step (corresponding to odd powers of 2) will make the system to work with more granularity obtaining greater coverage for a determined data rate [2]. As a result, cross-QAMs have been adopted in many practical systems. For example, cross-QAMs with constellations from 5 bits to 15 bits have been used in ADSL and VDSL [8, 9], and cross 32-QAM and cross 128-QAM are adopted in DVB-C [10]. On the other hand, cross-QAMs have special application in blind equalization [11‐14].

Despite the immense importance of cross-QAM, the implementation and the calculation of the average symbol error probability (SEP) of cross-QAM are more complicated compared to that of square and rectangular QAMs since the inphase and quadrature components of cross-QAMs cannot be demodulated independently. So, the calculation of SEP of cross-QAM cannot be reduced to a one-dimensional problem by using the independence of the inphase and quadrature components, as can be done for square QAM and rectangular QAM. It can be found the upper bounds of the SEP of cross-QAM in [1] and [15, Chapter 5]. Recently, [16] derives the bit error probability (BEP) of cross-QAM constellations with Smith-style Gray coding in additive white Gaussian noise (AWGN) and Rayleigh flat fading. However, the closed-form solutions to BEP for AWGN and Raleigh flat fading channels in [16] are very complicated, not to mention that [16] does not consider the SEP of cross-QAM and that it is not straightforward to extend the result of [16] to SEP calculation. More recently, an exact closed-form SEP expression for 32 cross-QAMs in AWGN has been derived in the form of a finite sum of Gaussian -functions [17]. Note, however, that the decision regions of the corner points for other cross-QAM modulation are more complicated than 32-cross-QAM. In [18], an infinite double series of the products of Gaussian -functions for the SEP of 128-cross-QAM and 512-cross-QAM in AWGN is derived. The difficulty arises when one tries to find the closed-form exact SEP expression of arbitrary -ary cross-QAM in AWGN, which involves the calculation of the two-dimensional joint Gaussian -function.

In fading channels, on the other hand, the signal-to-noise ratio (SNR) of the received symbol becomes a random variable. As such, the usual method to find the average SEP in fading channels is to first get the conditional SEP as if in an AWGN channel (conditioned on SNR or channel realization) then average the conditional SEP over SNR or channel realization. This approach will, in many cases, result in an integral, usually being two-dimensional integral even three-dimensional integral, that cannot be integrated into a closed form and hence must be integrated numerically. By using the "preaveraging" technique [19], opposed to the customary and widely adopted "postaveraging" technique, the exact SEP of cross-QAM in Rayleigh fading channels has been derived in [20], where the closed-form SEP expression obtained contains only elemental functions (trigonometric). However, the "preaveraging" technique involves tedious calculations again, and it would only apply for Nakagami- channels with integer parameter , not to mention whether it can be extended to encompass Rice and Hoyt fading. Perhaps, the best method to get SEP of digital modulation signaling over fading channels is given by Simom and Alouini [21]. They use the moment generating function- (MGF-) based approach to obtain SEP expressions for various digitally modulated signals over fading channels. Their basic technique is to rewrite the Gaussian -function into a preferred form of an integral with finite integration limits (many SEP expressions contain the Gaussian -function), so that the final average SEP expression can be numerically computed with more accuracy. In some special cases, the MGF-based method can even lead to an exact closed-form SEP without undone integrals.

In this paper, using the alternate representation of the two-dimensional joint Gaussian -function and the MGF-based method, the exact SEP expressions of arbitrary -ary cross-QAM in AWGN and fading channels, including Rayleigh, Nakagami-m, Rice, and Nakagami-q (Hoyt) channels, have been derived. The closed-form SEP expressions obtained contain Gaussian -functions and a finite (in proportion to ) sum of single integrals with finite limits and an integrand composed of elementary (exponential, trigonometric, and power) functions, thus readily enabling numerical evaluation.

The remainder of this paper is organized as follows. First, Section 2 briefly introduces the problem background. Next, Sections 3 and 4 derive the average SEP expressions for cross-QAM in AWGN channels and fading channels, respectively. And then, Section 5 gives the simulation and numerical results. Finally, Section 6 concludes the contributions of this work.

While the SEP expressions for the interior symbols and edge symbols can be deduced easily from the SEP expressions for square QAM in [15, pages 265 and 278], the SEP of corner symbols needs to be derived separately.

The SEP of any interior symbol and edge symbol can be written, respectively, as

For the corner symbols, from the left to right or from the bottom to the top as shown in Figure 2, the SEP of the th point is given by

where is the probability density function (PDF) of the standard normal distribution and given by

Having derived the exact SEP of all the points, the exact average SEP for the -ary cross-QAM constellation can be written as

Appendix shows that, with the integral function defined in (4), (17) can be rewritten as

where

Although it is difficult to express the function in a closed form without integration, it is a one-dimensional integral with finite limits, and its integrand only composes of elementary (exponential and trigonometric) functions, while the function actually is a two-dimensional integral of exponential function with infinite limits according to (A.2) in Appendix. With (4), in fact, can be easily and accurately evaluated numerically (see, e.g., using MATLAB). Note that the numerical calculation of (4) is much simpler and more precise than that of the infinite double series of the products of Gaussian -functions of equation (3) in [18].

Note that, when , only the first three terms of the right side of (18) are retained, while the last three terms disappear since . So, the exact SEP expression of 32-ary cross-QAM is

which is the same as equation (5) in [17].

If ignoring the last three terms and only retaining the first three terms in (18), we will obtain a very tight approximation of the exact SEP written as

Note that, when , (21) is also the exact SEP. On the other hand, the first term of (21) is just the average number of nearest neighbors or the Gilbert approximation. At SNR = 0 (i.e., ), especially, Gilbert approximation is , which is much larger than one, while (21) yields , which is the exact SEP.

The fading channels, including Rayleigh, Nakagami-m, Nakagami-q (Hoyt), and Nakagami-n (Rice) channels, are considered in this section. The probability density function (pdf) of the instantaneous received SNR in these channels can be written as, respectively, [21]

where is the instantaneous SNR of the received symbol, is the instantaneous fading amplitude of the channel, is the energy of each transmitted symbol, is the one-sided power spectral density of the zero-mean AWGN, is the average received SNR per symbol, and denotes the expectation operator. And is the modified Bessel function of the first kind and zeroth order. The moment generating function (MGF) corresponding to (22) is, respectively, given by [21]

Averaging the SEP expression (18) over the fading distribution of the received SNR, , induces the average SEP of arbitrary -ary cross-QAM over fading channel as given by

where denotes the PDF of and

Note that the relationships and for , mentioned above, are applied in the expression (24). Particularly, the approximate average SEP is only considering the first three terms in (24).

According to (4), it is possible to re-express the integral (25) in terms of the MGF of as given by

Substituting (23) into (26), the corresponding expressions of in (24) for Rayleigh, Nakagami-m, Nakagami-q (Hoyt), and Nakagami-n (Rice) channels are given as

where for the sake of notational convenience.

Using the above four integral expressions, the average SEP (24) over fading channels can be conveniently evaluated through numerical integration since these formulae are single integrals with finite limits and an integrand composed of elementary (exponential, trigonometric, and/or power) functions. Note that (27) and (28) with integer can also be evaluated in closed form using equation (5A.24) in [21, page 155]. Furthermore, the author in [20] has given a simpler closed-form expression for the SEP of cross-QAM in Rayleigh channel.

In this section, using the above formulations, numerical results concerning the SEP performance of cross-QAM and the comparison with rectangular QAM are presented, together with computer simulations. For fading channels, the Rayleigh channel is chosen as a special case since the results of the other type channels are similar to that of Rayleigh channel. So, the corresponding figures of the other type channels are not shown here for limited space. Figure 3 shows excellent agreement between the analytical expressions and the simulation results for cross -QAMs over AWGN channels and Rayleigh fading channels, respectively.

Shown in Figure 4 is the comparison of the exact SEP and its approximations over AWGN channels and Rayleigh fading channels, respectively. In AWGN channels, it was shown that although the approximations of Proakis and Gilbert are accurate enough at high SNR, both of them are very loose and the former is looser at low SNR. From Figure 5(a), it can be seen that at given SEP the SNR differences between the exact SEP and the approximations of Proakis and Gilbert become smaller and smaller as SEP decreases. However, it can be observed from Figure 4(b) that the difference between the exact SEP and the approximations of Proakis and Gilbert in Rayleigh fading channels is larger than that in AWGN channels at high SNR. From Figure 5(b), especially, it can be found that their SNR differences at the same SEP value are about 1 dB. On the other hand, the proposed approximation is very tight for all the SNR values whether in AWGN channels or in Rayleigh fading channels. In fact, since their SNR differences are almost 0 for all and all given SEP, so the proposed approximation is almost the same as the exact SEP.

Figure 6 compares the exact SEP of rectangular QAM and cross-QAM over AWGN channels and Rayleigh fading channels, respectively. Here, the -ary rectangular QAM signal constellation is assumed to be formed by drawing the inphase and quadrature components from the independent -ary pulse amplitude modulation (PAM) and -ary PAM, respectively, where , , , , and . Specifically, -ary, -ary, -ary, and -ary rectangular QAMs are considered here. In addition, the quadrature-to-inphase decision distance ratio in these constellations is set to be 1, which is the best ratio in terms of symbol error performance as shown in [25, 26]. The exact SEP of -ary rectangular QAM over AWGN and fading channels are given in [27‐29]. From Figure 7, it can be seen that cross-QAM exhibits at least 1.1 dB SNR gain over rectangular QAM when SEP < 0.3. Numerical evaluation under other fading scenarios with different fading parameters, which are not shown here for limited space, also reveals similar SNR gain.

In this paper, the exact SEP expressions of cross-QAM in AWGN channel and fading channels have been derived. The obtained closed-form SEP expressions contain a finite sum of single integrals with finite limits and an integrand composed of elementary (exponential, trigonometric, and/or power) functions, which can be easily and accurately evaluated numerically. Simple and very precise approximations, which contain only Gaussian -function for AWGN channel and contain three terms of the single integrals mentioned above for fading channels, respectively, are also given. The analytical expressions show excellent agreement with the simulation results, and the numerical evaluation with the proposed expressions reveals that cross-QAM can obtain at least 1.1 dB gain compared to rectangular QAM when SEP <0.3 in all the considered channels. The obtained exact SEP expressions will provide valuable insight into the design of wireless systems. In particular, the exact SEP performance of cross-QAM in AWGN channel will be very useful in adaptive modulation wherein the constellation size is adjusted depending on the channel quality through the SEP performance of the adopted modulations with AWGN.