## 1 Introduction

## 2 System model

_{1},b

_{2},…,b

_{ L }] and each has length L. The sequences of bits are then passed to a digital modulation scheme which converts them into sequences of symbols, each has length M and whose elements take values in constellation set Ω. The digital modulation can perform binary phase-shift keying (BPSK), quadrature phase-shift keying (QPSK), or high order modulation such as 16-quadrature amplitude modulation (QAM) and 64-QAM. Other techniques, such as single carrier frequency division multiple access (SC-FDMA) or orthogonal frequency division multiplexing (OFDMA), can be included in the transmitter to improve the system reliability. After bit-to-symbol mapping, the modulation transforms the symbol stream into an analog signal suitable to be sent through the transmission channel which can degrade the signal quality.

## 3 Semi-analytical error probability derivation

### 3.1 Bit error probability definition

_{1},x

_{2},…,x

_{ N }} at the output of the matched filter and makes a decision to estimate the information bits. Due to the channel effect, this decision can be erroneous. So, it is important to measure the communication system efficiency in terms of bit error probability (BEP). According to the system model presented in Fig. 1, this bit error probability is defined to be the conditional probability that the receiver makes a wrong decision on a transmitted information bit. Assuming that the ith bit is transmitted, the error probability is expressed as follows:

_{ i }as

_{ i }defined in (1) is then re-expressed as

_{ e }, we divide the set of the observed samples C into two subsets C

_{0}and C

_{1}. The first subset contains N

_{0}observed samples which corresponds to the transmission of b

_{ i }=0. The second subset consist of N

_{1}observed samples when the bit b

_{ i }=1 is transmitted. In this manner, the probability density function of X can be viewed as a mixture of two probability densities \(f_{X}^{(1)}(x)\) and \(f_{X}^{(0)}(x)\) of the observed samples corresponding to the transmitted information bits b

_{ i }=1 and b

_{ i }=0, respectively. Then, the average bit error probability is written as

_{ i }=k is transmitted.

_{ e }, one has to estimate the probability densities \({f_{X}^{1}}(x)\) and \({f_{X}^{0}}(x)\). In this paper, we will focus on the use of Fourier inversion approach and its use for estimating error probability.

### 3.2 Probability density function estimation

_{ X }is the characteristic function of a random variable X, defined as

_{1},x

_{2},…,x

_{ N }}, the expectation in (8) can be approximated by a finite sum. Hence, the characteristic function φ

_{ X }can be written as

_{ X }(t) given in (9). However, the Fourier integral in (7) can exhibit divergence for large values of the time variable t. To solve this limitation, the characteristic function estimator \(\widetilde {\varphi }_{X}(t)\) is multiplied by a damping function ψ

_{ h }(t)=ψ(h t) to control the smoothness of the estimated probability density function.

_{ i })

_{0}and (x

_{ i })

_{1}are the observed samples corresponding to the transmitted bits b

_{ i }=0 and b

_{ i }=1, respectively. h

_{1}(respectively, h

_{0}) is the smoothing parameter which depends on the number of observed samples, i.e., N

_{1}(respectively, N

_{0}). Q(:) denotes the complementary unit cumulative Gaussian distribution, that is

## 4 Smoothing parameter selection

_{opt}since it depends on the unknown quantity R(f

^{′′}). To solve this problem, several types of MISE-based methods have been suggested in literature. Hereafter, we detail the most popular ones.

### 4.1 Rule-of-thumb method

^{2}, i.e., \(\mathcal {N}(\mu,\sigma ^{2})\). In this manner, we get

### 4.2 Cross-validation method

^{′′}) in h

_{opt}formula. Furthermore, CV approach considers the integrated squared error (ISE) to select the optimal smoothing parameter. This error metric is expressed as [14]

_{ i }.

^{′′}) by the estimator:

### 4.3 Bootstrap method

_{1},…,x

_{ N }}, with a pilot smoothing parameter g.

^{∗}[24] as:

^{∗}(h) as

_{opt,boot}is obtained by minimizing MISE

^{∗}(h) with respect to h:

_{opt,boot}obtained from (35) is given as (see proof in Appendix D):

_{opt,boot}value depends on the second derivative of the estimate pdf \(\int _{}^{}(\,\widetilde {f}_{X}^{\prime \prime }(x;g))^{2}\,dx\) where the pilot smoothing parameter g is selected using least squares the cross-validation method [25]. This parameter is chosen so as to minimize

_{ N,−i }(x

_{ i },g) is the density estimate based on all of data expect x

_{ i }. To justify the choice of the bootstrap method for selecting the optimal smoothing parameter, we have presented the integrated squared error as a function of the smoothing parameter h. Figure 2 shows the obtained results with bootstrap, cross-validation, and rule-of-thumb methods. It is seen that the bootstrap method outperforms the other methods in terms of the integrated squared error between the true probability density and the estimated density.

## 5 Simulations and results

^{−4}, Monte Carlo simulation requires a number of 1,000,000 samples while Fourier inversion uses only 10,000 observed samples.

Computing time (s) | |||
---|---|---|---|

BEP | Proposed SPP | MC simulation | |

BPSK | 10 ^{−6}
| 2.106 | 154.179 |

10 ^{−5}
| 1.760 | 14.001 | |

10 ^{−4}
| 1.013 | 1.441 | |

QPSK | 10 ^{−6}
| 2.554 | 86.933 |

10 ^{−5}
| 1.734 | 9.013 | |

10 ^{−4}
| 1.025 | 2.752 | |

4-PAM | 10 ^{−6}
| 5.877 | 71.864 |

10 ^{−5}
| 5.309 | 8.1315 | |

10 ^{−4}
| 3.663 | 4.333 | |

SC-FDMA | 10 ^{−6}
| 2.631 | 54.810 |

10 ^{−5}
| 2.048 | 8.131 | |

10 ^{−4}
| 1.671 | 1.453 |

^{−3}, Monte Carlo simulation requires a number of samples equal to 100,000 while the proposed method uses only N=10,000 samples. In addition, the same performance in terms of bit error probability has been obtained when quadrature phase-shift keying (QPSK) modulation is considered. The simulation results are presented in Fig. 7.

## 6 Conclusions

## 7 Appendix A Proof of (11)

## 8 Appendix B Proof of (14)

_{ i })

_{1}and (x

_{ i })

_{0}, respectively, which corresponds to transmitted information bits b

_{ i }=1 and b

_{ i }=0, respectively. By using the obtained semi-analytical probability density function in (13), we can define

_{1}(respectively, h

_{0}) is the smoothing parameter which depends on the number of observed samples, i.e., N

_{1}(respectively, N

_{0}). By substituting the estimated pdf \(\widetilde {f}_{X}^{(1)}\) and \(\widetilde {f}_{X}^{(0)}\) in (B.1), we get

## 9 Appendix C Proof of (18)

^{′}th-order kernel, we take the expansion out to the ν

^{′}th-term to solve this integral:

## 10 Appendix D Proof of (36)

^{∗}, and Var

^{∗}all involve expectations conditionally upon \(x^{*}_{1}, x^{*}_{2},\ldots, x^{*}_{N}\) and all x

^{∗}are sampled from the smoothed distribution \(\widetilde {f}_{X}(x;h)\). Making a substitution followed by a Taylor series expansion, this assumes that h→0 as N→∞, gives an asymptotic approximation: