A New Approach for Evaluating the Error Probability in the Presence of Intersymbol Interference and Additive Gaussian Noise

An important problem in the analysis of binary digital data systems is the determination of the system performance in the presence of intersymbol interference and additive gaussian noise. Since it is usually the most meaningful criterion in designing a digital data * In April 1970. the authors were advised by R. W. Pulleyblank that a similar method was discovered independently by M. Celebiler and 0. Shimbo to bp presented in a paper which will be published in Conference Record, ICC, 1970. 2249 2250 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1970 system, the error probability is chosen as the measure of the system performance. Two alternatives are available at present. The first alternative 1,2 considers a truncated iV-pulse-train approximation of the true channel. The error probability is calculated by evaluating the conditional error probability of each of 2N possible data sequences and averaging over all 2N sequences. Since each calculation of the conditional error probability takes a great deal of computer time, the number of sequences must be held to several thousand. 3 This limitation leads to a poor approximation of the true channel, and the error probability so obtained is not very useful. The second alternative evaluates an upper bound of the error probability by either the worst-case sequence3 or the Chernoff: inequality. 4 - 5 In many cases, the bound is too loose. In this study we have developed a new way to evaluate the error probability in terms of the first 2k moments of the intersymbol interference.