A Comparison of Signalling Alphabets

C. E. Shannon's encoding theorems1 associate with the channel of a communications system a capacity C. These theorems show that the output of a message source can be encoded for transmission over the channel in such a way that the rate at which errors are made at the receiving end of the system is arbitrarily small provided only that the message source produces information at a rate less than C bits per second. C is the largest rate with this property. Although these theorems cover a wide class of channels there are two channels which can serve as models for most of the channels one meets in practice. These are: 1. The binary channel This channel can transmit only sequences of binary digits 0 and 1 (which might represent hole and no hole in a punched tape; open-line and closed line; pulse and no pulse; etc.) at some definite rate, say one digit per second. There is a probability p (because of noise, or occasional equipment failure) that a transmitted 0 is received as 1 or that a transmitted I is received as 0. The noise is supposed to affect different digits independently. The cpacity of this channel is C = 1 + p log p + (1 - p) log (1 - p) (1)