Papoulis [1] shows that a band-limited signal may be written

where is the value of at the sample time and is the bandwidth. The signal spectrum is assumed to vanish for frequencies outside the domain . Reza [2] considers a signal of this type that is approximately limited to a time interval of duration , e.g., the signal vanishes for times outside the domain . The duration of the functions is on the order of and the time between samples is . The duration of the signal is thus given approximately as (number of samples + 1). Now is assumed throughout these calculations so

Next, the average power in the signal is computed

as shown in detail in [3]. It turns out that the functions in Eq. (1) form an orthogonal set [2, 3]. Thus Reza argues that the sum in Eq. (2) can be viewed as the norm squared of a vector in a dimensional vector space. The vector coordinates are given by the and is assumed. If the length of the vector is then from Eq. (2)

Thus in the Gaussian noise channel with input , output and noise satisfying

the input signal is represented by a point a distance from the origin in the dimensional space. The output signal is represented by a point a distance

from the origin, given that the input signal and the noise are uncorrelated. The noise is represented by a point a distance

from the origin.

The requirement for transmission of signals without noise is that the allowable signal points in the dimensional space must be separated a distance given by twice the length of the noise vector. Each of the received signals are represented by a point on a sphere with radius in dimensional space.

The question is now how many distinct signals (points on the sphere) can be allowed while keeping the separation between the points equal to ? Enforcing this requirement permits decoding this signal without ambiguity. Alternatively, one can ask how many non-overlapping noise spheres can be embedded in the surface of the output signal’s sphere? Each noise sphere has radius and has it’s center on the surface of the output signal’s sphere. Reza argues this problem is equivalent to asking how many spheres of radius can be placed within the sphere of radius because for very large, e.g., in a very high dimensional space, most of the volume of a sphere is close to its surface.

According to these prescriptions the number of allowed signals is given by

where the volumes are to be computed in dimensional space. Since the volume of a sphere in dimensional space is proportional to where is the sphere’s radius,

The number of bits sent by these allowed signals is

and so the channel capacity in bits/s is

which is Shannon’s [4] famous result. A more detailed derivation is provided in [3] and is available by clicking below.

**References**

[1] A. Papoulis, *Probability, Random Variables and Stochastic Processes*, McGraw-Hill, N.Y. (1965), p. 176.

[2] F. M. Reza, *An Introduction to Information Theory*, McGraw-Hill, N.Y. (1961), pp. 318 – 320.

[3] H. L. Rappaport, *Notes on Information Theory II and the Geometric Interpretation of the Shannon Channel Capacity, *7G Communications*, *7GCTN02 (2014); infoII

[4] C.E. Shannon, *A Mathematical Theory of Communication*, Bell System Technical Journal (1948).

Tags: technology

## Leave a Reply