2: The Sampling Theorem

Analog-to-Digital Conversion

Because of the way computers are organized, signal must be represented by a finite number of bytes. This restriction means that both the time axis and the amplitude axis must be quantized: They must each be a multiple of the integers. ¹ Quite surprisingly, the Sampling Theorem allows us to quantize the time axis without error for some signals. The signals that can be sampled without introducing error are interesting, and as described in the next section, we can make a signal "samplable" by filtering. In contrast, no one has found a way of performing the amplitude quantization step without introducing an unrecoverable error. Thus, a signal's value can no longer be any real number. Signals processed by digital computers must be discrete-valued: their values must be proportional to the integers. Consequently, analog-to-digital conversion introduces error.

Figure \(\PageIndex{1}\): Analog to digital signal stages of signal representation. Sasche Spors, Digital Signal Processing - Theory and Computational Examples. Introduction [https://nbviewer.org/github/spatiala...duction.ipynb]

The Sampling Theorem

Digital transmission of information and digital signal processing all require signals to first be "acquired" by a computer. One of the most amazing and useful results in electrical engineering is that signals can be converted from a function of time into a sequence of numbers without error: We can convert the numbers back into the signal with (theoretically) no error. Harold Nyquist, a Bell Laboratories engineer, first derived this result, known as the Sampling Theorem, in the 1920s. It found no real application back then. Claude Shannon, also at Bell Laboratories, revived the result once computers were made public after World War II.

The sampled version of the analog signal s(t) is s(nT_s), with T_s known as the sampling interval. Clearly, the value of the original signal at the sampling times is preserved; the issue is how the signal values between the samples can be reconstructed since they are lost in the sampling process. To characterize sampling, we approximate it as the product:

\[x(t)=s(t)P_{T_{s}}(t) \nonumber \]

with P_Ts being the periodic pulse signal. The resulting signal, as shown in Figure 5.3.1 has nonzero values only during the time intervals :

\[\left ( nT_{s}-\frac{\Delta }{2}, nT_{s}+\frac{\Delta }{2}\right ),n\in \left \{ ...,-1,0,1,... \right \} \nonumber \]

For our purposes here, we center the periodic pulse signal about the origin so that its Fourier series coefficients are real (the signal is even).

\[P_{T_{s}}(t)=\sum_{k=-\infty }^{\infty }c_{k}e^{\frac{i2\pi kt}{T_{s}}} \nonumber \]

\[c_{k}=\frac{\sin \left ( \frac{\pi k\Delta }{T_{s}}\right )}{\pi k} \nonumber \]

If the properties of s(t) and the periodic pulse signal are chosen properly, we can recover s(t) from x(t) by filtering.

To understand how signal values between the samples can be "filled" in, we need to calculate the sampled signal's spectrum. Using the Fourier series representation of the periodic sampling signal,

\[x(t)=\sum_{k=-\infty }^{\infty }c_{k}e^{\frac{i2\pi kt}{T_{s}}}s(t) \nonumber \]

Considering each term in the sum separately, we need to know the spectrum of the product of the complex exponential and the signal. Evaluating this transform directly is quite easy.

\[\int_{-\infty }^{\infty }s(t)e^{\frac{i2\pi kt}{T_{s}}}e^{i2\pi ft}dt=\int_{-\infty }^{\infty }s(t)e^{-\left ( i2\pi \left ( f-\frac{k}{T_{s}} \right )\right )}dt=S\left ( f-\frac{k}{T_{s}} \right ) \nonumber \]

Thus, the spectrum of the sampled signal consists of weighted (by the coefficients c_k) and delayed versions of the signal's spectrum Figure 5.3.2 below.

\[X(f)=\sum_{-\infty }^{\infty }c_{k}S\left ( f-\frac{k}{T_{s}} \right ) \nonumber \]

In general, the terms in this sum overlap each other in the frequency domain, rendering recovery of the original signal impossible. This unpleasant phenomenon is known as aliasing.

If, however, we satisfy two conditions:

• The signal s(t) is bandlimited—has power in a restricted frequency range—to W Hz
• the sampling interval T_s is small enough so that the individual components in the sum do not overlap - Ts < 1/2 W

Aliasing will not occur. In this delightful case, we can recover the original signal by lowpass filtering x(t) with a filter having a cutoff frequency equal to W Hz. These two conditions ensure the ability to recover a bandlimited signal from its sampled version: We thus have the Sampling Theorem.

'''
    Aliasing Demonstration
'''
import numpy as np
import matplotlib.pyplot as plt

# Define the original signal
f_original = 5  # Frequency of the original signal (Hz)
fs = 200  # Sampling frequency (Hz)
t = np.arange(0, 1+1/fs, 1/fs)  # Time vector

# Generate the original signal
original_signal = np.sin(2 * np.pi * f_original * t)

# Undersample the signal
fs_undersampled = 4  # Undersampled sampling frequency (Hz)
t_undersampled = np.arange(0, 1+1/fs, 1/fs_undersampled)
undersampled_signal = np.sin(2 * np.pi * f_original * t_undersampled)

# Plot the signals
plt.figure(figsize=(10, 6))
plt.plot(t, original_signal, label=f'Original Signal ({f_original} Hz)')
plt.plot(t_undersampled, undersampled_signal, 'ro-', 
         label=f'Undersampled Signal (fs = {fs_undersampled} Hz)')
plt.xlabel('Time (s)')
plt.ylabel('Amplitude')
plt.title('Aliasing Demonstration')
plt.legend(loc='upper right')
plt.grid()
plt.show()

The frequency 1/2T_s, known today as the Nyquist frequency and the Shannon sampling frequency, corresponds to the highest frequency at which a signal can contain energy and remain compatible with the Sampling Theorem. High-quality sampling systems ensure that no aliasing occurs by unceremoniously lowpass filtering the signal (cutoff frequency being slightly lower than the Nyquist frequency) before sampling. Such systems therefore vary the anti-aliasing filter's cutoff frequency as the sampling rate varies. Because such quality features cost money, many sound cards do not have anti-aliasing filters or, for that matter, post-sampling filters. They sample at high frequencies, 44.1 kHz for example, and hope the signal contains no frequencies above the Nyquist frequency (22.05 kHz in our example). If, however, the signal contains frequencies beyond the sound card's Nyquist frequency, the resulting aliasing can be impossible to remove.

If we satisfy the Sampling Theorem's conditions, the signal will change only slightly during each pulse. As we narrow the pulse, making

 Continuous-Time Periodic Signals

A continuous-time period signal must meet the following requirement:

\[x(t) = x(t + T), \; -\infty < t < \infty\]

for example

\[x(t) = A \cos(2 \pi F t \; + \; \theta),  \; where \; T = 1/F\]

where F may be an irrational floating point number.

Discrete-Time Periodic Signals

A discrete-time period signal (or sequence) must meet the following requirement:

\[x(n) = x(n + N), \; -\infty < n < -\infty\]

where n and N are integers.

Since n and N must be integers, there an additional requirement for trigonometric functions.

\[x(n) = A \cos(\omega\, n) \]

is a discrete-time periodic signal if 

\[\omega \, N = 2 \pi \,k \]

where the period, N, and k are both integer, and the frequency, \(f = \frac{k}{N}\), is a rational fraction. 

'''
    Example of a Periodic function
'''
import numpy as np
import matplotlib.pyplot as plt

k, N = 3, 10
n = np.arange(0,2*N)

x = np.cos(2*np.pi*n*k/N)
plt.figure()
plt.stem(n,x)
plt.xlim(0,20)
plt.grid()
plt.xlabel('n')
plt.ylabel('x(n)')
plt.xticks(np.arange(0,2*N,2))
plt.show()

Figure \(\PageIndex{5}\): Example periodic sequence. Two periods.