Sound, signals and sampling

When a guitar string vibrates, it squashes and stretches the air beside it, and those squashes and stretches (compressions and rarefactions) travel outwards as a pressure wave at about 343 m/s in air at 20 °C. A microphone's diaphragm moves with the pressure, and the microphone turns that movement into a voltage: more pressure, higher voltage. That changing voltage is a signal, a quantity that changes over time and carries information.

The simplest signal is a sine wave, the sound of a tuning fork. Its amplitude is the height from the middle line to a peak, and a bigger amplitude sounds louder. Its period $T$ is the time for one complete cycle, in seconds, and its frequency $f$ is the number of cycles each second, in hertz (Hz). They are two views of the same thing:

The A that orchestras tune to is 440 Hz, so its period is $1/440 \approx 2.27$ ms. Doubling a frequency raises a note by an octave; halving it lowers it by one.

Phase says where in its cycle a wave is at time zero, in degrees or radians. You cannot hear the phase of a single tone, but when signals are added together, phase decides whether they reinforce or cancel.

Young, healthy ears hear from about 20 Hz up to about 20 kHz. The top end falls with age, and many adults hear little above 15 or 16 kHz. That 20 kHz figure is what decides how fast audio has to be sampled.

The microphone's voltage varies smoothly and continuously. It is analogue: a likeness, an analogue, of the pressure wave. Every time an analogue signal is copied, stored or sent, a little noise and distortion creep in, and in general they cannot be taken out again; copy a tape, then copy the copy, and by the fifth generation you can hear the hiss.

A digital signal is a list of numbers, and a number can be copied a million times without changing. Numbers can also be processed by a computer: filtered, mixed, compressed and sent round the world.

Turning a voltage into numbers takes two steps. Sampling measures it at regular moments in time, and is this lecture's subject. Quantisation rounds each measurement to one of a fixed set of levels, and is Lecture 2's. Sampling chops up time; quantisation chops up the voltage.

Imagine photographing the voltage at regular intervals and writing down each value. Each measurement is a sample. The number of samples taken each second is the sampling rate, or sampling frequency, $f_s$, in hertz. The time between samples is the sampling period:

A CD samples at 44.1 kHz, 44 100 samples every second, so its samples are $1/44\,100 \approx 22.7\ \mu\text{s}$ apart. Once a signal has been sampled the smooth wave is gone and only the dots remain. The question for the whole lecture is whether the wave can be got back from them.

On average, each cycle of a wave gets $f_s/f$ samples (any one cycle holds a whole number of them). At 44.1 kHz a 1 kHz tone gets 44.1 samples per cycle, and a 10 kHz tone gets 4.41: fewer, but still enough.

To capture a frequency you need more than two samples in each of its cycles. Put the other way round, the sampling rate must be more than twice the highest frequency in the signal:

This holds for signals that start at 0 Hz, such as audio. (A signal confined to a narrow band far above 0 Hz can be sampled more slowly, by a technique called bandpass sampling.)

This is the sampling theorem, often called the Nyquist theorem or the Nyquist–Shannon theorem. Its discovery was shared: E. T. Whittaker found the rebuilding formula in 1915, Harry Nyquist's 1928 telegraph work gave the factor of two, Vladimir Kotelnikov stated and proved it for communications in 1933, and Claude Shannon's 1949 paper, which gave the full proof for engineers and called the result common knowledge, made it famous. Half the sampling rate, $f_s/2$, is the Nyquist frequency: the limit for a given sampling rate, since only frequencies below it can be captured.

Why more than two, and not exactly two? At exactly two samples per cycle the samples might land on the peaks, and the wave shows up, though its true size and phase still cannot be told. But they might equally land on the zero crossings, and then every sample is zero and the wave vanishes. Exactly two is not safe; just over two is enough.

Books differ in their words. Here, as in most audio work, the Nyquist frequency is $f_s/2$. Oppenheim and Schafer instead call a signal's highest frequency its Nyquist frequency and twice that its Nyquist rate, and state the theorem with $\ge$ for a signal that has nothing at or above that frequency; Smith uses both names for $f_s/2$.

Hearing tops out at about 20 kHz, so audio needs a sampling rate above 40 kHz. A CD's 44.1 kHz gives a Nyquist frequency of 22.05 kHz, with a little room to spare, which the anti-aliasing filter needs (see below). The oddly precise number comes from history: the first digital recorders stored their samples on video tape, and 44 100 samples a second fitted the lines of a television picture exactly. In the European system that is 3 samples of each stereo channel on each of 294 usable lines, 50 fields (half-pictures) a second: $3 \times 294 \times 50 = 44\,100$. In the American system it is $3 \times 245 \times 60$, at the 60 fields a second of black-and-white television; colour American television runs at 59.94, which gives 44 056.

Different jobs use different rates:

A traditional telephone line carries about 300 Hz to 3.4 kHz, leaving room for the filter below its 4 kHz Nyquist frequency: enough to understand speech, but music loses its sparkle. Newer wideband ("HD voice") calls carry up to 7 kHz. Studios record at higher rates for more room while they work.

Break the rule and something strange happens. A film camera samples the world, usually 24 times a second. When a wheel's spoke moves almost a full spoke-spacing between one frame and the next, the camera cannot tell that it went nearly all the way forwards; the nearest match is a small step backwards. So a fast wheel seems to turn slowly backwards.

Sound does exactly the same. Sample a 7 kHz tone at 8 kHz, far fewer than two samples per cycle, and a gentle 1 kHz wave passes through every one of the samples. From the samples alone there is no way to tell which wave was there, and playing them back gives the 1 kHz tone. This impostor frequency is an alias, and the effect is aliasing.

A tone rising steadily past the Nyquist frequency is heard to turn round and fall, even though the real tone keeps rising. Everything above the Nyquist frequency folds back down, like a reflection in a mirror at $f_s/2$; above $f_s$ the folding repeats, as the first-year part shows.

Once aliasing has happened, it cannot be undone from the samples alone. The alias sits among the real sound, and nothing done afterwards can tell them apart. So the only cure is prevention: before a signal is sampled, it passes through a low-pass filter that removes everything above the Nyquist frequency (a real filter cuts it down until any alias is too quiet to matter). This is the anti-aliasing filter, the first block of an analogue-to-digital system.

A real filter cannot cut off like a cliff edge. It needs a range of frequencies, its transition band, to go from letting sound through to blocking it. On a CD that range is the gap between 20 kHz, the top of hearing, and 22.05 kHz, the Nyquist frequency. That is what the spare room above 20 kHz is for.

That is the safe, simple rule. Strictly, the filter can keep falling until 24.1 kHz, because anything between 22.05 and 24.1 kHz folds back to somewhere above 20 kHz, out of the band we hear; the first-year part shows why. Real converters use that extra room.

In Eduqas (WJEC) A Level Electronics this is the transmitter of a pulse code modulation (PCM) system. Its blocks, in order:

1. Low-pass filter: removes frequencies above $f_s/2$ so they cannot alias.
2. Sampling gate, driven by the sampling clock: takes a sample at each clock pulse.
3. Analogue-to-digital converter (ADC): turns each sample into a binary number.
4. Parallel-in serial-out (PISO) shift register, with its own clock: sends the number's bits out one at a time.

The low-pass filter comes first for exactly the reason above. The receiver mirrors the transmitter: a Schmitt trigger, a serial-in parallel-out (SIPO) shift register with its clock, a digital-to-analogue converter (DAC), and a second low-pass filter that smooths the output.

Write a continuous tone as $x(t) = A\cos(2\pi f t + \varphi)$ and keep it only at the sample times $t = nT_s$, where the integer $n$ (…, −1, 0, 1, 2, …) counts the samples. Since $T_s = 1/f_s$, the samples are

The samples depend only on the ratio $f/f_s$, the frequency measured in cycles per sample. Now replace $f$ with $f + f_s$. The ratio gains 1, so the angle gains $2\pi n$: a whole number of complete turns, which changes nothing. So $f$, $f + f_s$, $f - f_s$, and $f + kf_s$ for any integer $k$ (positive, negative or zero), all give identical samples.

Cosine is an even function, $\cos(-\theta) = \cos\theta$, so a tone at $-f$ gives the same samples as a tone at $f$ with its phase reversed ($\varphi$ becomes $-\varphi$). Put the two facts together and $f_s - f$ matches $f$ as well, again with its phase reversed: the mirror at the Nyquist frequency. For a cosine with zero phase the samples are identical; for a sine they come out upside down.

To find the alias of any frequency, shift it by whole multiples of $f_s$ until it lands between $-f_s/2$ and $+f_s/2$, then take its size:

At $f_s = 48$ kHz, 30 kHz gives $\lvert 30 - 48 \rvert = 18$ kHz, 50 kHz gives 2 kHz, and 47 kHz gives 1 kHz. A tone at exactly 24 kHz, the Nyquist frequency, maps to itself, but it is the edge case where the samples can vanish.

Drawn as the frequency that comes out against the frequency that goes in, the rule is a straight line up to $f_s/2$, then a zigzag: down to zero at $f_s$, up to $f_s/2$ again at $1.5f_s$, and so on. Every input frequency, however high, lands somewhere between 0 and $f_s/2$.

At 44.1 kHz anything above 22.05 kHz folds back, and the fold reaches down into the band we hear: 24.1 kHz lands exactly on 20 kHz. So the anti-aliasing filter must pass everything up to 20 kHz untouched and all but silence everything above 24.1 kHz. That is a very steep filter.

Modern converters avoid building it in analogue by oversampling. They sample far faster, often 64 times faster, so that a gentle analogue filter is enough; then they filter digitally and keep one sample in every 64. Lecture 4 shows how.

A digital-to-analogue converter can first produce a staircase, holding each sample's value until the next one arrives. The sharp edges of those steps are made of frequencies above the Nyquist frequency: copies of the signal, called images, around $f_s$, $2f_s$ and so on. A low-pass filter after the converter, the reconstruction or anti-imaging filter, removes them. Holding each value also tilts the top of the band down a little, by about 3 dB at 20 kHz for a CD, so real converters correct for that in the filter or oversample until the tilt is negligible. Provided the original held nothing at or above the Nyquist frequency, what is left is then not an approximation of the original wave: in the ideal case it is the original wave, slightly delayed and apart from the rounding of each sample, which is Lecture 2's subject.

To see why, give every sample a bump shaped like the sinc function,

which is 1 at its own sample time and exactly 0 at every other sample time. Scale each bump by its sample's value and add them all up:

The sum passes through every sample and wiggles no faster than the Nyquist frequency allows. This is the Whittaker–Shannon interpolation formula, and in theory it rebuilds a band-limited signal exactly from its samples, using every sample, past and future. Real filters approximate it.

So digital audio is not steppy: below the Nyquist frequency, ideal sampling loses nothing. It also explains why more samples are not automatically better. Once the sampling rate is safely above twice the highest frequency you care about, extra samples make the filters easier to build, rather than the sound itself better.