Quantization and data compression

Quantization plays a major part in lossy data compression. In many cases, quantization
can be viewed as the fundamental element that distinguishes lossy data compression from
lossless data compression, and the use of quantization is nearly always motivated by the
need to reduce the amount of data needed to represent a signal. In some compression
schemes, like MP3 or Vorbis, compression is also achieved by selectively discarding
some data, an action that can be analyzed as a quantization process (e.g., a vector
quantization process) or can be considered a different kind of lossy process.


One example of a lossy compression scheme that uses quantization is JPEG image
compression. During JPEG encoding, the data representing an image (typically 8-bits for
each of three color components per pixel) is processed using a discrete cosine transform
and is then quantized and entropy coded. By reducing the precision of the transformed
values using quantization, the number of bits needed to represent the image can be
reduced substantially. For example, images can often be represented with acceptable
quality using JPEG at less than 3 bits per pixel (as opposed to the typical 24 bits per pixel
needed prior to JPEG compression). Even the original representation using 24 bits per
pixel requires quantization for its PCM sampling structure.


In order to determine how many bits are necessary to effect a given precision, logarithms
are used. Suppose, for example, that it is necessary to record six significant digits, that is
to say, millionths. The number of values that can be expressed by N bits is equal to two to
the Nth power. To express six decimal digits, the required number of bits is determined
by rounding (6 / log 2)—where log refers to the base ten, or common, logarithm—up to
the nearest integer. Since the logarithm of 2, base ten, is approximately 0.30102, the
required number of bits is then given by (6 / 0.30102), or 19.932, rounded up to the
nearest integer, viz., 20 bits.


This type of quantization—where a set of binary digits, e.g., an arithmetic register in a
CPU, are used to represent a quantity—is called Vernier quantization. It is also possible,
although rather less efficient, to rely upon equally spaced quantization levels. This is only
practical when a small range of values is expected to be captured: for example, a set of
eight possible values requires eight equally spaced quantization levels—which is not
unreasonable, although obviously less efficient than a mere trio of binary digits (bits)—
but a set of, say, sixty-four possible values, requiring sixty-four equally spaced
quantization levels, can be expressed using only six bits, which is obviously far more
efficient.