Digital Representation
Content
Digital
Representations
Digital Video Special Effects
Fall 2006
Analog-to-Digital (A-D)
Conversion
Sampling
Quantization
Coding
Sampling -- Analog to Discrete
Analog signal to discrete-time signal
x(t) --> x[n]
Sampling procedure
∞
x[ nT ] =
∫
x ( t ) f (t − nT )dt
−∞
f(t) is the sampling function
Simple sampling
x[n] = x(t=n), i.e., f(t)=δ(t)
Reconstruction: Discrete to
Analog
Can we reconstruct analog signal from its discrete
time samples?
x[n] --> x(t) ? Generally not.
Nyquist (Shannon) sampling theorem for
bandlimited signals
If the simple sampling rate is at least twice bandwidth of
the analog signal, the analog signal can be perfectly
reconstructed:
∞
x (t ) =
∑ x ( nT )sinc
n = −∞
T
( t − nT )
Quantization -- Digitization
Discrete-time signal digital signal
Quantization error
Quantization level
Nonlinear quantization
How many bits to represent one sample?
Trade-off between error and bit rate (communication band
width)
Pre-compression and de-compression (µ law and A law)
Vector quantization
Raw Data Rate
Sampling frequency= f (Hz)
Each sample represented by R bits
Raw data rate (bit rate):
T = f x R (bits per second, or bps)
Digital Audio Signals
Frequency band of sound: human hearing
frequency range: 20Hz-20 KHz.
Sampling rate > 40 KHz (Actual sampling
rate of CD-Audio = 44.1 KHz)
Bit rate for CD quality audio signal (44.1 KHz,
Quantization:16 bits, 2 channels):
T = 44100 x 16 x 2 (bits per second, or bps)
CD quality stereo sound 10.6 MB / min
Examples
Sampling Rate Quantization
(KHz)
level (bits)
Bit Rate
(Kbps)
Telephone 8
8
64
AM Radio 16
16
256
FM Radio 22.05
16
352.8
CD Stereo 44.1
16
1411.2
DAT
48
16
1536
DVD
(Stereo)
192
24
9216
Speech Signals
Properties
Human ear: most sensitive to 600Hz-6000Hz
Quasi-stationary for around 30 ms
Characteristic maxima -- formants
Speech analysis and synthesis
Speech components, e.g., vowels and consonants
MIDI
A protocol that enables computer,
synthesizers, keyboards, and other musical
device to communicate with each other.
Bit rate: 31.25Kbps
A MIDI file stores the messages regarding
specific musical actions.
Commands, instead of actual waveforms, are
saved.
One minute of MIDI: 4KB storage.
Digital Image Representation
Picture elements (pixels)
Higher dimensional image -- voxels
Bi-level images (black/0 or white/1)
Grayscale images
1 byte/pixel: 256 gray levels
Color images
Sampling, quantization
True color: RGB 24bits/pixel
Image size, e.g. VGA 640x480
Grayscale image: 307,200 bytes
True color image: 921,600 bytes
Graphics Format
Graphics primitives and attributes
2-D objects: lines, rectangles, circles, ellipses,
text strings, etc.
Attributes: line style, line width, color, etc.
High-level representation: structured, objectbased
Low-level representation: bitmap
Computer Graphics
Computer animation
Computer Generated Images (CGI)
Photo-realistic rendering
Video Signal Requirements
Aspect ratio: TV 4/3; HDTV16/9
Luminance and chrominance
Continuity of motion > 15 frames/s
Flicker. Marginal at least 50 refresh cycles/s
TV 30 or 25 frames/s, movie 24 frames/s
Movie: 2x24=48
TV: Half picture by line-interleaving
Scanning rate: at lease 25Hz, finish one
frame in 1/25s
Color Representation in Video
RGB, normalized R=G=B=1 -- white color
YUV signal
Y=0.30R+0.59G+0.11B (Luminance)
U=(B-Y) x 0.493, V=(R-Y) x 0.877 (Chrominance channels)
Example: PAL, CD-I and DVI (Digital Video Interactive) video.
YIQ signal
Y=0.30R+0.59G+0.11B (Luminance)
I=0.60R-0.28G-0.32B, Q=0.21R-0.52G+0.31B
Example: NTSC
Avoid cross talk between luminance and colors: S-Video video
signals separate the luminance and chrominance information into
two separate analog signals.
Subsampling in Video
Different spatial sampling rates for different
chrominance channels
Human beings are more sensitive to
luminance (using more samples) while less
sensitive to colors (using less samples).
Different resolution for different components
Y:C1:C2 -- 4:2:2
Subsampling and upsampling techniques
Computer Video Format
CGA (Color Graphics Adapter): 4 colors,
320x200x2bits = 16,000 bytes
EGA: 640x350x4bits = 112,000 bytes
VGA: 640x480x8bits = 307,000 bytes
SVGA: 800x600 pixels
XGA: 1024x768 pixels
SXGA: 1280x1024 pixels
Video Quality
VCR Quality -- SIF (MPEG1)
NTSC: 240x352; PAL: 288x352 per frame
Videoconferencing quality
CIF (Common Interchange Format) -- H.261
288x352, subsampling 4:1:1(halving both
direction)
Q: what is the raw bit rate of CIF video
(30frames/s)?
QCIF (Quarter CIF)
144x176, subsampling 4:1:1(halving both direction)
Q: what is the raw bit rate of QCIF video (30frames/s)
Super-CIF:
576x704, subsampling 4:1:1(halving both direction)
The Need for Compression
Take, for example, a video signal with
resolution 320x240 and 256 (8 bits) colors,
30 frames per second
Raw bit rate = 320x240x8x30
= 18,432,000 bits
= 2,304,000 bytes = 2.3 MB
A 90 minute movie would take 2.3x60x90 MB
= 12.44 GB
Without compression, data storage and
transmission would pose serious problems!
Data Compression
Data compression requires the identification
and extraction of source redundancy.
In other words, data compression seeks to
reduce the number of bits used to store or
transmit information.
Lossless Compression
Lossless compression can recover the exact
original data after compression.
It is used mainly for compressing database
records, spreadsheets or word processing
files, where exact replication of the original is
essential.
Examples: Run Length Encoding (RLE),
Lempel Ziv Welch (LZW), Huffman Coding.
Lossy Compression
Result in a certain loss of accuracy in
exchange for a substantial increase in
compression.
More effective when used to compress
images and voice where losses outside visual
or aural perception can be tolerated.
Most lossy compression techniques can be
adjusted to different quality levels.
Example: DCT(JPEG), MPEG
Compression Ratio
Compression ratio
original data size
------------------------- :
compressed data size
1
Representations
Digital Video Special Effects
Fall 2006
Analog-to-Digital (A-D)
Conversion
Sampling
Quantization
Coding
Sampling -- Analog to Discrete
Analog signal to discrete-time signal
x(t) --> x[n]
Sampling procedure
∞
x[ nT ] =
∫
x ( t ) f (t − nT )dt
−∞
f(t) is the sampling function
Simple sampling
x[n] = x(t=n), i.e., f(t)=δ(t)
Reconstruction: Discrete to
Analog
Can we reconstruct analog signal from its discrete
time samples?
x[n] --> x(t) ? Generally not.
Nyquist (Shannon) sampling theorem for
bandlimited signals
If the simple sampling rate is at least twice bandwidth of
the analog signal, the analog signal can be perfectly
reconstructed:
∞
x (t ) =
∑ x ( nT )sinc
n = −∞
T
( t − nT )
Quantization -- Digitization
Discrete-time signal digital signal
Quantization error
Quantization level
Nonlinear quantization
How many bits to represent one sample?
Trade-off between error and bit rate (communication band
width)
Pre-compression and de-compression (µ law and A law)
Vector quantization
Raw Data Rate
Sampling frequency= f (Hz)
Each sample represented by R bits
Raw data rate (bit rate):
T = f x R (bits per second, or bps)
Digital Audio Signals
Frequency band of sound: human hearing
frequency range: 20Hz-20 KHz.
Sampling rate > 40 KHz (Actual sampling
rate of CD-Audio = 44.1 KHz)
Bit rate for CD quality audio signal (44.1 KHz,
Quantization:16 bits, 2 channels):
T = 44100 x 16 x 2 (bits per second, or bps)
CD quality stereo sound 10.6 MB / min
Examples
Sampling Rate Quantization
(KHz)
level (bits)
Bit Rate
(Kbps)
Telephone 8
8
64
AM Radio 16
16
256
FM Radio 22.05
16
352.8
CD Stereo 44.1
16
1411.2
DAT
48
16
1536
DVD
(Stereo)
192
24
9216
Speech Signals
Properties
Human ear: most sensitive to 600Hz-6000Hz
Quasi-stationary for around 30 ms
Characteristic maxima -- formants
Speech analysis and synthesis
Speech components, e.g., vowels and consonants
MIDI
A protocol that enables computer,
synthesizers, keyboards, and other musical
device to communicate with each other.
Bit rate: 31.25Kbps
A MIDI file stores the messages regarding
specific musical actions.
Commands, instead of actual waveforms, are
saved.
One minute of MIDI: 4KB storage.
Digital Image Representation
Picture elements (pixels)
Higher dimensional image -- voxels
Bi-level images (black/0 or white/1)
Grayscale images
1 byte/pixel: 256 gray levels
Color images
Sampling, quantization
True color: RGB 24bits/pixel
Image size, e.g. VGA 640x480
Grayscale image: 307,200 bytes
True color image: 921,600 bytes
Graphics Format
Graphics primitives and attributes
2-D objects: lines, rectangles, circles, ellipses,
text strings, etc.
Attributes: line style, line width, color, etc.
High-level representation: structured, objectbased
Low-level representation: bitmap
Computer Graphics
Computer animation
Computer Generated Images (CGI)
Photo-realistic rendering
Video Signal Requirements
Aspect ratio: TV 4/3; HDTV16/9
Luminance and chrominance
Continuity of motion > 15 frames/s
Flicker. Marginal at least 50 refresh cycles/s
TV 30 or 25 frames/s, movie 24 frames/s
Movie: 2x24=48
TV: Half picture by line-interleaving
Scanning rate: at lease 25Hz, finish one
frame in 1/25s
Color Representation in Video
RGB, normalized R=G=B=1 -- white color
YUV signal
Y=0.30R+0.59G+0.11B (Luminance)
U=(B-Y) x 0.493, V=(R-Y) x 0.877 (Chrominance channels)
Example: PAL, CD-I and DVI (Digital Video Interactive) video.
YIQ signal
Y=0.30R+0.59G+0.11B (Luminance)
I=0.60R-0.28G-0.32B, Q=0.21R-0.52G+0.31B
Example: NTSC
Avoid cross talk between luminance and colors: S-Video video
signals separate the luminance and chrominance information into
two separate analog signals.
Subsampling in Video
Different spatial sampling rates for different
chrominance channels
Human beings are more sensitive to
luminance (using more samples) while less
sensitive to colors (using less samples).
Different resolution for different components
Y:C1:C2 -- 4:2:2
Subsampling and upsampling techniques
Computer Video Format
CGA (Color Graphics Adapter): 4 colors,
320x200x2bits = 16,000 bytes
EGA: 640x350x4bits = 112,000 bytes
VGA: 640x480x8bits = 307,000 bytes
SVGA: 800x600 pixels
XGA: 1024x768 pixels
SXGA: 1280x1024 pixels
Video Quality
VCR Quality -- SIF (MPEG1)
NTSC: 240x352; PAL: 288x352 per frame
Videoconferencing quality
CIF (Common Interchange Format) -- H.261
288x352, subsampling 4:1:1(halving both
direction)
Q: what is the raw bit rate of CIF video
(30frames/s)?
QCIF (Quarter CIF)
144x176, subsampling 4:1:1(halving both direction)
Q: what is the raw bit rate of QCIF video (30frames/s)
Super-CIF:
576x704, subsampling 4:1:1(halving both direction)
The Need for Compression
Take, for example, a video signal with
resolution 320x240 and 256 (8 bits) colors,
30 frames per second
Raw bit rate = 320x240x8x30
= 18,432,000 bits
= 2,304,000 bytes = 2.3 MB
A 90 minute movie would take 2.3x60x90 MB
= 12.44 GB
Without compression, data storage and
transmission would pose serious problems!
Data Compression
Data compression requires the identification
and extraction of source redundancy.
In other words, data compression seeks to
reduce the number of bits used to store or
transmit information.
Lossless Compression
Lossless compression can recover the exact
original data after compression.
It is used mainly for compressing database
records, spreadsheets or word processing
files, where exact replication of the original is
essential.
Examples: Run Length Encoding (RLE),
Lempel Ziv Welch (LZW), Huffman Coding.
Lossy Compression
Result in a certain loss of accuracy in
exchange for a substantial increase in
compression.
More effective when used to compress
images and voice where losses outside visual
or aural perception can be tolerated.
Most lossy compression techniques can be
adjusted to different quality levels.
Example: DCT(JPEG), MPEG
Compression Ratio
Compression ratio
original data size
------------------------- :
compressed data size
1
