Implementation Techniques for Continuous Media Systems and Applications

Brian Christopher Smith

EECS Department, University of California, Berkeley

Technical Report No. UCB/CSD-94-845

, 1994

http://www2.eecs.berkeley.edu/Pubs/TechRpts/1994/CSD-94-845.pdf

In this thesis, I investigate issues in the development of continuous media (CM) applications. CM applications process, transport, or store CM data such as digital audio and video. Introduction of video into applications will require support for sophisticated video effects such as image processing, composition and digital filtering. Traditional image processing and composition algorithms operate on uncompressed images, while video is typically stored and delivered in a compressed form. High computational cost, along with the complexity of video decompression, makes traditional algorithms too slow for interactive use on video data. This thesis describes and evaluates a new family of algorithms for computing video effects that run one to two orders of magnitude faster than their traditional counterparts. The performance increase is achieved by performing the operations directly on the compressed data. Using these algorithms, I show that simple special effects can be computed in real time on current generation workstations. In addition to developing algorithms for special effects, I describe and evaluate a toolkit for constructing distributed CM applications, called CMT, and a best effort network protocol for delivering CM data, called cyclic-UDP. CMT is an extension to the Tcl/Tk graphical user interface toolkit. It simplifies the creation and coordination of the multiple processes in a distributed CM playback system that supports delivery and synchronized playback of multiple CM streams from one or more file servers. Besides making playback applications easier to construct, the toolkit also provides a foundation on which new delivery, display, and synchronization protocols can be built and tested. CMT uses a best-effort network protocol called cyclic-UDP for delivering CM data to applications. Cyclic-UDP differs from other CM transport protocols in two ways. First, it works well in local, metropolitan, and wide area network environments, adapting itself dynamically to each new scenario. Second, it uses properties of the data being transported to reduce to effect of network congestion, both chronic and acute, to lowered fidelity at the display. For example, network congestion will cause the frame rate of a video stream to drop or decrease the effective sampling rate of an audio stream. In other protocols, congestion causes bursty frame loss where the video streams "stops" until congestion abates. Cyclic-UDP achieves improved quality by supporting frame priority which gives high priority frames a better chance of delivery than low priority frames. When used in combination with protocols specifically tuned to a particular media format, called media-specific protocols, cyclic-UDP can amortize the degradation in output quality caused by burst packet losses over a long period. In this thesis, I describe and experimentally evaluate three media-specific protocols, one for MPEG video, one for motion-JPEG video, and one for uncompressed audio. Finally, using CMT and cyclic-UDP, I show that it is possible to play good quality video using existing network and operating system technology by building a working playback application, called the CM Player.

Advisors: Lawrence A. Rowe

BibTeX citation:

@phdthesis{Smith:CSD-94-845,
Author= {Smith, Brian Christopher},
Title= {Implementation Techniques for Continuous Media Systems and Applications},
School= {EECS Department, University of California, Berkeley},
Year= {1994},
Month= {Dec},
Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/1994/5848.html},
Number= {UCB/CSD-94-845},
Abstract= {In this thesis, I investigate issues in the development of continuous media (CM) applications. CM applications process, transport, or store CM data such as digital audio and video. Introduction of video into applications will require support for sophisticated video effects such as image processing, composition and digital filtering. Traditional image processing and composition algorithms operate on uncompressed images, while video is typically stored and delivered in a compressed form. High computational cost, along with the complexity of video decompression, makes traditional algorithms too slow for interactive use on video data. This thesis describes and evaluates a new family of algorithms for computing video effects that run one to two orders of magnitude faster than their traditional counterparts. The performance increase is achieved by performing the operations directly on the compressed data. Using these algorithms, I show that simple special effects can be computed in real time on current generation workstations. In addition to developing algorithms for special effects, I describe and evaluate a toolkit for constructing distributed CM applications, called CMT, and a best effort network protocol for delivering CM data, called cyclic-UDP. CMT is an extension to the Tcl/Tk graphical user interface toolkit. It simplifies the creation and coordination of the multiple processes in a distributed CM playback system that supports delivery and synchronized playback of multiple CM streams from one or more file servers. Besides making playback applications easier to construct, the toolkit also provides a foundation on which new delivery, display, and synchronization protocols can be built and tested. CMT uses a best-effort network protocol called cyclic-UDP for delivering CM data to applications. Cyclic-UDP differs from other CM transport protocols in two ways. First, it works well in local, metropolitan, and wide area network environments, adapting itself dynamically to each new scenario. Second, it uses properties of the data being transported to reduce to effect of network congestion, both chronic and acute, to lowered fidelity at the display. For example, network congestion will cause the frame rate of a video stream to drop or decrease the effective sampling rate of an audio stream. In other protocols, congestion causes bursty frame loss where the video streams "stops" until congestion abates. Cyclic-UDP achieves improved quality by supporting frame priority which gives high priority frames a better chance of delivery than low priority frames. When used in combination with protocols specifically tuned to a particular media format, called media-specific protocols, cyclic-UDP can amortize the degradation in output quality caused by burst packet losses over a long period. In this thesis, I describe and experimentally evaluate three media-specific protocols, one for MPEG video, one for motion-JPEG video, and one for uncompressed audio. Finally, using CMT and cyclic-UDP, I show that it is possible to play good quality video using existing network and operating system technology by building a working playback application, called the CM Player.},
}

EndNote citation:

%0 Thesis
%A Smith, Brian Christopher 
%T Implementation Techniques for Continuous Media Systems and Applications
%I EECS Department, University of California, Berkeley
%D 1994
%@ UCB/CSD-94-845
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/1994/5848.html
%F Smith:CSD-94-845