SBIR-STTR Award

We propose to develop practical video libraries allowing users to quickly find useful material. Commercial, home, or surveillance video programming captures information in multiple tracks: picture, sound, and closed captioning. Using multiple modalities cataloging information, visual information, and audio information is critical since no one track contains necessary information to identify clips. There is a growing market in extraction of relevant features from separate audio or video tracks of video programming. We will take advantage of these multiple modalities by developing a video library with powerful data fusion capabilities. Both analog videotape and current digital video services provide only linear access to the program material. Interactive search provides non-linear access to the library contents, reducing search time and increasing the efficiency of video library users. We propose to develop search engines allowing complex queries which take advantage of these multiple modalities. Our project will build on existing work in digital video libraries which have developed valuable technology for extracting information from individual tracks. While we may need to develop some new feature recognition algorithms for video or audio, we plan to emphasize data fusion to maximize the video library's utility and minimize time-to-market.

Built upon the Phase I project, the Phase II project will appropriately serve the purpose of producing a well defined deliverable product or process of digital video database systems based on multiple modalaties of information. The Phase II project will emphasize the commercialization of this product by adding commercial database support system security, and billing system to its functionality and making it robust, efficient, and user-friendly. The Phase II effort will also continue to perform research topics in the fields that have prominent potential such as novel algorithms for segmented video of different applications such as long shots without cuts; discovery of logical structures of video programs including making Scene Transition Graph (STG) components indexable and retrievable; indexing digital video with semantic features including motion analysis, face recognition, object identification, background recognition, speaker identification, human body counting, etc.; mythologies for incorporating annotations in video and copyright protection; using audio and closed captioning information to assist content-based image query; and using special effects, music, and sound as features for indexing and retrieval. This research effort will greatly enhance the capabilities of our digital video database system. Completion of these efforts will add a new set of tools for manipulating digital video contents, thus making it a real commercial product capable of satisfying sophisticated requirements from a different customers with different applications requirements.