The objective of this project is the development of a Video Information Retrieval system that will operate on video libraries composed of subtitled documents. Contrasting with previous projects, we restrict ourselves to movies and television series for which subtitles are available. Distinguishing aspects of this project include information retrieval based on the simultaneous analysis of three information streams: video signal, subtitles and audio signal. The system allows visualizing videos in meaningful ways and accepts queries from the user to find portions of video documents that match the queries.
The domains of application of such a system are vast. It can be used by movie industry professionals to access and visualize scenes that share some characteristic, or to produce a concise and detailed description of a movie by answering questions like «are there violent scenes?» or «does it feature a love story?» that could be a valuable input for a recommendation system. Another domain of application of this system is contextual advertisement. The semantic analysis of scenes provides a powerful tool for placing related advertisements on video documents.