Username: Password:

.Video Information Retrieval Using Subtitles

Objectives | The team | Funding | Publications | Prototypes / Demos |


The objective of this project is the development of a Video Information Retrieval system that will operate on video libraries composed of subtitled documents. Contrasting with previous projects, we restrict ourselves to movies and television series for which subtitles are available. Distinguishing aspects of this project include information retrieval based on the simultaneous analysis of three information streams: video signal, subtitles and audio signal. The system allows visualizing videos in meaningful ways and accepts queries from the user to find portions of video documents that match the queries.

The domains of application of such a system are vast. It can be used by movie industry professionals to access and visualize scenes that share some characteristic, or to produce a concise and detailed description of a movie by answering questions like «are there violent scenes?» or «does it feature a love story?» that could be a valuable input for a recommendation system. Another domain of application of this system is contextual advertisement. The semantic analysis of scenes provides a powerful tool for placing related advertisements on video documents.

The team

The team is composed by :
  • Thibault Langlois, PhD,
  • Teresa Chambel, PhD,
  • Paula Carvalho, PhD,
  • André Falcão, PhD,
  • Eva Oliveira, PhD candidate,
  • Gonçalo Marques, PhD candidate,
  • Pedro Martins, MSc candidate,
  • Nuno Silva, MSc candidate,
  • Nuno Gil, MSc candidate,
  • Eduardo Duarte, MSc candidate,
  • Jorge Gomes, MSc candidate.

Grants / Bolsas

Various grants for undergraduate and graduate students are available. Please contact Thibault Langlois ( for details.


This project is supported by the Fundação para a Ciência e a Tecnologia (PTDC/EIA-EIA/101012/2008)


Marques G., Domingues M., Langlois T., Gouyon F. "Three Current Issues in Music Autotagging." International Society for Music Information Retrieval (ISMIR) Conference, Miami, 2011.

Teresa Chambel, Eva Oliveira, Martins and Pedro Martins, "Being Happy, Healthy and Whole Watching Movies that Affect our Emotions". In Proc. of ACII 2011 - 4th International Conference on Affective Computing and Intelligent Interaction, Memphis, TN, USA, Oct 9-12, 2011.

Pedro Martins, Thibault Langlois, and Teresa Chambel, "MovieClouds: Content-Based Overviews and Exploratory Browsing of Movies". In Proc. of Academic MindTrek'2011, Tampere, Finland, Sep 28-30, 2011.

Eva Oliveira, Mitchel Bonovoy, Nuno Ribeiro and Teresa Chambel, "Towards Emotional Interaction: Using Movies to Automatically Learn Users' Emotional States". INTERACT' 2011, 13th IFIP TC13 Conference on Human-Computer Interaction. Lisboa, Portugal, Sep 5-9, 2011.

Eva Oliveira, Pedro Martins, and Teresa Chambel, "iFelt: Accessing Movies Through Our Emotions", EuroITV'2011, 9th. European Conference on Interactive TV and Video, in cooperation with ACM SIGWEB, SIGMM & SIGCHI, pp.105-114, Lisboa, Portugal, Jun 29-Jul 1, 2011. BEST PAPER AWARD.

Eva Oliveira, "Video Access and Interaction Based on Emotions". Doctoral Consortium. EuroITV'2010, 9th. European Conference on Interactive TV and Video, in cooperation with ACM SIGWEB, SIGMM & SIGCHI, 4.pgs, Lisboa, Portugal, Jun 29-Jul 1, 2011. Nominee for Best PhD Award.

Marques G., Lopes M., Sordo M., Langlois T., Gouyon F. "Additional evidence that common low-level features of individual audio frames are not representative of music genres." Sound and Music Computing Conference, Barcelona, 2010.

Thibault Langlois, Teresa Chambel, Eva Oliveira, Paula Carvalho, Gonçalo Marques, & André Falcão. "VIRUS: Video Information Retrieval Using Subtitles". In Proc. of Academic MindTrek'2010, poster session, pp.197-200, Tampere, Finland, Oct 6-8, 2010.

Eva Oliveira, Nuno Ribeiro and Teresa Chambel, "Towards Enhanced Video Access and Recommendation through Emotions". In "Brain, Body and Bytes: Psychophysiological User Interaction" Workshop, at ACM CHI'2010, Atlanta, GA, USA, Apr 10-15, 2010.

Eva Oliveira, Teresa Chambel, and Nuno Ribeiro, "A Semantic Representation of Users Emotions when Watching Videos". In Proceedings of XATA'2010, XML Aplicaçoes e Tecnologias Associadas, 9.pgs, ESEIG, Vila do Conde, Portugal, May 19-20, 2010.

Prototypes / Demos

click on the word -> SHOW