Audio-Visual Pattern Recognition using HMM for Content-Based Multimedia Indexing

The aim of this work consists in the development of automatic techniques for the extraction of content-based information from audiovisual data. The focus has been placed on providing tools for analyzing both audio and visual streams, for translating the signal samples into sequences of indices. The signal classification are performed by means of Hidden Markov Models (HMM), used in an innovative approach: the input signal is considered as a non-stationary stochastic process, modeled by a HMM in which each state stands for a different class of the signal. This defines an adaptive classification scheme for which a set of new training algorithms has been developed. Several samples from the MPEG-7 content set have been analyzed using the proposed classification scheme, demonstrating the performance of the overall approach to provide insights of the content of the audio-visual material.