Top-Down and Bottom-Up Semantic Indexing of Multimedia

The aim of this work consists in proposing a dual approach for the sake of semantic indexing of audio-visual documents. We present two dierent algorithms based respectively on a bottom-up and a top-down strategy. Considering the top-down approach, we propose an algorithm which implements a nite-state machine and uses low-level motion indices extracted from an MPEG compressed bit-stream. Simulation results show that the proposed method can eectively detect the presence of relevant events in sport programs. Using the bottom-up approach, the indexing is performed by means of Hidden Markov Models (HMM), with an innovative approach: the input signal is considered as a non-stationary stochastic process, modeled by a HMM in which each state is associated with a different property of audio-visual material. Several samples from the MPEG-7 content set have been analyzed using the proposed scheme, demonstrating the performance of the overall approach to provide insights about the content of audio-visual programmes. Moreover, what appears quite attractive instead is to use low-level descriptors in providing a feedback for non-expert users of the content of the described audio-visual programme. The experiments have demonstrated that, by adequate visualization or presentation, low-level features carry instantly semantic information about the programme content, given a certain programme category, which may thus help the viewer to use such low-level information for navigation or retrieval of relevant events.