Low Level Processing of Audio and Video Information for Extracting the Semantics of Content

The problem of semantic indexing of multimedia documents is actually of great interest due to the wide diffusion of large audio-video databases. We first briefly describe some techniques used to extract low-level features (e.g., shot change detection, dominant color extraction, audio classification etc.). Then the ToCAI (table of contents and analytical index) framework for content description of multimedia material is presented, together with an application which implements it. Finally we propose two algorithms suitable for extracting the high level semantics of a multimedia document. The first is based on finite-state machines and low-level motion indices, whereas the second uses hidden Markov models.