Automatic Indexing of Virtual Camera Features from Japanese Anime

The position, orientation, and the distance of the camera in relation to the subject(s) in a movie scene, namely camera level, camera angle, and shot scale, are essential features in the film-making process due to their influence on the viewer’s perception of the scene. Since animation techniques exploit drawings or computer graphics objects for making films instead of camera shooting, the automatic understanding of such “virtual camera” features appears harder if compared to live-action movies. Exploiting a dataset of animated movies from popular directors such as Hayao Miyazaki, Hideaki Anno and Mamoru Oshii, we finetune pre-trained convolutional neural networks and use One Cycle Learning Rate to reach convergence in a limited number of epochs. While some difficulties are revealed and discussed in classifying complex features, like camera angle and camera level, showing F1 scores of respectively 61% and 68%, the classification of the shot scale reaches a score of about 80%, which is comparable with state-of-the-art methods applied on live-action movies. The developed models will be useful in conducting automated movie annotation for a wide range of applications, such as in stylistic analysis, video recommendation, and studies in media psychology. The database and a demo of the classifiers are available at https://cinescale.github.io/anime