Scientists at the University of California at Viterbi, led by Professor Shrikant Narayanan, conducted a study to objectively examine the impact of music on cinematic genres. The goal of the work is to determine whether artificial intelligence-based technologies can predict the genre of a film based on the soundtrack alone.
In their research, the group examined a dataset of 110 popular films released between 2014 and 2019. They used the genre classification listed in the Internet Movie Database (IMDb) to designate each film as action, comedy, drama, horror, or melodrama.
They then applied a deep learning network that extracted auditory information such as timbre, harmony, melody, rhythm, and tone from the music and score of each movie. She used machine learning to analyze these musical characteristics and was able to accurately classify the genres of each film based on those characteristics alone.
Scholars have also interpreted these patterns to determine which musical characteristics most indicate differences between genres. The models did not indicate which types of notes or instruments were associated with each genre, but they found that tonal and timbre characteristics are most important in predicting a film genre.
It is intuitively clear that certain musical elements are used in the soundtracks of different film genres. Rom-coms feature-rich strings and lyrical melodies, while horror films are full of disturbing, shrill frequencies and frighteningly discordant notes.
However, scientists set out to find quantitative evidence that elements of a film’s soundtrack can be used to characterize the film’s genre. They were the first to apply deep learning models to a soundtrack to see if a computer can predict a movie genre based on music alone. It turned out that AI can indeed accurately classify the genre of a movie using machine learning. The study confirmed the scientists’ assumption that the soundtrack of the film strongly influences the way people perceive different pictures.