Real-Time MIR Algorithms for Music-Reactive Game World Generation

Abstract

In music education, an important learning objective is to acquire a sense of rhythm and melody, e.g., by tapping or singing along to a musical performance. In this context, music games are becoming increasingly popular due to their accessibility and interactivity. However, existing games are often tailored to a specific musical repertoire. In this paper, we will explore the potential of music information retrieval (MIR) algorithms for generating dynamic game worlds that are reactive to musical input in real-time, e.g., to music from a stereo system or a live performance. Such games may be used to explore musical performances in an interactive and playful way. As a case study, we therefore developed a jump-and- run game prototype that challenges players to tap along the beat of the music in their environment. Its music-reactive game world relies on past and predicted beat positions estimated by a real-time beat tracking algorithm.

BibTeX

@inproceedings{mci/Meier2022,
  author = {Meier, Peter AND Schwär, Simon AND Rosenzweig, Sebastian AND Müller, Meinard},
  title = {Real-Time MIR Algorithms for Music-Reactive Game World Generation},
  booktitle = {Mensch und Computer 2022 - Workshopband},
  year = {2022},
  editor = {Marky, Karola AND Grünefeld, Uwe AND Kosch, Thomas},
  doi = {10.18420/muc2022-mci-ws03-225},
  publisher = {Gesellschaft für Informatik e.V.},
  address = {Bonn}
}