Multimodal Interface for Games: A Case Study with TinyML

Multimodal interfaces go beyond the traditional interaction through keyboard and mouse by incorporating multiple modes of interaction, such as touch, voice, gesture, and even gaze, to create more intuitive and immersive user experiences. This paper investigates how TinyML can be employed for multimodal interfaces in the context of games. An endless game in which the character has to avoid obstacles and fight monsters to advance has been developed. An Arduino Nano 33 BLE Sense is then used as the input device for the game by recognizing the hand gestures of the players.