hagi-bot
Overview
Abstract
This is a system submitted to the Sixth Dialogue System Live Competition. It is a task-oriented multi-modal dialogue system that integrates (1) response generation module and (2) avatar control module. Within the response generation module, we use LLM (GPT-4) to generate responses along with emotion and action labels considering the dialogue history and the topics to be discussed. Specifically, by monitoring the dialogue state through slot filling and continuously changing prompts based on the situation, the module achieves a natural dialogue progression. The avatar control module enables human-like natural behavior by operating voice (pitch, volume, and speaking speed), facial expressions, and gestures. These operations are based on predefined rules designed in reference to models such as Russell’s Circumplex Model of Affect, corresponding to the content of speech together with emotion and action labels. The combination of these two modules achieves the generation of responses appropriate to the situation and natural behaviors based on the content of utterances and emotions. This system won the first place in the preliminary round.
The dialogue system is available