Backchannel prediction, based on who, when and what

Backchannels are fundamental elements within conversations that serve as essential tools for effective communication and interpersonal dynamics. A typical backchannel prediction model primarily utilizes audio signal and text information. But backchanneling can exhibit different patterns depending on who I am, who I talk to, when I talk to them, and what I talk about. Therefore, we propose to employ three related pieces of information to enhance the quality of backchannel prediction models: speaker & listener characteristics, conversation progress, and topic. In our experiments with Korean counseling data, incorporating the suggested information into the model resulted in a performance improvement of 4.1% compared to the baseline model, increasing the F1 score from 50.1% to 54.2% .