No-code Digital Human for Conversational Behavior

Hanseob Kim; Jieun Kim; Ghazanfar Ali; Jae-In Hwang

doi:10.1145/3550082.3564175

Published · poster

No-code Digital Human for Conversational Behavior

Hanseob Kim · Jieun Kim · Ghazanfar Ali · Jae-In Hwang

2022 · SIGGRAPH Asia Posters

No-code conversation flows coordinate digital-human behavior.

Publisher / DOI Download BibTeX Code repository

Research illustration

From the paper

Author abstract

In this poster, we present Flow Human, a no-code system that generates conversational behavior of digital humans from the text. Our users only need to build a conversation flow they want to talk to customers using the flow-based authoring tool we developed. Our system then automatically generates the verbal and non-verbal behavior of digital humans along the conversation flow, interacts with customers, and collects feedback. We believe that this work can serve the potential to be distributed to various services that have not been introduced because of the challenging task of controlling multiple factors in digital humans (e.g., conversation flow, co-speech gestures, and facial animation).

Author-written abstract from the author manuscript.

In plain language

What this work does

Flow Human lets service designers create a conversation flow using an authoring tool. The system turns that flow into verbal and nonverbal behavior, presents a digital human in a kiosk interaction, and collects user feedback.

01Conversation flow
02Behavior coordination
03Digital-human interaction

At a glance

Method, evidence, and scope

Method diagram from Figure 2 of the flow-human paper — Method diagram from the paper · Figure 2, PDF page 2. View full size

Method and evidence for Flow Human
Input	An authored conversation flow
Output	Speech, facial animation, co-speech gesture, and feedback collection
Method	Flow-based authoring and coordinated digital-human behavior
Data and scope	Kiosk use case described in the poster
Evaluation	System and use-case demonstration in a two-page poster
Limitations	Behavior depends on the authored flow and available modules; this poster does not establish unrestricted dialogue competence.

Implementation and artifacts

Code and setup

Independent implementation of the paper’s core ideas, with setup instructions and data preparation documented in the repository README. The institute’s original source, datasets and trained models are not distributed.

Browse code and setup guide

Reference this work

Citation

Hanseob Kim, Jieun Kim, Ghazanfar Ali, Jae-In Hwang. No-code Digital Human for Conversational Behavior. SIGGRAPH Asia Posters, 2022. Pages 1-2. DOI: 10.1145/3550082.3564175.

Download BibTeX

@inproceedings{flowhuman2022,
  title = {{No-code Digital Human for Conversational Behavior}},
  author = {Kim, Hanseob and Kim, Jieun and Ali, Ghazanfar and Hwang, Jae-In},
  year = {2022},
  booktitle = {SIGGRAPH Asia Posters},
  pages = {1-2},
  doi = {10.1145/3550082.3564175},
  url = {https://ghazanfarali.com/research/flow-human/}
}