Published · journal article
Enhancing doctor‐patient communication in surgical explanations: Designing effective facial expressions and gestures for animated physician characters
2024 · Computer Animation and Virtual Worlds
Expression and gesture support surgical explanations by animated physicians.
From the paper
Author abstract
Paying close attention to facial expressions, gestures, and communication techniques is essential when creating animated physician characters that are realistic and captivating when describing surgical procedures. This paper emphasizes the integration of appropriate emotions, co-speech gestures when medical experts explain the medical procedure, and designing animated characters. We can achieve healthy doctor-patient relationships and improvement of patients’ understanding by depicting these components truthfully. We suggest two critical approaches to developing virtual medical experts by incorporating these elements. First, doctors can generate the contents of the surgical procedure with a virtual doctor. Second, patients can listen to the surgical procedure described by the virtual doctor and ask if they have any questions. Our system helps patients by considering their psychology and adding medical professionals’ opinions. These improvements ensure the animated virtual agent is comforting, reassuring, and emotionally supportive. Through a user study, we evaluated our hypothesis and gained insight into improvements.
Author-written abstract from the author manuscript.
In plain language
What this work does
The system lets clinicians prepare explanations with a virtual physician and lets patients ask follow-up questions. It combines grounded medical information with designed facial expression and co-speech gesture, and evaluates how users perceive the presentation.
- 01Surgical content / question
- 02Grounded answer + behavior
- 03Animated explanation
At a glance
Method, evidence, and scope
| Input | Clinician-prepared surgical content and patient questions |
|---|---|
| Output | Animated surgical explanations and grounded follow-up responses |
| Method | Content preparation, grounded question answering, facial expression, and gesture |
| Data and scope | Surgical-explanation materials and study conditions described in the paper |
| Evaluation | 113-participant study; reported answer F1 comparison 0.492 to 0.779 |
| Limitations | This is communication research. Perceived social presence and answer scores do not demonstrate improved clinical outcomes or autonomous medical decision-making. |
Implementation and artifacts
Code and setup
Independent implementation of the paper’s core ideas, with setup instructions and data preparation documented in the repository README. The institute’s original source, datasets and trained models are not distributed.
Browse code and setup guideReference this work
Citation
Hwang Youn Kim, Ghazanfar Ali, Jae‐In Hwang. Enhancing doctor‐patient communication in surgical explanations: Designing effective facial expressions and gestures for animated physician characters. Computer Animation and Virtual Worlds, 2024. Volume 35. Issue 3. Article e2236. DOI: 10.1002/cav.2236.
@article{virtualphysician2024,
title = {{Enhancing doctor‐patient communication in surgical explanations: Designing effective facial expressions and gestures for animated physician characters}},
author = {Kim, Hwang Youn and Ali, Ghazanfar and Hwang, Jae‐In},
year = {2024},
journal = {Computer Animation and Virtual Worlds},
volume = {35},
number = {3},
eid = {e2236},
doi = {10.1002/cav.2236},
url = {https://ghazanfarali.com/research/virtual-physician/}
}