Published · adjunct paper
ASAP: Auto-generating Storyboard And Previz with Virtual Humans
2021 · IEEE ISMAR-Adjunct
An early ASAP tool turns script structure into animated previews.
From the paper
Author abstract
We present a tool for Auto-generating Storyboard And Previz for screenwriters and filmmakers, called ASAP. Our system allows users to easily simulate their stories in the form of 3D animated/visual scenes with virtual humans in a virtual environment. We only ask users to write their script using Final Draft, an exclusive screenwriting tool, and upload them to our system. The uploaded script is parsed into paragraphs of the action, character, and dialogue. From those paragraphs (i.e., text data), our system uses a combination of deep learning, data-driven, and rule-based approaches to instantly generate virtual human’s physical motions and co-speech gestures, presenting natural behavior/dialogue scenes. Thus, users can observe automatically generated pre-visualized animations (i.e., previz) from the script and can create the storyboard by capturing scenes being played. Our ASAP can minimize time-and-money consuming and labor-intensive work in the early stages of filmmaking, and do it as soon as possible. We believe that our tool and approach have a good potential for wide dissemination in the film industry.
Author-written abstract · Source record
In plain language
What this work does
The ISMAR paper presents a screenplay-driven tool for screenwriters and filmmakers. It parses character, dialogue, and action paragraphs and combines behavior-generation methods to animate virtual humans; users can capture played scenes into a storyboard.
- 01Final Draft screenplay
- 02Paragraph parsing + behavior
- 03Previz + storyboard
At a glance
Method, evidence, and scope
| Input | A screenplay in Final Draft format |
|---|---|
| Output | Previsualized virtual-human animation and captured storyboards |
| Method | Screenplay parsing with learned, data-driven, and rule-based behavior modules |
| Data and scope | System scenarios in the ISMAR paper |
| Evaluation | Tool and workflow demonstration |
| Limitations | The 2024-online journal is a later extension. Its multi-output benchmarks are not results of this 2021 paper. |
Watch the system
Related ASAP system: SIGGRAPH Asia 2022 live presentation
Open on YouTube · MRLab video gallery · This video presents the related ASAP system, not this paper’s evaluation.
Implementation and artifacts
Code and setup
Independent implementation of the paper’s core ideas, with setup instructions and data preparation documented in the repository README. The institute’s original source, datasets and trained models are not distributed.
Browse code and setup guideReference this work
Citation
Hanseob Kim, Ghazanfar Ali, Jae-In Hwang. ASAP: Auto-generating Storyboard And Previz with Virtual Humans. IEEE ISMAR-Adjunct, 2021. Pages 316-320. DOI: 10.1109/ismar-adjunct54149.2021.00071.
@inproceedings{asapismar2021,
title = {{ASAP: Auto-generating Storyboard And Previz with Virtual Humans}},
author = {Kim, Hanseob and Ali, Ghazanfar and Hwang, Jae-In},
year = {2021},
booktitle = {IEEE ISMAR-Adjunct},
pages = {316-320},
doi = {10.1109/ismar-adjunct54149.2021.00071},
url = {https://ghazanfarali.com/research/asap-ismar/}
}