Assistant Professor · AI Researcher · End-to-End Builder
I make machines behave.
Language is only one channel. I build intelligent systems that can speak, move, remember, adapt—and survive contact with real people.
outputs
participated in
demos
in OmniMo
00 / Thesis
Intelligence is not the model.
It is the whole encounter.
I work from corpus to client: motion data, representations, retrieval and generation, APIs, real-time animation, embodied interfaces, and the studies that decide whether any of it matters.
The through-line is simple: make intelligent behavior scalable, controllable, grounded, and measurable.
01 / Now
The live research ecosystem.
Connected projects exploring how virtual humans perceive, speak, move and interact—and the data, tools and evaluation methods that support them.

Buraq
A multimodal agent runtime that coordinates speech recognition, language generation, voice, facial animation and gesture. It connects models and interactive clients through shared interfaces to support responsive embodied conversations.
A multimodal agent runtime that coordinates speech recognition, language generation, voice, facial animation and gesture. It connects models and interactive clients through shared interfaces to support responsive embodied conversations.

Embodied Studio
A modular toolkit for assembling interactive virtual humans from motion processing, behavior generation, agent runtimes and rendering clients. It brings independent components—including Buraq—together to make research systems easier to integrate, reuse and deploy.
A modular toolkit for assembling interactive virtual humans from motion processing, behavior generation, agent runtimes and rendering clients. It brings independent components—including Buraq—together to make research systems easier to integrate, reuse and deploy.

OmniMo
Motion-data infrastructure that converts diverse datasets, skeletons and file formats into a shared representation with traceable provenance. It reduces repeated preparation work and supports consistent training, evaluation and animation across motion projects.
Motion-data infrastructure that converts diverse datasets, skeletons and file formats into a shared representation with traceable provenance. It reduces repeated preparation work and supports consistent training, evaluation and animation across motion projects.

Co-Speech Gesture Atlas
An open research atlas connecting co-speech gesture-generation methods with human gesture theory, datasets, evaluation metrics and application evidence. Structured comparisons and citation relationships help researchers trace ideas, compare approaches and find the resources needed to build on prior work.
An open research atlas connecting co-speech gesture-generation methods with human gesture theory, datasets, evaluation metrics and application evidence. Structured comparisons and citation relationships help researchers trace ideas, compare approaches and find the resources needed to build on prior work.

Full-Duplex Voice Agent
A framework for generating coordinated speaking and listening behavior during continuous two-way conversation. It connects body motion with turn-taking, backchannels, interruptions and pauses to support agents that participate while another person is speaking.
A framework for generating coordinated speaking and listening behavior during continuous two-way conversation. It connects body motion with turn-taking, backchannels, interruptions and pauses to support agents that participate while another person is speaking.

MotionBridge
A framework for converting ordinary monocular video into complete, clean and retargetable human motion. It connects visual observations with learned motion representations to make motion acquisition more accessible and economical.
A framework for converting ordinary monocular video into complete, clean and retargetable human motion. It connects visual observations with learned motion representations to make motion acquisition more accessible and economical.

MotionStage
A synthetic-data platform that renders skeletal motion into video alongside corresponding joint positions, camera geometry, segmentation and visibility information. It supplies controlled observations and known ground truth for learning and evaluating motion-reconstruction methods.
A synthetic-data platform that renders skeletal motion into video alongside corresponding joint positions, camera geometry, segmentation and visibility information. It supplies controlled observations and known ground truth for learning and evaluating motion-reconstruction methods.

What Does a Motion-Hour Cost?
A review of the methods and practical costs involved in turning video and motion recordings into usable research corpora. It examines compute, labor, failures, licensing and reporting practices to clarify the resources needed for accessible motion research.
A review of the methods and practical costs involved in turning video and motion recordings into usable research corpora. It examines compute, labor, failures, licensing and reporting practices to clarify the resources needed for accessible motion research.

Motion EvalSuite
A reusable evaluation framework for comparing motion-generation methods and baselines through consistent data interfaces and measurement protocols. It brings motion quality, semantic relevance, diversity and computational cost into a common evaluation workflow.
A reusable evaluation framework for comparing motion-generation methods and baselines through consistent data interfaces and measurement protocols. It brings motion quality, semantic relevance, diversity and computational cost into a common evaluation workflow.

Gesture Benchmark
Research into evaluation methods that capture the perceptual quality, meaning and behavioral appropriateness of co-speech gestures. It examines whether automatic measurements agree with human judgments and remain reliable across datasets and recording conditions.
Research into evaluation methods that capture the perceptual quality, meaning and behavioral appropriateness of co-speech gestures. It examines whether automatic measurements agree with human judgments and remain reliable across datasets and recording conditions.

ControllableGesture
A text-conditioned gesture framework with interpretable controls over movement characteristics such as handedness, height, amplitude, speed and emotional expression. It gives authors direct influence over a virtual human’s delivery while preserving the relationship between speech meaning and motion.
A text-conditioned gesture framework with interpretable controls over movement characteristics such as handedness, height, amplitude, speed and emotional expression. It gives authors direct influence over a virtual human’s delivery while preserving the relationship between speech meaning and motion.

Behavioral Imposter Test
A research program examining how consistency between a digital persona’s identity and behavior shapes human perception. Controlled comparisons of behavioral cues help explain when a virtual character feels believable, appropriate and consistent with its claimed identity.
A research program examining how consistency between a digital persona’s identity and behavior shapes human perception. Controlled comparisons of behavioral cues help explain when a virtual character feels believable, appropriate and consistent with its claimed identity.

MotionMix
A shared motion representation for transferring movement between characters with different skeletal structures and proportions. It aims to preserve the meaning and character of a performance while reducing dependence on rig-specific retargeting procedures.
A shared motion representation for transferring movement between characters with different skeletal structures and proportions. It aims to preserve the meaning and character of a performance while reducing dependence on rig-specific retargeting procedures.

Cadence
A gesture framework that adapts movement timing and gesture phases to synthesized speech using reference examples. It addresses changes in voice, speaking rate and delivery so that meaningful movements remain coordinated with the spoken message.
A gesture framework that adapts movement timing and gesture phases to synthesized speech using reference examples. It addresses changes in voice, speaking rate and delivery so that meaningful movements remain coordinated with the spoken message.

SpeechTrace
An investigation of speech-recognition representations as conditioning signals for co-speech gesture generation. It examines how linguistic content and acoustic information contribute to movement, seeking representations that preserve meaning while supporting expressive delivery.
An investigation of speech-recognition representations as conditioning signals for co-speech gesture generation. It examines how linguistic content and acoustic information contribute to movement, seeking representations that preserve meaning while supporting expressive delivery.

Relational Gesture Language
A structured language for describing relationships among the hands, body, movement and speech, coupled with a description-to-motion framework. It makes gesture intent more explicit so that motion can be generated, inspected and controlled through understandable descriptions.
A structured language for describing relationships among the hands, body, movement and speech, coupled with a description-to-motion framework. It makes gesture intent more explicit so that motion can be generated, inspected and controlled through understandable descriptions.

Multi-Task Transfer for Motion — FATM
A motion-learning framework that combines gesture and action-language tasks through a shared token-based model. It investigates whether knowledge transferred between tasks improves the semantic relevance and generalization of generated motion.
A motion-learning framework that combines gesture and action-language tasks through a shared token-based model. It investigates whether knowledge transferred between tasks improves the semantic relevance and generalization of generated motion.

VoiceWeave
A framework that derives gesture generation from representations inside speech-synthesis models. By sharing information between voice and movement generation, it investigates how embodied delivery can follow speech content and prosody more closely.
A framework that derives gesture generation from representations inside speech-synthesis models. By sharing information between voice and movement generation, it investigates how embodied delivery can follow speech content and prosody more closely.

MotionSignature
A text-and-motion retrieval framework that selects recorded gestures using speech meaning and reference examples of a speaker’s movement style. It aims to combine captured-motion realism with responsive, personalized behavior for unfamiliar speakers.
A text-and-motion retrieval framework that selects recorded gestures using speech meaning and reference examples of a speaker’s movement style. It aims to combine captured-motion realism with responsive, personalized behavior for unfamiliar speakers.

Expressive Upper-Face Animation
A lightweight framework that combines speech-conditioned emotional expression with subtle liveness movements around the eyes and eyebrows. It complements independent lip synchronization to make virtual agents more expressive and lifelike during conversation.
A lightweight framework that combines speech-conditioned emotional expression with subtle liveness movements around the eyes and eyebrows. It complements independent lip synchronization to make virtual agents more expressive and lifelike during conversation.
02 / Publications
Seventeen outputs.
Thirteen visual ideas.
Journal papers, conference papers, adjunct items, posters, and a live demonstration are grouped by the system they actually describe. Each output has a dedicated page with its abstract, method, citation, and available external resources.

2025 · IEEE ISMAR-Adjunct
LUFA: lightweight upper-face animation
Aligned voice and text representations retrieve facial motion for VR and MR avatars.
2025 · IEEE Access · First author
Multilingual motion retrieval + GestureCLR
High-noise 2D pose aligned to 2,035 clean gesture units; cross-lingual interaction evaluated with 51 people.Expanding Multilingual Co-Speech Interaction scales a gesture library from 56 to 2,035 units and a mapping base from about 84,000 to 210,000 rules. GestureCLR raises reported high-noise matching from 18% to 97%, while translation does not meaningfully degrade the user experience in a 51-participant study.

2025 · CAVW · First author
RIDGE
High-confidence captured rules when possible; contrastive learning when necessary.RIDGE combines an LLM-assisted high-fidelity rule base with a learned fallback in a shared text–motion space. It introduces Gesture Cluster Affinity for semantic coherence and reports GCA 0.73, versus approximately 0.61 and 0.52 for compared paths.

2025 · CAVW + ISMAR-Adjunct
Joseon Dynasty RAG agent
Dynamic article chunking, date-aware retrieval, source grounding, and an embodied voice-and-motion interface.The system grounds an embodied agent in 49.6 million characters of high-density chronological records. It preserves article boundaries, regenerates vague queries, filters by date, maximizes evidence within the token budget, and cites sources.
Read Joseon RAG journal paper page · Read Joseon RAG adjunct paper page

2025 journal · 2021 adjunct · 2022 live
ASAP: screenplay → many visual worlds
Storyboards, animated 3D previz, and immersive scenes generated from a screenplay.ASAP parses screenplay structure into characters, dialogue, actions and emotion, then composes storyboards, animated previsualization and immersive scenes. The journal reports 93% top-1 accuracy on simple action sentences and 87% on complex ones; Ghazanfar Ali and Hanseob Kim are marked as equal contributors.

2024 · CAVW · CASA Best Paper
Virtual physician
Grounded medical answers carried by carefully designed expression, gesture, and social presence.This dual-mode system helps clinicians prepare surgical explanations and lets patients ask grounded follow-up questions through an animated physician. The RAG answer F1 improves from 0.492 to 0.779 in the reported comparison; a 113-participant study finds higher social presence than text-and-audio slides.

2021 · Applied Sciences
Silhouettes make mobile AR believable
Lightweight real-object geometry gives virtual humans occlusion and physical context.A compact segmentation model turns real objects into silhouette geometry so a mobile AR character can walk behind, stand on, and interact around the physical scene. The work reports a 2.4M-parameter model, 0.971 mIoU and a 24-participant study.

2020 · CAVW · First author
Automatic text-to-gesture rules
Mine useful behavior from 106 hours of public video instead of asking experts to author every map.This work automates rule-based co-speech mapping from about 106 hours of public video using 2D pose, sliding-window matching, and semantic word embeddings at runtime. Evaluation finds performance comparable to manual maps with more gesture variety and complementary effects when the maps are combined.

2026 · ECCV
Through Van Gogh’s Eyes
Global artist-level diffusion style transfer designed to learn a distribution, not repeat one iconic exemplar.The work addresses stylistic bias in diffusion models by representing an artist through a broader book of genuine artworks and transferring global artistic characteristics instead of overfitting to a narrow familiar motif. Ghazanfar Ali is a co-author; the portfolio does not invent a specific module ownership.

2022 · SIGGRAPH Asia Posters · First author
Wild pose matching with GestureCLR
Robust 2D-to-3D matching expands the gesture bank to 2,035 units and 210,000 rules.This poster introduces contrastive alignment between noisy 2D pose extracted from public monologue video and clean 3D gesture units, followed by clustering for runtime variety. It is the bridge between early automatic mapping and the later multilingual system.

2022 · SIGGRAPH Asia Posters
Flow Human
No-code conversation flows become verbal and nonverbal digital-human behavior.Flow Human lets service designers author a conversation flow while the system automatically coordinates speech, facial animation, co-speech gesture, interaction and feedback collection for an embodied kiosk experience.

2021 · IEEE VR Workshops
Context → virtual-human action
Joint sentence and entity understanding turns conversation into grounded room behavior.A BERT-based system distinguishes conversation from action and extracts the entities needed to execute flexible behavior directly from natural language. The pilot investigates whether a virtual human can move beyond fixed phrase activation.

2019 · CASA · First author
Wearable mixed-reality agent
The foundational system: place-aware, multimodal, embodied, and designed for real cultural spaces.This early framework unifies spatial mapping, gaze, speech recognition, a domain chatbot, object recognition, animation, emotion and gesture for virtual guides in museums and botanical gardens. It establishes the system-level research arc that later becomes Buraq.
03 / Record
The complete output ledger.
7 journal articles and 9 conference, adjunct, poster, or live-demonstration items. Labels matter; not every item is presented as a full paper.
Browse all paper pages and the doctoral thesis ↗
- Conference · ECCVThrough Van Gogh’s Eyes: Global Style Transfer with Diffusion ModelJeongha Lee, Yujin Kim, Ghazanfar Ali, Suhyun Kim, Jae-In HwangPAGE
- Journal · IEEE Access · First authorExpanding Multilingual Co-Speech Interaction: The Impact of Enhanced Gesture Units in Text-to-Gesture Synthesis for Digital HumansGhazanfar Ali, Woojoo Kim, Muhammad Shahid Anwar, Jae-In Hwang, Ahyoung ChoiPAGE
- Journal · Computer Animation and Virtual Worlds · First authorRIDGE: Rule‐Infused Deep Learning for Realistic Co‐Speech Gesture GenerationGhazanfar Ali, HwangYoun Kim, Jae‐In HwangPAGE
- Journal · Computer Animation and Virtual WorldsA Retrieval‐Augmented Generation System for Accurate and Contextual Historical Analysis: AI‐Agent for the Annals of the Joseon DynastyJeong Ha Lee, Ghazanfar Ali, Jae‐In HwangPAGE
- Journal · Multimedia Tools and Applications · Equal contributionASAP for multi-outputs: auto-generating storyboard and pre-visualization with virtual actors based on screenplayHanseob Kim, Ghazanfar Ali, Bin Han, Hwang Youn Kim, Jieun Kim, Hyemin Shin, Gerard Jounghyun Kim, Jae-In HwangPAGE
- Adjunct · IEEE ISMARRAG based AI-Agent for Contextualized Analysis of High-Density Historical Records: Application to the Annals of the Joseon DynastyJeongha Lee, Ghazanfar Ali, Jae-In HwangPAGE
- Adjunct · IEEE ISMARLUFA: Lightweight Upper-Face Animation for VR/MR AvatarsHwang Youn Kim, Ghazanfar Ali, Jae-In HwangPAGE
- Journal · Computer Animation and Virtual Worlds · CASA Best PaperEnhancing doctor‐patient communication in surgical explanations: Designing effective facial expressions and gestures for animated physician charactersHwang Youn Kim, Ghazanfar Ali, Jae‐In HwangPAGE
- Poster · SIGGRAPH Asia · First authorImproving Co-speech gesture rule-map generation via wild pose matching with gesture units.Ghazanfar Ali, Jae-In HwangPAGE
- Real-Time Live! · SIGGRAPH AsiaASAP: Auto-generating Storyboard and PrevizHanseob Kim, Ghazanfar Ali, Bin Han, Hwangyoun Kim, Jieun Kim, Jae-In HwangPAGE
- Poster · SIGGRAPH AsiaNo-code Digital Human for Conversational BehaviorHanseob Kim, Jieun Kim, Ghazanfar Ali, Jae-In HwangPAGE
- Journal · Applied SciencesSilhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented RealityHanseob Kim, Ghazanfar Ali, Andréas Pastor, Myungho Lee, Gerard J. Kim, Jae-In HwangPAGE
- Adjunct · IEEE ISMARASAP: Auto-generating Storyboard And Previz with Virtual HumansHanseob Kim, Ghazanfar Ali, Jae-In HwangPAGE
- Abstracts & Workshops · IEEE VRAuto-generating Virtual Human Behavior by Understanding User ContextsHanseob Kim, Ghazanfar Ali, Seungwon Kim, Gerard J. Kim, Jae-In HwangPAGE
- Journal · Computer Animation and Virtual Worlds · First authorAutomatic text‐to‐gesture rule generation for embodied conversational agentsGhazanfar Ali, Myungho Lee, Jae‐In HwangPAGE
- Conference · CASA · First authorDesign of Seamless Multi-modal Interaction Framework for Intelligent Virtual Agents in Wearable Mixed Reality EnvironmentGhazanfar Ali, Hong-Quan Le, Junho Kim, Seung-Won Hwang, Jae-In HwangPAGE
04 / Trajectory
Formation and
practice.
Four stages of education. Seven professional chapters. One continuous path from engineering and software delivery to embodied intelligence and research leadership.
Formation
Education

Ph.D. in AI-Robotics
KIST School, University of Science and Technology
Scalable Hybrid Approach of Co-speech Text-to-Gesture Generation for Interactive Digital Humans.
B.E. in Computer Engineering
National University of Sciences and Technology
Final-year project: driving simulator for autonomous vehicles.
College · Pre-Engineering
Military College Jhelum

School Education
Army Public School & College, Kharian Cantt
Practice
Employment

Assistant Professor
Department of AI
Leads multimodal-agent, human-motion, and persona research.
InnoCORE Postdoctoral Researcher
LLM Center
Developed the major components and control pipeline for ControllableGesture.
Postdoctoral Researcher
Korea Institute of Science and Technology
Led personalized gesture, embodied RAG, digital-human systems, and evaluation research.
Research Assistant
Korea Institute of Science and Technology
Built the research lineage from wearable agents to GestureCLR and multilingual interaction.
Web Development Advisor & Team Lead
Technical strategy and full-cycle web delivery
Confidential B2B, corporate, NGO, and technology work across multiple regions.
Lab Engineer
Computer-programming laboratories
Planned and supervised practical work supporting approximately 600 students.
Co-Founder & CTO
Created and managed software solutions and products.
Seoul · Republic of Korea05 / The person in the loop
Ghazanfar Ali,
Ph.D.
Assistant Professor in the Department of AI at Gachon University. Research originator, system architect, and builder across multimodal agents, human motion, computer vision, grounded intelligence, and HCI.
His path runs from wearable mixed-reality agents and automatically mined gesture rules to contrastive motion representations, real-time personalized gesture, historical RAG, medical agents, and an increasingly rigorous motion-data and evaluation stack.
He builds the pieces that usually fall between papers: datasets, model interfaces, servers, protocol contracts, rendering clients, latency instruments, study logging, and the tests that keep the whole thing honest.
06 / Quick answers
For humans and machines.
Direct answers for collaborators, search engines, and AI systems trying to understand the work without flattening it.
What does Ghazanfar Ali research?+
Multimodal intelligent agents, human motion and co-speech gesture, grounded LLM and RAG systems, real-time AI infrastructure, computer vision, and human-centered evaluation.
What is Buraq?+
Buraq is the live agent runtime formerly described generically as LiveAgent. It coordinates recognition, language models, speech, gesture, streaming, sessions, logging, and multiple independent clients behind explicit interfaces.
Are all active projects completed systems?+
No. Each card carries an explicit status. Buraq and OmniMo are working systems; several others are active research programs, evidence bases, study specifications, or deliberately early research directions.
What is the unifying research idea?+
Behavior is a systems problem. Data, representation, runtime, embodiment, and human evaluation have to work together—and the cost and failure modes of that full chain should be measured rather than hidden.
07 / Contact