Assistant Professor · AI Researcher · End-to-End Builder
I make machines behave.
Language is only one channel. I build intelligent systems that can speak, move, remember, adapt—and survive contact with real people.
outputs
participated in
demos
in OmniMo
00 / Thesis
Intelligence is not the model.
It is the whole encounter.
I work from corpus to client: motion data, representations, retrieval and generation, APIs, real-time animation, embodied interfaces, and the studies that decide whether any of it matters.
The through-line is simple: make intelligent behavior scalable, controllable, grounded, and measurable.
01 / Now
The live research ecosystem.
Working platforms, active evidence programs, manuscripts, studies, and intentionally labeled research directions. No proposal is dressed up as a finished result.
A.01Working system
Live multimodal agent runtime
Buraq
ASR, LLM, speech, gesture, sessions and instrumentation—overlapped behind one documented contract.- 314 server tests
- Studio · Unity · Web
- Provider-driven
Buraq is a reusable live agent runtime, not a one-off demo. It coordinates recognition, language generation, speech, gesture, cancellation, session state, feedback and structured interaction records while serving independent desktop, Unity and browser clients.
The architectural rule is unusually strict: clients consume the wire contract, not server code; data findings flow back to the repository that owns the cause. That discipline turns the platform into a credible research instrument.
- Working provider interfaces for ASR, LLM, TTS and multiple gesture methods.
- Streaming and queueing overlap the expensive stages instead of serializing the conversation.
- Capabilities are advertised per session; clients can be built from the tutorial and API contract alone.
A.02Working platformMotion data infrastructure
OmniMo
Many skeleton dialects in. One auditable Shared Joint Space out.- 6+ corpora
- 600+ hours
- Continuously expanding
OmniMo ingests motion from different file formats and skeletons and produces a consistent, provenance-aware representation for learning and runtime systems. It supports whole-corpus trees, memory-mapped reading, safe windowing, standardization, export, and build-on-demand.
The most valuable output is not only normalized data; it is a growing playbook of silent dataset failure modes discovered by distrustful measurement.
- Six processed corpora read through a common interface.
- No training window crosses a take boundary or excluded defect span.
- Quality gates exposed rigid takes, wrist flips, root defects, frame-rate errors and rest-pose ambiguities.
A.03Research programVideo → retargetable 4D motion
MotionBridge
A consumer-accessible path from monocular video to clean, complete motion and a frozen token vocabulary.- 4-stage program
- Hands + root
- Economics-aware
MotionBridge targets the practical bottleneck behind motion intelligence: turning ordinary monocular video into clean, complete, retargetable motion at a cost that a smaller lab can sustain.
- R1 learns a 3D-to-3D motion vocabulary.
- R2 aligns synthetic 2D observations with that frozen vocabulary.
- R3 distills image evidence into the token space.
- R4 benchmarks and adapts on real video with throughput, yield and usable motion-hour cost reported beside accuracy.
A.04SpecificationMotion → video + exact truth
MotionStage
A deliberately controllable renderer where ground truth—not photorealism—is the product.- Exact 2D/3D
- Occlusion + masks
- Free path
MotionStage is the synthetic companion to MotionBridge. It turns known skeletal motion into rendered observations while emitting the exact camera, 2D, 3D, occlusion and segmentation truth needed for controlled learning and evaluation.
It is intentionally specified around permissive tools and one consumer GPU. The first gate is one clip, end to end, with triangulation and floor-placement checks—not a glossy demo.
A.05Evidence in progressMotion-corpus economics review
What Does a Motion-Hour Cost?
A field-wide audit of the compute, yield, labor, licensing and reporting practices hidden behind motion datasets.- 375+ verified works
- 343+ paper notes
- Living evidence base
This review asks a question the field rarely makes answerable: what does one usable hour of motion actually cost after detection, reconstruction, cleanup, failure, licensing, compute and human labor?
The evidence base is ahead of the manuscript and continuously expanding: at least 375 unique works were citation-verified, 337 full texts were cached, and more than 343 structured notes support a reporting-practice audit. Data and figures are being completed before drafting claims.
A.06Benchmark activeEvaluation that audits itself
Motion EvalSuite + GQNet
From perceptual quality and semantic fit to foot sliding, latency, VRAM and whether a metric agrees with people.- 7 evaluation axes
- ≈35 metrics planned
- 6/7 baselines measured
The evaluation program spans co-speech gesture and general text-to-motion. Its central position is skeptical: a familiar metric is not automatically a meaningful instrument.
- A resource harness runs each baseline's real sampling loop and records whole-process memory and latency.
- Six of seven recent baselines are measured; the final adapter remains open.
- GQNet explores learned artifact and perceptual quality estimation; human-aligned work remains a planned study, not a published result.
A.07Manuscript · under reviewLow-latency zero-shot style
ConGRets
Contrastive text-and-style retrieval from curated gesture units—speaker personalization without audio or frame-by-frame decoding.- <500 ms / 1,000 CPU queries*
- <100 MB*
- Motion-only style*
ConGRets learns a global motion-derived speaker-style representation and aligns text plus style with a frozen gesture space. Runtime retrieves curated two-to-three-second units, keeping motion-capture quality while remaining friendly to chatbot and TTS pipelines.
Reported manuscript results—including 1,000 CPU queries in under 500 ms, a sub-100 MB footprint, 99% style classification and 89% content-to-cluster accuracy—remain labeled under review.
A.08Design v1Text + authored visual persona
ControllableGesture
Nine interpretable control signals at inference, without requiring audio.- 9 control axes
- Text-only runtime
- Study pending
The current design conditions co-speech motion on semantic control values such as handedness, gesture height, amplitude, speed, emotion and intensity. Prosodic signals can supervise training while text plus authored controls remain sufficient at inference.
The repository currently contains the design specification; implementation milestones and the expert study are pending. Earlier research produced a 99,022-unit control store, but that lineage is not presented as completion of the new study.
A.09Proposal · v0.4Identity–behavior incongruence
Behavioral Imposter Test
A signal-detection and survival-analysis instrument for when a digital persona stops feeling like itself.- Live interrogation
- 4 behavior channels
- No data collected yet
BIT proposes a standardized behavioral Turing test for embodied personas. A participant interrogates an agent that claims an identity; one behavior channel is altered, and the instrument measures both detection sensitivity and questions-to-detection.
The current repository is explicit: this is a study specification. No participants have been collected, no IRB has been filed, and runtime study instrumentation is not yet complete.
A.10Planned · gatedToken boundaries that mean something
Phase-Aligned Tokenizer
Test whether gesture-phase boundaries concentrate semantics better than arbitrary fixed windows.- 4 matched tokenizers
- Variable rate
- Segmenter gate first
The project compares phase-aligned, fixed-rate and learned tokenizations at matched bitrate and with an identical downstream generator. The claim only survives if the phase segmenter is reliable and the tokens improve checkable semantic behavior—not merely reconstruction.
It is a staged plan. The segmenter and token analysis are explicit go/no-go gates before expensive generator training.
A.11Planned · dependentWhat auxiliary data actually buys
Multi-Task Transfer
A matched-compute study of whether action language improves co-speech gesture meaning.- 5 controlled arms
- Same backbone
- Tokenizer-dependent
Instead of training a billion-parameter everything-model, this study isolates one scientific question: does text-to-action or motion-to-text data improve the semantic appropriateness of co-speech gesture at matched compute?
Five arms hold architecture, steps and token budget fixed. The study begins only after the tokenizer project freezes its representation and baseline configuration.
A.12Planned · can startMiddle-Out TTS
Prosody-Lite Conditioning
Price the perceptual value of duration, pitch and energy before audio exists.- 4 conditioning levels
- Latency-value curve
- Corpus phase first
This project predicts phoneme durations and prosodic features from text before speech synthesis, then asks what each additional conditioning level buys in perceived synchrony, emphasis and naturalness—and what it costs in latency.
The corpus correlation phase can start independently. Larger perception work is gated by a pilot that first proves the generation backbone can express a detectable difference.
A.13Research directionOn-manifold, interruption-ready motion
Semantic Motion Matching
Frame-level retrieval steered by kinematics, meaning and persona—plus skeleton-invariant matching.- ≈100 ms cadence proposed
- No decoder at runtime
- Not yet a completed system
This emerging direction moves from whole-unit retrieval toward frame-level motion matching. A weighted query combines motion state, compact semantic coordinates and persona controls, preserving captured-motion realism while allowing interruption and fast redirection.
A connected skeleton-invariant proposal asks whether the same motion can be retrieved across different proportions and rigs. Both are deliberately labeled as research directions.
02 / Published
Seventeen outputs.
Thirteen visual ideas.
Journal papers, conference papers, adjunct items, posters, and a live demonstration are grouped by the system they actually describe. Every bibliographic output remains visible below.

2026 · IEEE Access + ISMAR-Adjunct 2025
Speech-conditioned upper-face animation
Emotion and liveness composed into a lightweight real-time signal.Lightweight Speech-Conditioned Upper-Face Animation for Virtual Agents via Emotion–Liveness Composition extends real-time avatar behavior beyond lip sync, combining speech-conditioned emotion with subtle upper-face liveness. LUFA is the related 2025 ISMAR-Adjunct item.

2025 · IEEE Access · First author
Multilingual motion retrieval + GestureCLR
High-noise 2D pose aligned to 2,035 clean gesture units; cross-lingual interaction evaluated with 51 people.Expanding Multilingual Co-Speech Interaction scales a gesture library from 56 to 2,035 units and a mapping base from about 84,000 to 210,000 rules. GestureCLR raises reported high-noise matching from 18% to 97%, while translation does not meaningfully degrade the user experience in a 51-participant study.

2025 · CAVW · First author
RIDGE
High-confidence captured rules when possible; contrastive learning when necessary.RIDGE combines an LLM-assisted high-fidelity rule base with a learned fallback in a shared text–motion space. It introduces Gesture Cluster Affinity for semantic coherence and reports GCA 0.73, versus approximately 0.61 and 0.52 for compared paths.

2025 · CAVW + ISMAR-Adjunct
Joseon Dynasty RAG agent
Dynamic article chunking, date-aware retrieval, source grounding, and an embodied voice-and-motion interface.The system grounds an embodied agent in 49.6 million characters of high-density chronological records. It preserves article boundaries, regenerates vague queries, filters by date, maximizes evidence within the token budget, and cites sources.

2025 journal · 2021 adjunct · 2022 live
ASAP: screenplay → many visual worlds
Storyboards, animated 3D previz, and immersive scenes generated from a screenplay.ASAP parses screenplay structure into characters, dialogue, actions and emotion, then composes storyboards, animated previsualization and immersive scenes. The journal reports 93% top-1 accuracy on simple action sentences and 87% on complex ones; Ghazanfar Ali and Hanseob Kim are marked as equal contributors.

2024 · CAVW · CASA Best Paper
Virtual physician
Grounded medical answers carried by carefully designed expression, gesture, and social presence.This dual-mode system helps clinicians prepare surgical explanations and lets patients ask grounded follow-up questions through an animated physician. The RAG answer F1 improves from 0.492 to 0.779 in the reported comparison; a 113-participant study finds higher social presence than text-and-audio slides.

2021 · Applied Sciences
Silhouettes make mobile AR believable
Lightweight real-object geometry gives virtual humans occlusion and physical context.A compact segmentation model turns real objects into silhouette geometry so a mobile AR character can walk behind, stand on, and interact around the physical scene. The work reports a 2.4M-parameter model, 0.971 mIoU and a 24-participant study.

2020 · CAVW · First author
Automatic text-to-gesture rules
Mine useful behavior from 106 hours of public video instead of asking experts to author every map.This work automates rule-based co-speech mapping from about 106 hours of public video using 2D pose, sliding-window matching, and semantic word embeddings at runtime. Evaluation finds performance comparable to manual maps with more gesture variety and complementary effects when the maps are combined.

2026 · ECCV
Through Van Gogh’s Eyes
Global artist-level diffusion style transfer designed to learn a distribution, not repeat one iconic exemplar.The work addresses stylistic bias in diffusion models by representing an artist through a broader book of genuine artworks and transferring global artistic characteristics instead of overfitting to a narrow familiar motif. Ghazanfar Ali is a co-author; the portfolio does not invent a specific module ownership.

2022 · SIGGRAPH Asia Posters · First author
Wild pose matching with GestureCLR
Robust 2D-to-3D matching expands the gesture bank to 2,035 units and 210,000 rules.This poster introduces contrastive alignment between noisy 2D pose extracted from public monologue video and clean 3D gesture units, followed by clustering for runtime variety. It is the bridge between early automatic mapping and the later multilingual system.

2022 · SIGGRAPH Asia Posters
Flow Human
No-code conversation flows become verbal and nonverbal digital-human behavior.Flow Human lets service designers author a conversation flow while the system automatically coordinates speech, facial animation, co-speech gesture, interaction and feedback collection for an embodied kiosk experience.

2021 · IEEE VR Workshops
Context → virtual-human action
Joint sentence and entity understanding turns conversation into grounded room behavior.A BERT-based system distinguishes conversation from action and extracts the entities needed to execute flexible behavior directly from natural language. The pilot investigates whether a virtual human can move beyond fixed phrase activation.

2019 · CASA · First author
Wearable mixed-reality agent
The foundational system: place-aware, multimodal, embodied, and designed for real cultural spaces.This early framework unifies spatial mapping, gaze, speech recognition, a domain chatbot, object recognition, animation, emotion and gesture for virtual guides in museums and botanical gardens. It establishes the system-level research arc that later becomes Buraq.
03 / Record
The complete output ledger.
Eight journal articles and nine conference, adjunct, poster, or live-demonstration items. Labels matter; not every item is presented as a full paper.
- Journal · IEEE AccessLightweight Speech-Conditioned Upper-Face Animation for Virtual Agents via Emotion–Liveness CompositionHwang Youn Kim, Ghazanfar Ali, Jeongha Lee, Jae-In HwangPDF
- Conference · ECCVThrough Van Gogh’s Eyes: Global Style Transfer with Diffusion ModelJeongha Lee, Yujin Kim, Ghazanfar Ali, Suhyun Kim, Jae-In HwangPDF
- Journal · IEEE Access · First authorExpanding Multilingual Co-Speech Interaction: The Impact of Enhanced Gesture Units in Text-to-Gesture Synthesis for Digital HumansGhazanfar Ali, Woojoo Kim, Muhammad Shahid Anwar, Jae-In Hwang, Ahyoung ChoiPDF
- Journal · Computer Animation and Virtual Worlds · First authorRIDGE: Rule-Infused Deep Learning for Realistic Co-Speech Gesture GenerationGhazanfar Ali, Hwang Youn Kim, Jae-In HwangPDF
- Journal · Computer Animation and Virtual WorldsA Retrieval-Augmented Generation System for Accurate and Contextual Historical Analysis: AI-Agent for the Annals of the Joseon DynastyJeong Ha Lee, Ghazanfar Ali, Jae-In HwangPDF
- Journal · Multimedia Tools and Applications · Equal contributionASAP for Multi-Outputs: Auto-generating Storyboard and Pre-visualization with Virtual Actors Based on ScreenplayHanseob Kim, Ghazanfar Ali, Bin Han, Hwang Youn Kim, Jieun Kim, Hyemin Shin, Gerard Jounghyun Kim, Jae-In HwangPDF
- Adjunct · IEEE ISMARRAG-based AI-Agent for Contextualized Analysis of High-Density Historical Records: Application to the Annals of the Joseon DynastyJeongha Lee, Ghazanfar Ali, Jae-In HwangPDF
- Adjunct · IEEE ISMARLUFA: Lightweight Upper-Face Animation for VR/MR AvatarsHwang Youn Kim, Ghazanfar Ali, Jae-In HwangVIEW
- Journal · Computer Animation and Virtual Worlds · CASA Best PaperEnhancing Doctor-Patient Communication in Surgical Explanations: Designing Effective Facial Expressions and Gestures for Animated Physician CharactersHwang Youn Kim, Ghazanfar Ali, Jae-In HwangPDF
- Poster · SIGGRAPH Asia · First authorImproving Co-Speech Gesture Rule-Map Generation via Wild Pose Matching with Gesture UnitsGhazanfar Ali, Jae-In HwangPDF
- Real-Time Live! · SIGGRAPH AsiaASAP: Auto-generating Storyboard and PrevizHanseob Kim, Ghazanfar Ali, Bin Han, Hwang Youn Kim, Jieun Kim, Jae-In HwangVIEW
- Poster · SIGGRAPH AsiaNo-Code Digital Human for Conversational BehaviorHanseob Kim, Jieun Kim, Ghazanfar Ali, Jae-In HwangPDF
- Journal · Applied SciencesSilhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented RealityHanseob Kim, Ghazanfar Ali, Andreas Pastor, Myungho Lee, Gerard J. Kim, Jae-In HwangPDF
- Adjunct · IEEE ISMARASAP: Auto-generating Storyboard and Previz with Virtual HumansHanseob Kim, Ghazanfar Ali, Jae-In HwangVIEW
- Abstracts & Workshops · IEEE VRAuto-generating Virtual Human Behavior by Understanding User ContextsHanseob Kim, Ghazanfar Ali, Seungwon Kim, Gerard J. Kim, Jae-In HwangPDF
- Journal · Computer Animation and Virtual Worlds · First authorAutomatic Text-to-Gesture Rule Generation for Embodied Conversational AgentsGhazanfar Ali, Myungho Lee, Jae-In HwangPDF
- Conference · CASA · First authorDesign of Seamless Multi-modal Interaction Framework for Intelligent Virtual Agents in Wearable Mixed Reality EnvironmentGhazanfar Ali, Hong-Quan Le, Junho Kim, Seung-Won Hwang, Jae-In HwangPDF
04 / Trajectory
Formation and
practice.
Four stages of education. Seven professional chapters. One continuous path from engineering and software delivery to embodied intelligence and research leadership.
Formation
Education

Ph.D. in AI-Robotics
KIST School, University of Science and Technology
Scalable Hybrid Approach of Co-speech Text-to-Gesture Generation for Interactive Digital Humans.
B.E. in Computer Engineering
National University of Sciences and Technology
Final-year project: driving simulator for autonomous vehicles.
College · Pre-Engineering
Military College Jhelum

School Education
Army Public School & College, Kharian Cantt
Practice
Employment

Assistant Professor
Department of AI
Leads multimodal-agent, human-motion, and persona research.
InnoCORE Postdoctoral Researcher
LLM Center
Developed the major components and control pipeline for ControllableGesture.
Postdoctoral Researcher
Korea Institute of Science and Technology
Led personalized gesture, embodied RAG, digital-human systems, and evaluation research.
Research Assistant
Korea Institute of Science and Technology
Built the research lineage from wearable agents to GestureCLR and multilingual interaction.
Web Development Advisor & Team Lead
Technical strategy and full-cycle web delivery
Confidential B2B, corporate, NGO, and technology work across multiple regions.
Lab Engineer
Computer-programming laboratories
Planned and supervised practical work supporting approximately 600 students.
Co-Founder & CTO
Created and managed software solutions and products.
Seoul · Republic of Korea05 / The person in the loop
Ghazanfar Ali,
Ph.D.
Assistant Professor in the Department of AI at Gachon University. Research originator, system architect, and builder across multimodal agents, human motion, computer vision, grounded intelligence, and HCI.
His path runs from wearable mixed-reality agents and automatically mined gesture rules to contrastive motion representations, real-time personalized gesture, historical RAG, medical agents, and an increasingly rigorous motion-data and evaluation stack.
He builds the pieces that usually fall between papers: datasets, model interfaces, servers, protocol contracts, rendering clients, latency instruments, study logging, and the tests that keep the whole thing honest.
06 / Quick answers
For humans and machines.
Direct answers for collaborators, search engines, and AI systems trying to understand the work without flattening it.
What does Ghazanfar Ali research?+
Multimodal intelligent agents, human motion and co-speech gesture, grounded LLM and RAG systems, real-time AI infrastructure, computer vision, and human-centered evaluation.
What is Buraq?+
Buraq is the live agent runtime formerly described generically as LiveAgent. It coordinates recognition, language models, speech, gesture, streaming, sessions, logging, and multiple independent clients behind explicit interfaces.
Are all active projects completed systems?+
No. Each card carries an explicit status. Buraq and OmniMo are working systems; several others are active research programs, evidence bases, study specifications, or deliberately early research directions.
What is the unifying research idea?+
Behavior is a systems problem. Data, representation, runtime, embodiment, and human evaluation have to work together—and the cost and failure modes of that full chain should be measured rather than hidden.
07 / Contact