Virtual Human Creation Pipeline

William Swartout, Jonathan Gratch, Randall Hill, Eduard Hovy, Stacy Marsella, Jeff Rickel and David Traum, “Toward Virtual Humans,” in AI Magazine, v.27(1), 2006

Jonathan Gratch, Jeff Rickel, Elisabeth André, Norman Badler, Justine Cassell, and Eric Petajan, "Creating Interactive Virtual Humans: Some Assembly Required," in IEEE Intelligent Systems July/August 2002, pp. 54-63

Kenny, P., Hartholt, A., Gratch, J., Swartout W., Traum, D., Marsella, S., Piepol, D., (2007).  Building Interactive Virtual Humans for Training Environments. Interservice / Industry, Training, Simulation & Education Conference. 

Speech Recognition or Speech-to-Text conversion: The first block of a virtual character system

Statistical Approaches for Language Understanding in Virtual Humans

Anton Leuski, Ronakkumar Patel, David Traum, and Brandon Kennedy. "Building effective question answering characters". In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue, Sydney, Australia, July 2006.

Anton Leuski, David Traum, A Statistical Approach for Text Processing in Virtual Humans, Army Science Conference, Dec 2008 (In Press).

Dialogue Modeling for Virtual Humans
David Traum, Jonathan Gratch, Stacy Marsella, Jina Lee, Arno Hartholt, Multi-party, Multi-issue, Multi-strategy Negotiation for Multi-modal Virtual Agents, 8th International Conference on Intelligent Virtual Agents, Tokyo, Japan, September 2008

Perception of Human Nonverbal Behaviors

Head Tracker:
Morency, L.-P., Rahimi, A., and Trevor Darrell, Adaptive View-based Appearance Model, CVPR 2003, vol. 1, pp. 803-810, Wisconsin, June 2003

Gesture Recognition:
L.-P. Morency, A. Quattoni and Trevor Darrell,  Latent-Dynamic Discriminative Models for Continuous Gesture Recognition , Proceedings IEEE Conference on Computer Vision and Pattern Recognition, June 2007

Context-Based Recognition:
L.-P. Morency, C. Sidner, C. Lee, and T. Darrell, Head Gestures for Perceptual Interfaces: The Role of Context in Improving Recognition. Artificial Intelligence. Elsevier, 171(8-9):568-585, June 2007

Emotion and Cognition

Stacy Marsella and Jonathan Gratch, “EMA: A Model of Emotional Dynamics,” Journal of Cognitive Systems Research (attached)

Stacy Marsella and Jonathan Gratch, “EMA: A computational model of appraisal dynamics,” in Agent Construction and Emotions, Vienna, Austria 2006

Nonverbal Behavior

Non-Verbal Behavior Generation:
Lee, J., Marsella, S. Nonverbal Behavior Generator for Embodied Conversational Agents. 6th International Conference on Intelligent Virtual Agents, Marina del Rey, CA, 2006. pp 243-255.

SmartBody:
Thiebaux, M., Marshall, A., Marsella, S. and Kallmann, M. SmartBody: Behavior Realization for Embodied Conversational Agents Proceedings of Autonomous Agents and Multi-Agent Systems (AAMAS), 2008.

Behavior Markup Language:
Vilhjalmsson, H., Cantelmo, N., Cassell, J., Chafai, N., Kipp, M., Kopp, S., Mancini, M., Marsella, S., Marshall A., Pelachaud, C., Ruttkay Z., Thorisson, K., van Welbergen, H. and van der Werf, R. The Behavior Markup Language: Recent Developments and Challenges IVA 2007, LNAI 4722: 99-111, Springer-Verlag Berlin Heidelberg

Stefan Kopp, Brigitte Krenn, Stacy Marsella, Andrew N. Marshall, Catherine Pelachaud, Hannes Pirker, Kristinn R. Thórisson and Hannes Vilhjálmsson, Towards a Common Framework for Multimodal Generation: The Behavior Markup Language, 6th International Conference on Intelligent Virtual Agents, Marina del Rey, CA, 2006.

Knowledge Modeling for Domains

Hartholt, A., T.A. Russ, D. Traum, E.H. Hovy, and S. Robinson. 2008. A Common Ground for Virtual Humans: Using an Ontology in a Natural Language Oriented Virtual Human.  Proceedings of the LREC conference. Marrakech, Morocco. May 2008.

Presentation Slides:

Gratch - Virtual Human Intro

Kenny - Virtual Human System

Georgiou - Speech Recognition

Leuski - Language Understanding

Gandhe - Dialogue

Morency - Perception

Gratch - Emotion

Marsella - Non Verbal Behavior

Hovy - Knowledge Modeling

Kenny - Clue Game

Tutorials Slides:

Virtual Human Toolkit Overview

NPC Editor - Dialog

Smartbody - Animation

Non-Verbal Behavior

Video Demo of Gaze