Overview
We are currently preparing a new version of the Virtual Human Toolkit, aimed to be released on GitHub in the fall of 2025. The original version is no longer available, but its documentation can still be accessed from the Legacy menu.
The Virtual Human Toolkit (VHToolkit) is a research and development platform for the creation of virtual humans, also called embodied conversational agents or socially intelligent agents. The VHToolkit enables the creation and deployment of real-time interactive characters that perceive end-users and respond both verbally and nonverbally.
The VHToolkit has the following features:
- Integrated framework; the VHToolkit combines audio-visual sensing and speech recognition with natural language processing, text-to-speech synthesis, and nonverbal behavior generation within a single framework.
- Flexible architecture; select from multiple technology vendors or open source solutions as well as cloud services or local technologies (e.g., OpenAI ChatGPT, Anthropic Claude, AWS Lex V2, or RASA).
- Extendable API; add your own technology by implementing the principled API.
- Multi-platform support; the VHToolkit supports Windows, MacOS, Linux, Android, iOS, and AR/VR.
- Custom character creation; create your own character with Character Creator and import into the VHToolkit to make it interactive. Requires a separate license.
- Personalized avatar creation; use Reallusion’s Character Creator and headshot to create avatars based on real people. Optionally clone their voice with ElevenLabs. Requires separate licenses.
The VHToolkit is powered by the Rapid Integration & Development Environment platform (RIDE) and targets game engine Unity. It can be used under a permissive license for academic and personal use. For commercial and government purpose use, please contact us.
When publishing work that uses the VHToolkit, please cite one of the following papers:
Hartholt, A., Fast, E., Li, Z., Kim, K., Leeds, A. and Mozgai, S., 2022, September. Re-architecting the virtual human toolkit: towards an interoperable platform for embodied conversational agent research and development. In Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents (pp. 1-8). Link
Hartholt, A., Traum, D., Marsella, S.C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.P. and Gratch, J., 2013, August. All together now: Introducing the virtual human toolkit. In International Workshop on Intelligent Virtual Agents (pp. 368-381). Berlin, Heidelberg: Springer Berlin Heidelberg. Link