Natural User Interface for Physics-Based Character Animation

Using natural user interface to interact with digital worlds is becoming commonplace in our daily life, as evidenced by the high demand for Microsoft Kinect since its launch in 2010. However, comparatively little research effort has focused in the past on harnessing these capabilities to create great applications. In this paper, we introduce unified framework for combining natural user interface and physics-based character animation. Our framework takes the form of a continuous controllable space for the combination of the two techniques. We also propose a human-in-the-loop control paradigm which allows a player or performer to sense and act for the character. With the information encapsulated in the human performance, the proposed framework accomplishes its goal in two steps: first, by recognizing and potentially modifying the performance such that it is appropriate for the given scenario; and second, by balancing interactivity and control in order to maintain both physical responsivity to the virtual world and faithfulness to the human performance.