Computer vision based interfaces for interactive simulations

Abstract

3D environments are commonplace in applications for simulation, gaming and design. However, interaction with these environments has traditionally been limited by the use of 2D interface devices. This paper explores the use of computer vision to capture the 3D motion of a handheld object by tracking known features. Captured motion is translated into control of an object onscreen, allowing 3D interaction with a rendered environment. Objects are tracked in real-time in video from a single webcam. The technique is demonstrated using two real-time interactive applications.