Untitled Document

3D Puppetry: A Kinect-based Interface for 3D Animation

Download and Installation

Disclaimer:
Our code is provided under a BSD 2.0 license (see the LICENSE.txt file in kinect_tracker.tar.gz--instructions for download are below). We provide all our code as-is without further support.

Operating System

Our system was designed and tested using Ubuntu 11.10 (64-bit) and the 'electric' version of the Robotics Operating System. Please ensure that you also have the latest versin of the g++ compiler by running:

g++ --version

We have tested our system using version 4.6.1. Earlier versions may not compile correctly.

Hardware

Aside from a Kinect, we recommend a powerful multi-core PC for running our software. Our development machine, which was able to run our tracker at real-time frame rates, was a desktop PC with a 24-core, 3.33GHz Xeon CPU, 12GB RAM, and an NVIDIA GTX 580 video card.

Physical Setup

If you plan to create your own animations, we recommend you use a similar physical setup to us. We use our system in a well-lit room with even illumination. We use a tabletop covered with white foam board and mount the Kinect on a tripod so the sensor rests 75cm from the tabletop. It is important that the area visualized by the kinect is uniform, with no visible seams or noticeable blemishes. We orient the Kinect so its optical axis makes a 22.5-degree angle with the tabletop's surface normal, which provides a trackable performance volume of roughly 90cm by 60cm by 30cm. Both during capture of a puppet's template images and during performance, it is important that the white poster board fill the Kinect's entire field of view.

Step-by-step Software Installation

1. Install third-party software and drivers:

Install the 'electric' version of ROS:
http://ros.org/wiki/electric/Installation/Ubuntu
Install cuda:
http://developer.nvidia.com/cuda-downloads

2. Install our software and setup system variables:

Create a new folder titled '3D_Puppetry.' Within that folder, expand the tar files we have provided:

>ROS package dependencies: kinect_tracker_dependencies.tar.gz (33MB)
>Image-based puppet identifier: rth_SIFT.tar.gz(56KB)
>The main program: kinect_tracker.tar.gz (942MB)
>NOTE: If you've already downloaded the code from github, you can separately download our sample puppet models, backgrounds, and animations (which are also included in the kinect_tracker tar ball above): 3d_models_and_animations.tar.gz (482MB)

Edit .bashrc by adding this line (Substituting the correct directory for your setup):

    export ROS_PACKAGE_PATH=~Documents/Code/3D_Puppetry:$ROS_PACKAGE_PATH

Also add to .bashrc:

    PATH=/usr/local/cuda/bin:$PATH
     export PATH=/usr/local/cuda/bin:$PATH
     export LPATH=/usr/lib/nvidia-current:$LPATH
     export LIBRARY_PATH=/usr/lib/nvidia-current:$LIBRARY_PATH
     export LD_LIBRARY_PATH=/usr/lib/nvidiacurrent:/usr/local/cuda/lib64:/usr/local/cuda/lib:$LD_LIBRARY_PATH

Install necessary libraries (from command prompt):

     $ roscd kinect_tracker directory
     $ rosmake --rosdep-install

3. Install the example backgrounds:

They are found in the archive backgrounds.tar.gz within the files/backgrounds/ directory. Make sure the folders are all placed within backgrounds/

4. (Recommended) If you are using our example animations or have physical versions of our puppet models:

Extract ply models. They are found in the archive all_models.tar.gz within the kinect_tracker/files/models/ directory. Make sure the ply files are all placed within models/

Extract SIFT templates. They are found in the archive SIFT_templates.tar.gz within the kinect_tracker/ files/SIFT/ directory. Make sure the folders are all placed within SIFT/, and not SIFT/SIFT_templates/

Extract animation input. They are found in the archive 3D_Puppetry_animation_input.tar.gz within the kinect_tracker/files/animation/ directory.    Make sure the files are all placed within the animations/ directory.

If you would like to buy physical versions of some of the models we include as PLYs, they can be found on Amazon.com:
       Construction Vehicles
       Emergency Vehicles

5. (Recommended) To enable antialiasing:

In the terminal, run nvidia-settings
Choose Antialiasing Settings
Under "Antialiasing Settings," choose "Override Application Settings" and set to 8x
Enable Anisotropic Settings and set it to 2x

6. Test

To test the system, make sure a kinect is attached to your computer and execute:
roslaunch kinect_tracker all_puppets_launch.launch
Press 'p' to playback the stored animation