Real-time Motion Capture
The main goal
is to capture real time human data and apply it to computer-generated
characters. The characters are modeled in Maya and animated using incoming
data from the motion capture system.
What is Motion Capture?
Motion Capture is the process of capturing the movement of a real object in 3D space and mapping it onto a computer generated object. The result is a computer character that moves exactly like the real object. Usually, Motion Capture is used to capture the motions of humans.
How does Motion Capture work?
For ìopticalî motion capture, a series of
high resolution cameras with special strobe lights are set up around an area
to be ìcaptured.î Small spheres (markers) covered with a
retro-reflective substance are placed in strategic locations on the person
(or object). Specialized software will locate the markers seen through
the cameras. The markers are then recorded as 3D coordinates (xyz). The
collected marker data over time creates motion data.
What is real time Motion Capture?
There are two types of motion capture:
offline and online (real time). In the case of the offline, the data is
captured, stored, processed (cleaning data from noise is common) and then
later applied to the computer character.İ
However, in real time capture the motion data is captured and applied
to the character simultaneously. Depending on the capture system, there
should be no noticeable delay in the characterís movement compared to the
movement of the real subject.
A common problem with real time motion capture
is the presence of noise. If you plot the motion data you will notice that
the curves arenít always smooth, there are some unsettledness or jerkiness.
There are many filtering systems and techniques to eliminate the ìbad dataî.
During offline capture, one can go back to the data and ìclean it upî before
applying it to the computer character. But how about real time capture?
. . . Stages of Work . . .
İ Stage 1: Understanding how motion capture is done
The first step was to
familiarize myself with the motion capture lab and learn about conducting a
motion capture session. I attended one of the capture sessions and went
through a basic tutorial of how to work the equipment and what are the
necessary steps in capturing a motion. Here are the basic steps:
Calibration Axisİİİİİİİİİİİİİ İİİİİİİİİİİİİİİİCalibration Stick
(photo from the Vicon website)İ
İ Stage 2: Writing a plugin for Maya
İİİİİİİİİİİİ The captured data
from Vicon will be applied to a computer character generated in Maya. Maya
is a 3D modeling and animation software. In order to apply the real time data
to the character, there must be some type of a link or communication method
between Maya and Vicon. This part of the project involves writing a plugin (
an additional program that enhances the features of an existing program) that
will allow Maya to receive data from Vicon.
İİİİİİ Writing the plugin requires a good
understanding of networking concepts, an in-depth knowledge of how data is
handled by Tarsus (a Vicon software used for motion capture) and how Maya
maps incoming data to objects. This project seemed to be heavily involved
since I didnít know much about networking, Tarsus or Maya. Luckily, I was
able to contact the people at Vicon and they told me that they had already
written a plugin for Maya and that theyíll e-mail a copy to me.
İ Stage 3: Modifying the plugin
İİİİİİİİİİİİ This is the general idea of how
data is communicated between Tarsus and Maya:
Tarsus outputs data using named channels. Channels
are strings of double values varying over time. Each channel corresponds to a
specific source that could be either a marker or a body segment. In turn, the
plugin creates channels (vectors) to store the incoming channel data. coming from
the Tarsus channels.İ
The data stored in the vector channels is parsed,
modified and sent out to Maya using routines in the Maya server library.
consists of the position(x,y,z) and orientation (qx, qy, qz ) of the body parts for the real time subject and the
position (only) of the corresponding markers. However, Tarsus doesnít provide
joint angle data, which is what we need. Therefore, the code must be modified
to extract joint angle information from the available position/orientation
the position and orientation of two body parts we can calculate the angle for
the joint joining the two bodies using quaternions. A quaternion is a 4-tuple
of real numbers [s,x,y,z] or [s,v], s is a scalar and v is a
three-dimensional vector.İ First,
represent the position/orientation data of both body parts as quaternions,
find the difference of the two quaternions and the result is the angle for
connecting the two body parts.
İ Stage 4: Modifying the Vicon files
İİİİİİİİ Part of the Vicon system are files
called .MKR and .KM files, they contain lists of: 1) marker labels, 2) body
segments and their corresponding markers. By default, the marker file (.MKR)
for a human contains 40-41 markers and the kinematic model file (.KM)
contains 17 body segments. İ
İİİİİİİİİİ A typical human
skeleton file contains around 30 body parts, which adds more degrees of
freedom to the body and results in a better looking motion. To match with the
current skeleton files being used, the marker and kinematic model files need to
be modified to include more body segments. Both files can be viewed as text
files and to change the number of segments requires typing in few additional