lowerback to upperback
upperback to thorax
rfemur to rtibia
lfemur to ltibia
rtibia to rfoot
ltibia to lfoot
6/6-Monday
-----------------------------------
Came back from Boston
6/7-Tuesday
------------------------------------
met with Prof. Pollard to discuss my project ideas
1. Modal decomposition of an object surface, and finding a corresponding decomposition of the hand poses. (The bases hand poses can be obtained by motion capturing a hand grasping the bases shapes.)
2. Conduct studies and talk with health professions who are knowledgable about the human hand.
Borrowed some reading material about function and dysfunctional hand movements. Found papers on modal decomposition in 1D, 2D, 3D.
6/8
-----------------------------
Modal Decomposition of object surface - Started experimenting with 1D modal decomposition in Matlab.
Wrote a program to read in profile of a bottle, and use SVD to find bases to reconstruct the bottle.
Q: How much of the high frequencies bases can we cut off? For every basis, we would need to create that shape and motion capture the hand grasp. High frequencies would not be useful.
6/9
------------------
Got the bottle profile approximation program to work.
Start thinking about 2D modal decomp - to read paper "Multiresolution Approach for Texture Synthesis Using CHF". Think about intersections of the bottle.
Think about parts of hand most important for applying force
- palm
- grove between thumb and index finger
Noticed limitation of matching surface curve with a hand pose. When doing a linear combination of curves. 2*(pic of sine curve) = (another pic) but twice the weight of a pose won't make the whole grasp twice as wide.
-> We can have each pose component be the distance from closed fist pose! - greater than 1 coefficients seem to only happen with the flat surface
Read "Fourier Principles for Emotion-based Human Figure Animation"
Their params - step, speed, gait, jump
I think my params would be finger angles with relative to object and spread of fingers.
(put marks where fingers will be on the object (?) control spread of fingers)
6/10
-----------------
Got polar coordinates modal decomp to work.
Looked over mocap data of hand grasps from earlier hand studies.
- will we need to stretch/squash time for diff parts of the hand?
- palm, pinky and ring fingers make contact earlier
6/13/Mon
-----------------
Met with Nancy.
Hmmm... thinking of new ideas. Modal decomp project might not pan out.
- physical simulator with hands?
- do something with pen flip motion captured hand motion?
- extracting 3D motions from 2D movies?
6/14
-----------------
Arghhhhhhhhh!!!!
Trying to get SWIFT lib to compile for math calculations. It's not working.
6/15
-----------------
More brainstorming for project ideas
1. something to analyze cartoon actions?
- is there a physical model for the "cartoony motions"?
- can we numerically model how 2 characters interact and also interact with their environment and ground?
- can we find methods to extrapolate beyond sample space?
2. an engine to generate new motions in a physically based world
- based on experimentation and learning
6/16
-----------------
More brainstorming, online searching, expanding 2 ideas.
Library trip - got anatomy and physics books. Reading up and making notes.
6/18
-----------------
Made a list of sample cartoons and sample cartoony actions for idea inspiration.
Online research of the "Cartoon Laws of Physics".
Reading through "The Illusion of Life: Disney Animation"
Question that came up: How do we separate motion into style of that character's walk, and effects of physical laws?
6/20
-----------------
Talked to Nancy. Decided to focus on idea of cartoony motion. Yay!
6/21
-----------------
Some key points of notes gathered while reading "The Illusion of Life: Disney Animation".
- being able to express weight is key
- stretch and squash emphasizes interesting parts of motion
- instances of breaking physical laws - defying gravity when scared/pricked by sharp object
- instances of bending/exaggerating physical laws - careening around a corner (based on speed and mass)
- heavy things make an arc when they move (nearly all things travel in smooth curves)
- much of the effective exaggerations are based off of emphasis on an object/character's weight
- anticipation and follow through also help to emphasis the moment right between them
6/23
-----------------
Through looking at cartoony walks, I noticed that the head, torso, and root of the body move in smooth, sinusoidal-like movements in the y direction. A spring-mass system might be a good model of this. Skeleton will have to be deformable.
Brainstorming about deformable skeletons, and a springy system with attached weights that responds to push and pulling.
Reading through physics books to brush up on spring systems, in case I have to program a physically based system.
Traced some images sequences from "The Illusion of Life: Disney Animation", and marked the change in center of mass of the characters as they moved.
Talked to Nancy, and got some insight/advice.
- create something that will add "cartoonyness" to motion capture data
- program as a plugin for Maya - (ease of use will be important. possible to have it be real time? exaggerating the motion as it's being captured.)
- ask Jessica for insight about coding the Maya plugin
In the mean time, messing with Maya to try and set up a spring-mass system.
6/29
-----------------
The spring-mass system using basic Maya tools failed spectacularily. Don't think there's an easy way to do this in Maya.
More brain storming... In traditional cartoons, exaggerated walks have about the same "mean pose", but the low position is more low, and high position is more high.
In order to exaggerate mocap data (just walks for now)
- take changes in joint angles and scale them up (naive... but would this work?)
- more head bob (up and down)
- skeleton expand and extend
- how much control for user?
6/30
-----------------
Looking at motion capture data.
Here is the head position over time for one walk
frame number: 44 -> 76 -> 107
not as sinusoidal-like as the cartoon walks
7/1
-----------------
Borrowed a few Mickey Mouse tapes from Jessica Hodgins.
Found a few good clips for reference:
Mickey walks onto stage
Donald's dance
Donald hitches up pants (stretch and squash)
Donald yanked off stage
Whole week spent on
- converting video clips from VHS to digital format
- using Maya to rotoscope Mickey walking motion (with human motion capture skeleton) from video clip
(very time consuming! and results were not terribly smooth)
7/6
------------------
Okay... I have a rough Mickey Mouse walk motion now... how do I export all the joint angles?
Trying export clip. I have to figure out the .ma (Maya ASCII) file format. ...Do I have to write a parser for the .ma file?
7/7
------------------
... Running into some major problems with Maya... what is libSharedUI.so?
7/8
------------------
Help Desk helped solve problem. Yay. Back to work.
Some bone lengths to look at:
lowerback to upperback
upperback to thorax
rfemur to rtibia
lfemur to ltibia
rtibia to rfoot
ltibia to lfoot
Center of Mass study
Here is the distribution of weights I will use for now to calculate Mickey's center of mass. Calculated based on approximations of Mickey's body part shapes, and the assumption he stands about 3 feet tall in real life, and that his body parts are of uniform density approximately equal to that of human flesh.
Code written so far:
- mel script to export all joint data
- java code to load joint data
Code I need to write:
- traverse skeleton and convert relative joint data to global position/orientation data
Talked to Nancy. Got some suggestions.
- ask Mo for a walk sequence with stretch and squash motion. It might be a lot smoother to work with.
- look for center of mass popping in Mickey walk
- read through Hodgins, Pollard 1997 paper "Adapting Simulated Behaviors For New Characters" for body mass numbers for a normal person. (for comparison)
7/14
--------------------
Finished java program that traverses skeleton and gets global positions/orientations of joints.
Jessica invited Jay (motion actor) to do some motion captures for her, and suggested that I have Jay try to mimick some of of Mickey's motions for comparison.
Compiled list of motions:
- Mickey walk on stage
- Mickey sneak walk
- Mickey casting spell
- Mickey dance1
- Mickey dance2
- Mickey conducting symphony
- Mickey surprised
Motion capture session went well =)
7/20
------------------------
All motion captured Mickey Mouse walks have been cleaned, and data is usable. The two Mickey dances have noisy arm data because markers are blocked when Jay's arms are behind his back.
Beginning to analyze captured motion with old code used on old rotoscoped walk data. Found that...
- Joint angular acceleration data is EXTREMELY noisy, will have to be filtered.
7/23
-------------------------
Even after filtering acceleration data, there's a spike in the graph...
Nancy suggested filtering the velocities also.
In the mean time. Working on derivation for bivariate linear regression. If we imagine that the bone connecting two body masses is like a spring, then we want to find some relationship between the length of the spring, and the velocity of the moving body masses, and the acceleration of the moving body masses.
Springs abide by the rule F=m*a= -k*d - b*v
Ideally, we would like to find that the way a particular cartoon character stretches and squashes correspondes certain values of k and b (the spring stiffness coefficient and damping coefficient). This would make it clean and easy to transplant the same springyness/cartoonyness to a different skeleton, and different motions.
If we can use bivariate linear regression to approximate the values of k and b from the rotoscoped mickey walks, then we can use those values of k and b on the motion-captured data. We can calculate new values of d and alter the bone lengths over time, making it cartoony in the same way as the rotoscoped character.
But right now... something's wrong with the derivation...
7/27
-------------------------
Got the derivation right!! Coded it up, and getting some sample k and b values for Mickey's springyness. The original data was very hard to fit because acceleration data is so noisy, so I'm not sure if the approximation is reasonable.
Some on-going questions when dealing my rotoscoped data...
- there are still spikes in the data graphs at frame 139 and 159 of rotoscoped motion... maybe my ghetto keyframes are just too noisy
- the movement of Mickey's torso and hips look like the motion of two coupled springs?
- what are good rest lengths for each "spring" that connects major body masses? just take the middle of the range?
7/29
------------------------
Looked at references on physics of coupled oscillations. Not too helpful...
Inserting the springyness into the torso motion captured walk. It looks interesting. I'm not sure if this is what we're going for, but it's a good start. There is an issue with the nature of the skeleton hierarchy and how the feet go through the floor now that the hips are lowering...
8/8
------------------------
Animation of the mocap walk with springyness doesn't look quite right. Maybe because...
- are k and b values not right because of badly rotoscoped data?
- is the single oscillator model not accurate?
Added a second piece of code to hold feet in place even as hips are moving.
8/11
-------------------------
Got another paper to read
"Animation from video: Cartoon textures" by Christina de Juan, Bobby Bodenheimer, August 2004
8/12
------------------------
Talked to Nancy. Our results so far seem to indicate that cartoon motion doesn't really fit to neatly into a physical system. Maybe we think about cartoony motion as style.
Relevant paper to read, includes Inverse Motion Warping algorithm for motion alignment.
"Style Translation for Human Motion" by Eugene Hsu, Kari Pulli, Jovan Popovic
Coding up a triangle filter to replace box filter code.
8/14
------------------------
Synthesizing some results an working on the presentation!!! We're scheduled to give a 10 minute talk at the upcoming lab meeting.
8/19
------------------------
Found animation book called "Animator's Survival Kit". Very good reference on tradition 2D animation, and tricks of the trade.