Introduction
I will be performing this experiment as part of my summer 2005 DMP research. I wish to test an important part of my project – the evaluation metric developed to compare two arm movements.The metric rates the similarity of two arm movements on a scale from 1-10. The algorithm I designed allows the motion comparison of more complex skeletal structures, with a larger number of joints, but this experiment will be testing the metric only for arm movement.
Hypothesis:
When rating the similarity of two movements, the difference between the score given by the evaluation metric and the avearge of the scores given by three human observers will not be statistically significant.
Participants:
Test subjects will be fellow lab members. The handedness of the subjects will not be taken into account. All subjects will have to perform the movements with their right arm.
My movements will be videotaped to be shown as stimuli to the experimental subjects. I will participate in a special session to produce reference data for comparison.
Stimuli
The stimuli consist of 6 video clips, each showing an individual sequence of human arm movement. I will have performed and video recorded the movements. Each movement will start with a straightened arm and typically involve movements of the upper-arm and the lower-arm. The stimuli involve no external objects or recognized patterns, so they can be thought of as goal-independent; the only goal the subjects will have is to imitate the arm movement as accurately as possible.
Apparatus
Stimuli videos will be presented on a 15'' computer monitor.
Subjects' arm movements will be tracked using a custom-made motion capture system developed in the USC Interaction Lab.
Method
The subject and the experimenter will enter a room equipped with a computer and the motion capture suit. Two motion tracking sensors will be strapped on the subject's arm, one above the elbow and one above the wrist. The sensor suit will then be turned on by the experimenter, making sure that the sensors are initiated correctly.
Prior to each recording trial, one of the 6 video stimuli will be shown to the subject on the computer monitor. The trials will be divided into two conditions: "rehearsal" and "no rehearsal". In the "rehearsal" condition, the subject will be asked to imitate the movement while watching the video. In the "non-rehearsal" condition, the subject will be asked to imitate the movement right after having watched the video.
To initiate the imitation process in either condition, the experimenter will give the verbal signal "ready". The subject will then straighten his/her arm vertically. Next, the subject will be given the instruction "imitate", at which point s/he can start imitating the movement shown in the video (concurrently or just prior the imitation). When finished imitating, the subject will say "done". The movements of the subject will be recorded, through the motion tracking sensors, during the time period between the "imitate" and "done" signals.
Each subject will be presented with 6 stimuli, 3 in the "rehearsal" condition and 3 in the "non-rehearsal" condition. The order the stimuli will be presented in will be randomized across subjects. Each stimulus will be shown and imitated three times in succession.
Evaluation
First, the automated metric will be applied to the imitations, as follows:
After all movements have been captured, the evaluation metric will be run on pairs of files corresponding to the same movement.
Next, the metric results will be applied to human evaluation. Human evaluation will be obtained as follows:
The execution the movements will be videotaped and observed by three unbiased viewers (not the experimenter or any subjects). The viewers will rate how similar the movements are on a scale of 1-10 for each of the imitation trials.
Finally, the scores of the three evaluators of the same experiments will be averaged, and the values compared to the automated metric.
Related Documents
Marc Pomplun and Maja J Mataric´, "Evaluation Metrics and Results of Human Arm Movement Imitation", Proceedings, First IEEE-RAS International Conference on Humanoid Robotics (Humanoids-2000), MIT, Cambridge, MA, Sep 7-8, 2000 [PDF].
Odest Chadwicke Jenkins, "Data-driven Derivation of Skills for Autonomous Humanoid Agents", PhD dissertation, University of Southern California Computer Science Department,Sep 2003 [PDF].
Nathan Miller, Odest Chadwicke Jenkins, Marcelo Kallmann, Evan Drumwright, and Maja J Mataric´, "Motion Capture from Inertial Sensing for Untethered Humanoid Teleoperation", Proceedings, IEEE-RAS International Conference on Humanoid Robotics (Humanoids-2004) , Santa Monica, CA, Nov 10-12, 2004 [PDF] .