Action Recognition for a Job Coach System in Vocational Rehabilitation

Individuals with cognitive disabilities face great difficulty in education and vocational rehabilitation. Nevertheless, research results indicate that traditional "brain-train" approaches are ineffective in addressing the impact of cognitive disability in general. Therefore, a "job coach" system is set up to provide direct interventions and feedback to address complex work problems, in order to provide a natural environment for vocational rehabilitation for an individual with a disability. However, such "one-to-one" job coaching is often costly and inefficient.

This project is proposed to develop an automatic and remote job coach system that uses the task of cooking hamburgers as a test bed, which can possibly reduce the involvement of a full-time "coach".

Goal:

• Simulate the job coaching system using motion capture data with a limited number of markers.
• Develop an effective approach to distinguish different hamburger cooking actions: placing burgers, seasoning, flipping, pressing and picking burgers off the grill.
• Provide support in a training environment by recognizing the necessary steps in cooking hamburgers and providing guidance when a step is missed.