Emily Yu Williams College Class of 2011 Email Me Home

Another feature of the Camera Mouse software is a smoothing device that smoothes the path of the cursor. I ran six tests: one with the mouse, one with the camera mouse with the smoothing feature off, one with the smoothing feature on low, medium, high and extreme.

Continuing this, I ran the test again, but used all seven smoothing settings: very low, low, medium, medium-high, high, very high, and extreme, as well as with the smoothing off and with the camera mouse off. I found that the smoothing effect did help to make the trajectories of the mouse straighter, but at the same time made the mouse much harder to control. As a result, I had to go much slower to prevent overshooting for each button. The extreme setting also resulted in 3-4 extraneous mouse clicks due to the speed at which I had to go. For all other settings, there was, on average, a 2-5 second difference between the time needed to complete the task on the current setting and the time needed on the previous setting. Between very high and extreme however, the difference was 23.36 seconds. At this point, I'm not sure what I will be going with this data.

Another task I started this week was working on adding blink detection to the Camera Mouse. This turned out to be a very difficult task for me. There is some code existing for blink detection, but it is in C++ and the Camera Mouse code is in C#. Both of these languages I saw for the first time when I started in May. The method I am trying to implement takes the video frames and finds the difference in brightness. Therefore, if one stays relatively still and blinks, the area with the biggest difference is where the eyes are.

The way I have it currently implemented is quite problematic because it takes much too much time to process the image. Also, I have not found a way to make a bitmap of the image using brightness values yet.

After coming back from a week and a half vacation in California visiting relatives, I resumed my work on blink detection for the Camera Mouse. After finding an equation online to convert color images to grayscale (red component * .3 + green component * .59 + blue component * .11), I made difference images by finding the grayscale component of two consecutive frames and subtracting them. This way I was able to create bitmaps of the difference image.

When testing this out, I found that this method was good at finding the eyes when everything else was still and just the eyes were blinked. The resulting difference image generated is primarily black, with white spots at the locations of each eye. When the head is moved, however, the eyes cannot be located because there is too much other movement that causes white space on the difference image.

One major problem that I ran into was that generating these difference images every five frames took a long time to create, causing the video image to lag behind real time severely. To speed up the process slightly, I added a short method to return the image as a byte array instead of a bitmap. This way, each individual pixel could be accessed more easily because indexing from the array is faster than calling the getPixel method on the bitmap every time.

To detect the blinks, we (graduate student John Magee and I) decided sum the pixel values of the rows and columns. This way, a blink can be detected when there are two isolated peaks. This can be differentiated from head movement because this causes a bunch of peaks.

Blink:



Head Movement:



Back