This week we started off getting to know the environment in which we would be working. The main program we are using for this summer research is a PC game called Second Life. In this first week we had to learn how to code in Linden Code which is the main program used in Second Life. Later this past week we then had to switch over and learn how to use another program called Dragon Naturally Speaking which is a voice program that when used, can translate spoken language to text and vice versa. Within the Dragonspeak program, we also had to learn how to code in Visual Basic script which is somewhat different than normal Visual Basic programming. Finally we topped the week off by beginning to learn how to code in SQL. This program is good for managing various database systems which, for us, will be used to hopefully hold the responses that the computer will give back to the user when spoken to.
By the end of this week, we had a fully functioning voice command program over the Avatars for Second Life in which we could speak a command to them and the avatar would perform the actions correctly. We also were able to achieve a functioning yet basic version of a program that, when spoken to in Second Life, would respond based off various keywords. This only worked for about 2 or 3 statements, but we were able to achieve a computer response with voice back to the user who spoke the initial statement.
This week the rest of the REU students arrived and had to go through the same training procedures that Hugh and I had to the week before. For us this week was more or less a chance to refine the knowledge that we had as well as working on a few more advanced concepts within the various programming languages. Towards the second half of this week we were able to decide more specifically what tasks need to be done and on Friday we were assigned various groups that we would either lead or be a part of for the projects that we are to accomplish. For this summer, the group that I am going to lead is the Voice Recognition group but I am also a part of the group that has to deal with the content flow of the voice recognition software to recognize and interact with a database which will hold the responses of the avatar which will interact with the person. Overall this week was a lot of fun but looking ahead to next week, we are going to be very busy because of a project assigned to us which is a preliminary to the actual project we will be working on.
This week we were busy trying to complete the rest of the "mini" project. By Wednesday we had a working demo however it didn't work for everyone's voice due to the fact that the voice recognition software was synced with my voice. The rest of the week was then refining the other various problems that we had such as camera angles/movement, avatar movement(i.e. getting up out of a chair), voice recognition (including adding new phrases and getting rid of the keyword used to send the text), and getting everything to work with each other. By Friday, there wasn't a demo ready but progress could be seen in the work that we had done in such a short amount of time.
For the next weeks preparation, we were told that we would have to have everything working by Wednesday of the next week in order to demo this to a group of 7th graders participating in a Computer Science camp here at Clemson. For this week though we made a lot of progress towards our goal of a working demo that gives the user the appearance of actually interacting with a virtual character within a virtual world instead of just playing a game with a virtual character.
This week we were able to run the demo for the group of 7th graders that were at a camp for a week here at Clemson. For the most part it ran ok with the exception of some of the female 7th graders due to their voices being higher than expected as well as some speaking too soft into the microphone.
After the demo, it was time for us to sit down and reevaluate what we will do for our final project. We ended up putting CARA on "freeze" so that way we do still have a working demo but we are now switching gears though to work on what will be our overall project for this summer. The last 2 days of this week were spent on brainstorming the new project so that we can have an idea about what we wanted to do and how we were going to achieve our goal.
This week was all about beginning the new project which will be our final project. We did have a few demos for which we had to pull out the first test project and display. Other than that the whole week was devoted to putting together the new project and at the end of this week we had the whole area set up for where the new demo was going to take place as well as a good bit of the logic in place for it too. A lot of this was easier the second time because we had to learn how to do it the first time through. The only thing that is really different between the first project and this one is the way we are doing voice over. This time, instead of using dragonspeak to do the voice recognition, we are using Microsoft's Speech SDK and coding in C#. Overall, this week we made great progress towards completing the 2nd and final project that we have for this summer.
This week we worked some more on this second project. This project is a lot bigger than the first and so progress on it is a little slower. I had to fine tune the grammar files so that the voice recognition was a little better but it seems to be working now.
Other than that we got to celebrate July 4th by only having a half day Friday while Clemson's campus is closed on Monday and I got a chance to go to a family reunion in Louisiana over the weekend.
This week was all about finishing the rest of the project. All that remained for this week was to get the user's view to turn when you arrived at various exhibits as well as fine tuning other small things such as the voice recognition so that instead of only recognizing "yes", it would recognize "yeah" and "sure." We also had to adjust the logic for the whole program so that it would bring you to the correct locations as well as loop around when the demo completed.
Overall the project is looking quite impressive. We have to have it working by Monday so that we can show what we have accomplished. I believe anyone who sees this demo will be quite impressed. After this project is finished all that is left is to start working on the final reports required.
This week we started working on the final documents/posters required of us. We were told there was going to be a poster competition with judges so we are all required to make posters. To go along with these posters is a 2 page paper that expands what information was presented in the poster. For the most part mine is complete, however, I am going to fine tune it to make sure that it is perfect. We were also told that we would have to have those ready to print by next week Monday. The next week I am going to start working on the paper required for DREU.
This week the whole focus was on poster presentations. We finished up whatever posters we were working on and after multiple drafts were completed, had them printed on a 36" X 48" poster paper. When Wednesday came around, we had to practice our presentation in front of the entire group with Dr. Hodges, Dr. Ulinski, and PhD student Toni Bloodworth acting as our judges. On Thursday, we were practicing again except this time they made it more like an actual conference in that they were walking around to each poster at different times so we had to explain it to everyone. Friday was the actual competition with the real judges and other people who were interested in the work that we accomplished over the summer. After all was said and done, they handed out certificates/awards and everyone else said their good-bye's. I am gonna miss everyone that is leaving that I had the privilege of working with.
This week it was just Hugh and I left. Our focus for the remainder of the DREU was to make a video that showcases what our project was all about. After adding a short verbal intro/ending to the various video clips that we recorded using Camtasia, our demo video was complete and all that is left is to pack up. I really enjoyed the time I had during this DREU experience.