Helena Wotring

My Lab Notebook Home page | About Me | Mentors | The Project | Project Journal | Summer Journal | Final Report | Links

Interaction Lab

Day one.

I've been working on getting the Esra head able to speak.  I started out using ATT's trial speech to text driver and played around with getting it to speak in time with the sound file.  After getting this to work, I decided that the easiest way to get this to work would be to map the shape of the mouth to the English phonemes.  After getting this more or less done nicely, the remaining thing was to get them fine-tuned so that they looked correct when they were mapped to speech.  After a few days, I think I've finally gotten it about where I want it to be.  Now I just need to figure out how to do what I just did with the Windows interface with C++ and Linux.  On the actual robot I'm using Festival to generate the actual speech, I just need to figure out how to get it to work.

For a short video of the Esra head working, click here.  The voice clip I used was of Manny from A Bug's Life.  The reason that the arms are "not working" in the demo is that for the actual Esra head that will be going on the robot, there will be no arms, but the eyes will move.  Accordingly, I did not bother practicing on getting the arms to work.

Well, getting Festival to work on my machine is turning out to be a bit of an adventure.  I've mainly been trying to get a simple API to work with Festival to generate some basic speech samples, but it seems that while Festival was installed it didn't actually compile any of the libraries it needed.  In addition to that, it now turns out that the sound card on my computer is not configured properly.  In theory it should work now, jut the actuality of that assumption has yet to be seen.

I finally got Festival to do what i wanted it to do.  Basically, I give it what I want it to say, and it tells me what Phones it used and their durations.  The happy dance did commence.

Since getting Festival to work properly, I have created a program that will read in the Phones used to generate the wave file and in time with the wave file will send commands to the ESRA head to move.  To do this much easier than trying to create a separate mouth shape for each of the 44 phones used by Festival, I found a website that grouped the phones into phoneme groups for their use in generating an animated talking head.  While I had to adapt these mouth shapes for the ESRA head since I have a more limited capability to shape the face than a computer animation, this served as a great springboard for my work.

I have now officially gotten a pat on the back for my work. Now, in addition to having my API say text through Festival and have the mouth correctly synced with what is being said, it will make faces. What I mean by this is in a text file which will be said at a specific stop on the tour, there can be various emoticons (ie :) ) within the text. At this point, the ESRA head will pause and make that face.

When I got my pat on the back was once I finally got all of this to work, and I had all of the lab's permanent researchers standing around my desk watching my test output and the ESRA head speaking along to the text and making its faces along the way. It was a lot of fun. Now I just get to play around with getting my IO button board to work and messing with Festival's voice so that it doesn't sound so fake...wee.

This past week has been pretty slow since I'm waiting on external technologies. We've decided to use AT&T Natural Voices which means that I'll have to re-work a good portion of my program. Fortunately, this will make the program run more directly and a lot easier, but this also means that I get to wait on getting the program before I can start adjusting for that. Also, when I was working on my IO board, evidently it didn't like how I wired up the buttons, and I may have killed my Phidget board. Now, it may quite possibly be a waiting game for a while.

As a side note, batteries are becoming the bane of my existance.

Since I've been at a bit of a slow point in working on my coding, I've been helping out with the localization bit of programming. Basically it's currently entailing going through the Player drivers and creating a new driver for it. For our system, we are using Adaptive Monty Carlo Localization. As it is designed, it uses laser scans to determine where it is likely to be, however since the laser draws so much power from the robot (it takes 12 volts from the robot and steps it up to the 24 volts the laser requires), the robot would likely run out of power in the middle of a tour, which is not a desirable thing. Instead, we're using letters on the ceiling and a blobfinder. It's my job to figure out how the AMCL driver works as designed, and now how to adapt it to localize itself based on the letters, or blobs, on the ceiling.

This is going to be messy.

Eventually I was able to sort through the driver information and architecture. As of now, I have the driver reading in the locations of all of the letters on the ceiling based on the ceiling tiles and outputting their actual locations to the data map, and taking data from the blobfinder driver and updating the weights of the poses as it would with the laser. Now I just have to figure out how to get the data from the blobfinder in the first place. This, I have no idea how to do, so it should be interesting. I certainly hope it doesn't require creating new drivers, because it's just so messy. I was just rejoicing when I got it to compile in the first place.

Alright, what I thought that I had working for updating the weights was a lot more complicated that what I actually needed, so I had to re-do that this week. In addition to this, I got the button pannel (now a single button) working from my computer. Now I just have to work with someone else in my group to get the listener working so that different script events will happen based on when and if the button is pressed. This is going to be a lot of fun, and by fun I mean an event that lacks all meaning of the word.

So it turns out that I ended up writing the button pannel code, and it was a lot easier than I thought it was going to be. I'll probably have to update the scripts so that it has a different introduction for each person and not just something like "if this person is here, they'll press yes." That's a job for another day. Right now, I've been trying to get new TTS software ordered so that I can change over the voice driver before I leave, and getting the Player drivers moved over to the robot so that it can actually run on its own. Until those drivers are on the robot, I can't test and fine- tune the robot, so that is of great importance to get done as well.

I've gotten all of the script code updated and working nicely with the button pannel, including updates for more dynamic script transitions. I even demonstrated the code for one of the stops during a tour yesterday. It was quite amusing to see someone's reaction to the talking head that hadn't seen it being developed, and all the tests that worked up to it. I have the TTS order cleared to be ordered, but when it will get here is anyone's guess, probably the day after I leave with my luck. I'm currently working on getting the AMCL drivers to update and work correctly on the robot so that we can get the localization up and running.

VICTORY IS MINE!! The localization driver I've been troubleshooting for the past couple of days is up and working. The reason that the robot appeared not to be working was because the sonar is out. 14 of 16 sonars are supposedly right up against something, which is not very fun when you are trying to work on something completely different. AND the setup for player was in degrees when everything else was in radians, so after fixing that it worked much better as well.

Unfortunately, Geeves is not ready to give a tour, but his interaction is working quite well. I guess after three months, I can't expect it to really be working perfectly, because that would be just crazy talk, but I'm very happy and proud that I got my part working effectively.