Week One
June 14-19

Monday morning I flew in, napped, and made my way to the lab.
My assignments for the week:
  1. Get this web page up and running.
  2. Learn more about the projects going on here, and figure out what I am interested in working on, and what I can help with. On Friday I need to submit a proposal of my summer project to Justine.

There are two main projects going on at the lab: Sam and NUMACK, both Embodied Conversational Agents (ECAs). Sam is an storytelling peer that interacts with children by taking turns with them using toys to tell stories. Sam is displayed on a large screen behind a playhouse, where toys can go back and forth between his virtual world and the child's real world through the attic of the playhouse.
NUMACK is designed to answer questions about and give directions around the Northwestern University campus. He understands a question by interpreting a user's speech, expression, and gestures, and answers appropriately, using speech, expression and gesture.

Marc Flury is the main IT person at the lab, and he does most of the code for both projects. Yolanda Rankin is a PhD studend who works on the Sam project, right now collecting and coding African American childrens' stories to take a look at story structures from different cultures. Paul Tepper, also a PhD student, and Stefan Kopp, a post-doc, both work on NUMACK. At the AAMAS 2004 conference at Columbia U. they are presenting a paper they and Justine wrote on ECAs and gesture perception. Chelsea Robertson is another DMP intern working here for the summer. This is her 4th week into the internship. Visit her DMP website to learn about her experience. Also visit the Articulab website for more information about the projects and the people.

Tuesday morning I went with Justine and Yolanda to a meeting at a nearby preschool-kindergarden school to set up a pilot Sam project for the summer. The teacher there was very enthusiastic about it, so hopefully that will work out. On Wednesday I moved into Inglehart with Chelsea, where we are staying until our dorms open on Sunday. On Friday morning another woman working at a charter school came in to get a demo of Sam, and she's also interested in working with us on the project.

Week Two
June 20-26

At the end of last week I proposed my research project for the remaining time, and Justine and I sat down and looked it over. The goal is for this proposal to keep expanding until it morphes into my final paper at the end of the internship. My project is to start the content planning category of the NUMACK project. I will design a route planner so that when a use asks how to get from A to B, NUMACK can come up with an appropriate answer: the route from A to B. On Monday I spent the day revising my proposal and specifically searching for lots of prior research on the subject.

Chelsea is doing a similar project. Once NUMACK comes up with the route from A to B, she is designing a map of the route that the user will see, which will complement NUMACK's dialogue.

Paul and Stefan ran an experiment that consists of 28 videotapes, each are of two people sitting in a room, one person familiar with the Northwestern campus, and one who isn't familiar at all (most of the time this person was either Marc or Paul, pretending not to be familiar with campus). The two people are given a task. The person familiar with NU needs to explain directions to 3 different checkpoints around the campus, then the person unfamiliar with the campus should be able to get to the checkpoints with no problem. The direction giving interaction between these two people is videotaped from four different angles to watch the gestures made by the direction giver.

Chelsea and I spent Tuesday and Wednesday and part of Thursday watching these videos and got through about half of them. We have a map of the campus in front of us during each video and highlight the route that the direction giver indicates, including every landmark mentioned, i.e. buildings, bikeracks, the lake. With this information we can figure out what landmarks are mentioned the most, and so what to include in our routes. The plan is to eventually be able to predict the types of landmarks that will be used in any route, so I can set up my route algorithm that way.

My family lives in Portland, OR, and every couple of years we fly out to Maryland and Delaware for 3 weeks or so, where both my parents grew up, and where most of both of their brothers and sisters and their parents still live. It's a giant vacation, and I'm missing it this year, so Thursday afternoon I flew out to join them for and extended weekend. It was a great time!

Week Three
June 27-July 3

I flew back in to O'Hare Monday. I got really sick on the plane, which doesn't usually happen to me...I was sick for the rest of the night. I figured it was a motion thing, so Tuesday I went to work. It turned out to be a bad idea, I was still throwing up all over the place, and thankfully Paul came in with a car and was able to drive me back to the dorms. I went to the ER Tuesday night because I couldn't even hold down water and I had a fever, and my ear also had started throbbing. It's so odd...no one knows how I contracted all these things. Except that I went swimming on Friday and then flew in an airplane on Monday...

Wednesday I stayed at the dorm but worked on the webpage a little while I wasn't sleeping. Thursday I went in to work for part of the day and watched more videos. I couldn't sleep at all Thursday night and my ear was only hurting worse, so the next morning I went into the ER again, and had a slew of doctors crowd around my ear and not really know what it was, so they sent me to an ear, nose, and throat doctor, who right away knew it was an outer ear infection. Hopefully I will start to feel relief from that soon.

Week Four
July 4-10

July 4th here was really good. There were city fireworks on the beach, and it was a bigger show than I've ever seen back home. The people operating it just must have been having a blast. I can imagine them being excited for parts that went well or dissapointed for duds. Even though I'm sure the crowd could barely tell the difference.

On Wednesday I'm scheduled to give a 10 min. presentation at the lab about my project. I spent Sunday and Monday searching for more prior research. This part of research is a lot of work, because it takes time to simply find a paper that relates to the topic, and many times the only way to know if it relates is by reading it, and the times it takes to do all this adds up. So far I have 5 papers that are good examples of prior research, that is they relate well to my project, and I can reference them in my paper.

On Tuesday I went to the hospital a third time because now that the one ear was starting to feel better, the other ear suddenly got infected. What.
But before I left the lab to do that, we had a long discussion about the representation of the route that I'm planning, what exactly I should be figuring out on the fly and what I should be grabbing from the already existing knowledge base.

Wednesday's presentation went well, I mean it was only in front of 7 people or so. Chelsea and I are going to create a spreadsheet of how many times each video subject giving directions from Frances Searle to Allen uses each landmark that is used. This will be the hard data I use for my route planner, and what she uses for designing her map.
After the presentation Chelsea and I headed over to the CS building where they were having a graphics open house. Four main projects were setup on display. One covered realtime interactive rendering, including shadows and reflections. There was a large scene of a kitchen with metal pots and metal chairlegs, and as the point of view of the user changed (like getting closer to the table, or walking through the room) the shadows and metal reflections changed appropriately. Really interesting.

For the rest of the week Chelsea and I went through each video again to make sure we'd written down everything, then we created the spreadsheet. I was also busy with prolog and trying to figure out my route algorithm. On Saturday a random guy from the dorms offered to pay me to tutor him on his Linguistics homework! I explained to him that I've never taken a linguistics class, but he said it was math oriented, so I looked at it and it was all logic and semantics, really interesting stuff. So I helped him and made a little extra cash. Funny.

Week Five
July 11-17

Sunday was great fun, almost everyone from the lab went downtown to watch USA vs. Poland at Soldier field. Here's a fact for you: Chicago has the largest Polish population outside of Warsaw. The stands were clearly dominated by Polish fans (meaning...we saw 1 section in the stadium who looked like they were mostly US fans), who chanted "Polska!" and threw streamers, blew up balloons; they were decked out. It was cool to watch. USA and Poland tied 1-1...

I was approved for DMP funding to attend the AAMAS 2004 conference in NY a week ago or so, but I've been afraid to sign up because of how debilitating my last flight experience turned out to be. I hate being sick. I can't afford to get sick again, for the sake of this project. But I think really I'm being ridiculous about it, so I signed up for the conference, a flight, and a hotel. Looks like I'm flying to New York next Monday!

On Tuesday I got up at 7:15--2 hours earlier than I usually get up, mind you--and went with Yolanda to REBA Early Learning Center. We spent the morning there collecting stories from the kids, which will be eventually coded into Sam. We had them play in a room with a wooden dollhouse, and a video camera and tape recorder were capturing their play.

Paul brought in his prolog book this week (which he's been meaning to do), and it is incredibly helpful. The book is Computational Intelligence: A Logical Approach by David Poole, Alan Mackworth, and Randy Goebel. It's actually an AI text book, but all the implementations and examples are done in prolog. This is a well written text, I recommend it. It's helped with a lot of my circular logic problems, and my route algorithm is really developing. By the end of the week, it almost works.

Week Six
July 18-24

Monday morning I flew to New York to attend the AAMAS 2004 conference. I meant to go early so I could mill around New York for the day, since the workshop I was going to wasn't until Tuesday. But I took Dramamine for the flight in case of nausea, since the last time I flew it didn't work out so well. I pretty much zonked out when I got to the hotel and slept for most of the day. Too bad.

Paul, Stefan and Justine are co-authors of Content in Context: Generating Language and Iconic Gesture without a Gestionary. Paul presented this paper at the Tuesday workshop, ECAs in the Loop: Balanced Perception and Action. I'm really glad I went to the workshop, and here's why: I learned a lot about what other people working on this subject are doing around the world. In fact, most of the research isn't from the United States, because most US funding comes from DARPA, and work in this field isn't normally categorized as helpful for military defense. Places outside the US are more likely to provide funding. Another reason I'm glad to have gone is that I learned a lot about a 'good' presentation. Too often someone might have been presenting a paper with a strong topic, but they were so monotone that barely anyone was following. Hearing so many presentations in a day, it's hard to follow if it's not engaging. Alternatively, there were presenters who didn't have good empirical evidense for their thesis, or were simply stating 'here's what we programmed' without any background at all. Paul did a good job, his presentation was captivating and he obviously knew what he was talking about. He answered questions asked of him concisely. So...kudos!

I was also happy about the trip because it was really my first time flying into somewhere and booking my hotel and getting places by myself, and it was in New York which made it awesome. The place is so dense. I was staying a few blocks south of Time Square. Paul and Stefan were the only other two people I knew in the city, and they made arrangements for themselves long before, so I was staying somewhere completely different and a bit farther a way from the conference. Anyway, it was kind of peaceful to just be somewhere in the thick of things.

So I was back at the lab on Thursday and Friday and the route alg is coming right along! It works and I fixed the cycles problem. I think it still might die if a route from A to B doesn't actually exist, I should check that one out. I'm not sure what the coding edicate says...I mean there shouldn't ever BE a case where a route doesn't exist, but probably something should be hardcoded into the alg just in case the non-existent route error ever happens.

Chelsea and I also created spreadsheets for all the rest of the subject data. We need to come up with a hypothesis about routes and landmarks in general. Given some points A and B that we haven't seen yet, we should be able to predict a route and landmarks that someone would use for directions from A to B. This is high priority for Chelsea and me since she's leaving at the end of next week :(

Week Seven
July 25-31

Sad! Our friend Lauren, who lived across the hall in the dorm, left this weekend. Now we don't know what to do with ourselves :(
Sunday we ate with a bunch of people from down the hall and it was a good time. On Wednesday we went to dinner with some Northwestern U. computer science undergraduates. It was fun, we should do it again. A couple of them had just graduated, and were either going to grad school or getting a job for the next year. Wow, real life is getting closer. Then on Thursday we went to dinner with hall people again, and again it was good times.

Justine is still out of town this week. She left at the beginning of last week for a conference in Barcelona, and then she went on a fun bike trip ...somewhere over in Europe. I forget. She won't be back until after Chelsea leaves on Friday, so that's too bad. But the lab was completely empty for the first part of the week, because everyone was on vacation, so Chelsea and I ended up working from the dorms for a day or two.

Right now the route program uses a lowest cost first algorithm, and I can alter it a little to include a heuristic, making it an A* search. I have to figure out what my heuristic would be. Chelsea and I were supposed to come up with a theory about routes, using the data from Stefan and Paul's experiment. Of course, the point of the experiment was to collect gestures that people made while giving directions, and we learned that Stefan had actually told them how to get from one place to another if they weren't familiar with it. So, a route theory is kind of out of the question.

Chelsea left Friday. Good luck with school and GRE's and such friend!

Week Eight
August 1-7

The week was busy. Busier than expected. I thought I would wallow in my aloneness in the dorms and eat by myself every night, but it hasn't happened. Good.

I've made a lot of changes to the route program. Changed the vertices of the graph to be slightly arbitrary points on paths of the route, and from those points information about the landmarks around it will be available. Also, I implemented a coordinate system with norh-south and east-west being the two axis, and each building and route vertex at a location on the graph. This way, the difference in their coordinates can reveal a relative direction of one in terms of another.

Thursday and Friday were spent downtown at the 2004 Cog Sci conference. I am so glad to have experienced that. There were a lot of great sessions, and good books that I wanted to get. Justine had a talk on Friday morning in one of the sessions. She is a great speaker, and people were so interested in the research going on at our lab! After the session she was bombarded with questions of interest. And I met a lot of Cog Sci and Linguistic people. Really cool.

On Thursday night the lab went out to dinner at a really good Ethiopian place in Chicago. It was Stefan's last night in Chicago. He'll be on vacation for a few weeks and then back in Germany teaching. And still long range working on NUMACK. We all hate to see him go. Good luck with everything!
The lab is slowly depleting.

Week Nine
August 8-14

This week I spent a lot of time reading up on discourse to see clearly what part my project plays in the big picture of NUMACK. That is, I learned the details of Content Planning and Microplanning so that I can discuss what I'm doing concisely in my paper. I also had a long discussion about discourse with Paul and he helped by clarifying quite a few things.

So there are two components of Content Planning; there's Knowledge Representation and then there's Planning. The Knowledge Representation in this case is both the way that information is understood in the route planner and delivered to the next module, and also way the domain of information is represented in the database. Planning is the part of the route planner that runs the algorithm to figure out the information to pass on to the next module.

This week also brought four birthdays of my friends from home. Happy Birthday David, Emily, Sonia, and Amy! We've got two 20's, an 18, and a 22 year old!

Also, on Friday my friend Joel from Portland was travelling through Chicago so he came with me, Rachel, and Aaron to "Too Much Light Make the Baby Go Blind", a comedy show downtown. It was great, I completely recommend it to anyone coming to Chicago and looking for a good comedy show.

Week Ten
August 15-21

Wow, the 10th week. I am sad to go! For the beginning of the week I went over to Andrea's house after work to eat and watch the olympics! The olympics impress me. They bring everyone together for some good clean sports fun. It sounds odd to say, but it leaves me with a good feeling about humanity.

I had another presentation at the lab meeting on Wednesday in which I presented my project again as it is now at the end of my stay here. It has come a long way. This week I finished another couple of predicates so that the route planner returns a description of the route by points on the route that the traveller passes over as well as landmarks that can be seen from each point, the direction to look for those landmarks, and the turning direction of the traveller at each step. Good.

This week I also went to see the movie Garden State with Yolanda and Rachel. It was a great movie for my generation, and was written by, directed by, and starred Zach Braff, who is a relatively young guy and went to Northwestern University. I'm impressed, and that's good PR for Northwestern.

Friday night was my going away dinner at Jusitne's house. We ate good! We barbecued with lots of goodies, like corn, mushrooms and sausages. And we all got to meet Justine's addition to the family, a puppy named Inca! She is absolutely adorable. Thanks for the dinner everyone!

I had a great 10 weeks, even with the spout of sickness. I will miss the cuteness of the town of Evanston. I will miss the 'L' and Chicago, and especially I will miss the Articulab, Justine, and everyone who I met through this experience. Thank you all for a great summer!