Monday morning I flew in, napped, and made my way to the lab.
My assignments for the week:
1. Get this web page up and running.
2. Learn more about the projects going on here, and figure out what I
am interested in working on, and what I can help with. On Friday I need to
submit a proposal of my summer project to Justine.
There are two main projects going on at the lab: Sam and NUMACK, both Embodied
Conversational Agents (ECAs). Sam is an storytelling peer that interacts with
children by taking turns with them using toys to tell stories. Sam is displayed
on a large screen behind a playhouse, where toys can go back and forth between
his virtual world and the child's real world through the attic of the playhouse.
NUMACK is designed to answer questions about and give directions around the
Northwestern University campus. He understands a question by interpreting a user's
speech, expression, and gestures, and answers appropriately, using speech,
expression and gesture.
Marc Flury is the main IT person at the lab, and he does most of the code for
both projects. Yolanda Rankin is a PhD studend who works on the Sam project,
right now collecting and coding African American childrens' stories to take a
look at story structures from different cultures. Paul Tepper, also a PhD
student, and Stefan Kopp, a post-doc, both work on NUMACK. At the AAMAS 2004
conference at Columbia U. they are presenting a paper they and Justine wrote on
ECAs and gesture perception. Chelsea Robertson is another DMP intern working
here for the summer. This is her 4th week into the internship. Visit her
to learn about her experience. Also visit the
Articulab website for more
information about the projects and the people.
Tuesday morning I went with Justine and Yolanda to a meeting at a nearby
preschool-kindergarden school to set up a pilot Sam project for the summer. The
teacher there was very enthusiastic about it, so hopefully that will work out.
On Wednesday I moved into Inglehart with Chelsea, where we are staying until
our dorms open on Sunday.
On Friday morning another woman working at a charter school came in to get a demo
of Sam, and she's also interested in working with us on the project.
At the end of last week I proposed my research project for the remaining time,
and Justine and I sat down and looked it over. The goal is for this proposal
to keep expanding until it morphes into my final paper at the end of the
internship. My project is to start the content planning category of the NUMACK
project. I will design a route planner so that when a use asks how to get from
A to B, NUMACK can come up with an appropriate answer: the route from A to B.
On Monday I spent the day revising my proposal and specifically searching for
lots of prior research on the subject.
Chelsea is doing a similar project. Once NUMACK comes up with the route from
A to B, she is designing a map of the route that the user will see, which
will complement NUMACK's dialogue.
Paul and Stefan ran an experiment that consists of 28 videotapes, each are of
two people sitting in a room, one person familiar with the Northwestern campus,
and one who isn't familiar at all (most of the time this person was either Marc
or Paul, pretending not to be familiar with campus). The two people are given
a task. The person familiar with NU needs to explain directions to 3 different
checkpoints around the campus, then the person unfamiliar with the campus should
be able to get to the checkpoints with no problem. The direction giving interaction
between these two people is videotaped from four different angles to watch the
gestures made by the direction giver.
Chelsea and I spent Tuesday and Wednesday and part of Thursday watching these videos
and got through about half of them. We have a map of the campus in front of us
during each video and highlight the route that the direction giver indicates,
including every landmark mentioned, i.e. buildings, bikeracks, the lake. With
this information we can figure out what landmarks are mentioned the most, and
so what to include in our routes. The plan is to eventually be able to predict
the types of landmarks that will be used in any route, so I can set up my route
algorithm that way.
My family lives in Portland, OR, and every couple of years we fly out to
Maryland and Delaware for 3 weeks or so, where both my parents grew up, and
where most of both of their brothers and sisters and their parents still live.
It's a giant vacation, and I'm missing it this year, so Thursday afternoon I
flew out to join them for and extended weekend. It was a great time!
June 27-July 3
I flew back in to O'Hare Monday. I got really sick on the plane, which doesn't
usually happen to me...I was sick for the rest of the night. I figured it was
a motion thing, so Tuesday I went to work. It turned out to be a bad idea, I
was still throwing up all over the place, and thankfully Paul came in with a car
and was able to drive me back to the dorms. I went to the ER Tuesday night
because I couldn't even hold down water and I had a fever, and my ear also
had started throbbing. It's so odd...no one knows how I contracted
all these things. Except that I went swimming on Friday and then flew in an
airplane on Monday...
Wednesday I stayed at the dorm but worked on the webpage a little while I wasn't
sleeping. Thursday I went in to work for part of the day and watched more
videos. I couldn't sleep at all Thursday night and my ear was only hurting worse,
so the next morning I went into the ER again, and had a slew of doctors crowd
around my ear and not really know what it was, so they sent me to an ear, nose,
and throat doctor, who right away knew it was an outer ear infection. Hopefully
I will start to feel relief from that soon.
July 4th here was really good. There were city fireworks on the beach, and it
was a bigger show than I've ever seen back home. The people operating it just
must have been having a blast. I can imagine them being excited for parts that
went well or dissapointed for duds. Even though I'm sure the crowd could barely
tell the difference.
On Wednesday I'm scheduled to give a 10 min. presentation at the lab about my
project. I spent Sunday and Monday searching for
more prior research. This part of research is a lot of work, because it takes
time to simply find a paper that relates to the topic, and many times the only
way to know if it relates is by reading it, and the times it takes to do all this
adds up. So far I have 5 papers that are good examples of prior research, that
is they relate well to my project, and I can reference them in my paper.
On Tuesday I went to the hospital a third time because now that the one ear was
starting to feel better, the other ear suddenly got infected. What.
But before I left the lab to do that, we had a long discussion about the
representation of the route that I'm planning, what exactly I should be figuring
out on the fly and what I should be grabbing from the already existing knowledge
Wednesday's presentation went well, I mean it was only in front of 7 people or so.
Chelsea and I are going to create a spreadsheet of how many times each video
subject giving directions from Frances Searle to Allen uses each landmark that
is used. This will be the hard data I use for my route planner, and what she
uses for designing her map.
After the presentation Chelsea and I headed over to the CS building where
they were having a graphics open house. Four main projects were setup on display.
One covered realtime interactive rendering, including shadows and reflections.
There was a large scene of a kitchen with metal pots and metal chairlegs, and as
the point of view of the user changed (like getting closer to the table, or walking
through the room) the shadows and metal reflections changed appropriately. Really
For the rest of the week Chelsea and I went through each video again to make sure
we'd written down everything, then we created the spreadsheet. I was also busy
with prolog and trying to figure out my route algorithm. On Saturday a random guy
from the dorms offered to pay me to tutor him on his Linguistics homework! I
explained to him that I've never taken a linguistics class, but he said it was
math oriented, so I looked at it and it was all logic and semantics, really
interesting stuff. So I helped him and made a little extra cash. Funny.
Sunday was great fun, almost everyone from the lab went downtown to watch
USA vs. Poland at Soldier field. Here's a fact for you: Chicago has the largest
Polish population outside of Warsaw. The stands were clearly dominated by
Polish fans (meaning...we saw 1 section in the stadium who looked like they were
mostly US fans), who chanted "Polska!" and threw streamers, blew up balloons; they
were decked out. It was cool to watch. USA and Poland tied 1-1...
I was approved for DMP funding to attend the AAMAS 2004 conference in NY a week
ago or so, but I've been afraid to sign up because of how debilitating my
last flight experience turned out to be. I hate being sick. I can't afford to
get sick again, for the sake of this project. But I think really I'm being
ridiculous about it, so I signed up for the conference, a flight, and a hotel.
Looks like I'm flying to New York next Monday!
On Tuesday I got up at 7:15--2 hours earlier than I usually get up, mind you--and
went with Yolanda to REBA Early Learning Center. We spent the morning there
collecting stories from the kids, which will be eventually coded into Sam.
We had them play in a room with a wooden dollhouse, and a video camera and tape
recorder were capturing their play.
Paul brought in his prolog book this week (which he's been meaning to do), and it
is incredibly helpful. The book is Computational Intelligence: A Logical
Approach by David Poole, Alan Mackworth, and Randy Goebel. It's actually an
AI text book, but all the implementations and examples are done in prolog. This
is a well written text, I recommend it. It's helped with a lot of my circular
logic problems, and my route algorithm is really developing. By the end of the
week, it almost works.
Monday morning I flew to New York to attend the AAMAS 2004 conference. I
meant to go early so I could mill around New York for the day, since the
workshop I was going to wasn't until Tuesday. But I took Dramamine for the
flight in case of nausea, since the last time I flew it didn't work out so
well. I pretty much zonked out when I got to the hotel and slept for most of
the day. Too bad.
Paul, Stefan and Justine are co-authors of Content in Context: Generating
Language and Iconic Gesture without a Gestionary. Paul presented this
paper at the Tuesday workshop, ECAs in the Loop: Balanced Perception and
Action. I'm really glad I went to the workshop, and here's why: I learned
a lot about what other people working on this subject are doing around the
world. In fact, most of the research isn't from the United States, because
most US funding comes from DARPA, and work in this field isn't normally
categorized as helpful for military defense. Places outside the US are more
likely to provide funding. Another reason I'm glad to have gone is that I
learned a lot about a 'good' presentation. Too often someone might have
been presenting a paper with a strong topic, but they were so monotone that
barely anyone was following. Hearing so many presentations in a day, it's
hard to follow if it's not engaging. Alternatively, there were presenters
who didn't have good empirical evidense for their thesis, or were simply
stating 'here's what we programmed' without any background at all. Paul did
a good job, his presentation was captivating and he obviously knew what he
was talking about. He answered questions asked of him concisely. So...kudos!
I was also happy about the trip because it was really my first time
flying into somewhere and booking my hotel and getting places by myself,
and it was in New York which made it awesome. The place is so dense. I was
staying a few blocks south of Time Square. Paul and Stefan were the only
other two people I knew in the city, and they made
arrangements for themselves long before, so I was staying somewhere completely
different and a bit farther a way from the conference. Anyway, it was kind
of peaceful to just be somewhere in the thick of things.
So I was back at the lab on Thursday and Friday and the route alg is
coming right along! It works and I fixed the cycles problem. I think it still might
die if a route from A to B doesn't actually exist, I should check that one
out. I'm not sure what the coding edicate says...I mean there shouldn't ever
BE a case where a route doesn't exist, but probably something should be
hardcoded into the alg just in case the non-existent route error ever happens.
Chelsea and I also created spreadsheets for all the rest of the subject
data. We need to come up with a hypothesis about routes and landmarks in
general. Given some points A and B that we haven't seen yet, we should be
able to predict a route and landmarks that someone would use for directions
from A to B. This is high priority for Chelsea and me since she's leaving
at the end of next week :(
Sad! Our friend Lauren, who lived across the hall in the dorm, left this
weekend. Now we don't know what to do with ourselves :(
Sunday we ate with a bunch of people from down the hall and it was a good
time. On Wednesday we went to dinner with some Northwestern U. computer
science undergraduates. It was fun, we should do it again. A couple of
them had just graduated, and were either going to grad school or getting
a job for the next year. Wow, real life is getting closer. Then on
Thursday we went to dinner with hall people again, and again it was good
Justine is still out of town this week. She left at the beginning of last
week for a conference in Barcelona, and then she went on a fun bike trip
...somewhere over in Europe. I forget. She won't be back until after
Chelsea leaves on Friday, so that's too bad. But the lab was completely
empty for the first part of the week, because everyone was on
vacation, so Chelsea and I ended up working from the dorms for a day or
Right now the route program uses a lowest cost first algorithm, and I can
alter it a little to include a heuristic, making it an A* search. I have
to figure out what my heuristic would be. Chelsea and I were supposed to
come up with a theory about routes, using the data from Stefan and Paul's
experiment. Of course, the point of the experiment was to collect gestures
that people made while giving directions, and we learned that Stefan had
actually told them how to get from one place to another if they weren't
familiar with it. So, a route theory is kind of out of the question.
Chelsea left Friday. Good luck with school and GRE's and such friend!
The week was busy. Busier than expected. I thought I would wallow in my
aloneness in the dorms and eat by myself every night, but it hasn't
I've made a lot of changes to the route program. Changed the vertices of the
graph to be slightly arbitrary points on paths of the route, and from those
points information about the landmarks around it will be available. Also,
I implemented a coordinate system with norh-south and east-west being the
two axis, and each building and route vertex at a location on the graph.
This way, the difference in their coordinates can reveal a relative direction
of one in terms of another.
Thursday and Friday were spent downtown at the 2004 Cog Sci conference. I am
so glad to have experienced that. There were a lot of great sessions, and
good books that I wanted to get. Justine had a talk on Friday morning in one
of the sessions. She is a great speaker, and people were so interested in
the research going on at our lab! After the session she was bombarded with
questions of interest. And I met a lot of Cog Sci and Linguistic people.
On Thursday night the lab went out to dinner at a really good Ethiopian place
in Chicago. It was Stefan's last night in Chicago. He'll be on vacation for
a few weeks and then back in Germany teaching. And still long range
working on NUMACK. We all hate to see him go. Good luck with everything!
The lab is slowly depleting.
This week I spent a lot of time reading up on discourse to see clearly what
part my project plays in the big picture of NUMACK. That is, I learned the
details of Content Planning and Microplanning so that I can discuss what I'm
doing concisely in my paper. I also had a long discussion about discourse
with Paul and he helped by clarifying quite a few things.
So there are two components of Content Planning; there's Knowledge
Representation and then there's Planning. The Knowledge Representation in
this case is both the way that information is understood in the route planner
and delivered to the next module, and also way the domain of information is
represented in the database. Planning is the part of the route planner that
runs the algorithm to figure out the information to pass on to the next
This week also brought four birthdays of my friends from home. Happy Birthday
David, Emily, Sonia, and Amy! We've got two 20's, an 18, and a 22 year old!
Also, on Friday my friend Joel from Portland was travelling through Chicago
so he came with me, Rachel, and Aaron to "Too Much Light Make the
Baby Go Blind", a comedy show downtown. It was great, I completely recommend
it to anyone coming to Chicago and looking for a good comedy show.
Wow, the 10th week. I am sad to go! For the beginning of the week I went
over to Andrea's house after work to eat and watch the olympics! The
olympics impress me. They bring everyone together for some good
clean sports fun. It sounds odd to say, but it leaves me with a good feeling
I had another presentation at the lab meeting on Wednesday in which I
presented my project again as it is now at the end of my stay here. It has
come a long way. This week I finished another couple of predicates so that
the route planner returns a description of the route by points on the route
that the traveller passes over as well as landmarks that can be seen from
each point, the direction to look for those landmarks, and the turning
direction of the traveller at each step. Good.
This week I also went to see the movie Garden State with Yolanda and Rachel.
It was a great movie for my generation, and was written by, directed by, and
starred Zach Braff, who is a relatively young guy and went to Northwestern
University. I'm impressed, and that's good PR for Northwestern.
Friday night was my going away dinner at Jusitne's house. We ate good! We
barbecued with lots of goodies, like corn, mushrooms and sausages. And we all
got to meet Justine's addition to the family, a puppy named Inca! She is
absolutely adorable. Thanks for the dinner everyone!
I had a great 10 weeks, even with the spout of sickness. I will miss the
cuteness of the town of Evanston. I will miss the 'L' and Chicago, and
especially I will miss the Articulab, Justine, and everyone who I met through
this experience. Thank you all for a great summer!