Weekly Blogposts

Week 3: Assigned a Research Project!


On Monday, Dave introduced me to my research project. Well, actually, it was Dave, Samantha, Evelyn, and Amy who introduced me to the project. They gave me a very general idea, telling me that I would be responsible for shaping this idea into something more concrete. Essentially, I would be designing the lab's next embodied conversational agent - that is, the next Alex. However, I would need to decide whether this agent would 1. serve mostly the Rapport Team's needs (i.e. be a "Rapport Team agent") or 2. act in a more general capacity, having different modules that would allow the agent to take on certain characteristics based on whether it was used by the Rapport or Alex Team.

Hearing about my research project was incredibly exciting - but also very daunting. As an artist, I love the idea of designing a virtual agent - determining how it would look, actually modeling the agent, creating animations for it...On the other hand, I wasn't sure if I was responsible for the underlying architecture of the agent, which I'm sure is extremely complex in any virtual agent. Dave suggested that I start by interviewing each lab member individually to guage what the lab needed in a virtual agent. Then, I could write a Needs Analysis report (due Friday). This was a great idea; and I proceeded to do just that, scheduling interview times with everyone in the lab.

For the rest of the day, I went back and edited the "completed" creatures based on feedback Samantha gave me. You might be surprised to know that a great deal of "thought" is required for the the creation of these creature pictures. I put quotation marks around the word thought because it's not exactly high-level thinking, more like brain grunt work. Since Samantha's experiment is a choosing task, the two pictures in each diorama must be completely different from each other. That is, if one picture depicts the creature picking up sticks in the forest, the other picture must not only contain a completely different background but also avoid having any of the same color palette (greens, browns, colors of the forest, etc). In addition, for each individual picture, the creature should not blend in with its background (i.e. no green creature against a green or even greenish background). Overall, this means a lot of tedious work that consists mostly of just choosing pictures that will work for the necessary color palette - because, as one soon discovers, many free Google images are either not of high quality or wouldn't match the creatures' art style.

Around 3:30, Samantha took Shannon and me to a GEMS event. The GEMS program is an afterschool program for middle school girls that promotes science/technology. At this event, Shannon and I were able to meet a few of the students and chat to other females from technology/science related backgrounds. To be honest, I am not very good with kids, so the interactions were a little awkward for me. Nonetheless, I enjoyed meeting the students and walking around Pittsburgh. Did you know that Pittsburgh houses the National Aviary? I certainly didn't! I'm thinkig I might bring my parents to see it when they come visit at the end of August.


On Tuesday, I came in with a short list of interview questions. Luckily, however, I was intercepted by Samantha, who suggested I look over a book specifically discussing design techniques. She thought some of them would be applicable to me in my scenario, since I would be inteviewing a lot of people with varying opinions. I was definitely surprised to learn just how methodical design can be; the book goes through hundreds of different methods by which one can brainstorm and then decide on design ideas. For instance, one might use storyboards (a thing which I thought was used exclusively in the art/animation industry) to formalize one's ideas.

Given the wealth of new ideas provided by Samantha's book, I postponed my meeting with Amy to Thursday and instead started browsing the book for relevant ideas. Although I didn't finish reading the book until later that night, I'll interrupt the chronological flow of this post to say that ultimately, I didn't find any applicable ideas. However, the book did help me structure my interview questions in a better way. Instead of a long list of rambling questions, I organized them into categories that would help me (and the interviewee) think through the design process. In fact, I created a survey from the questions I had brainstormed (although I ended up only using them for one or two interviews).

Later that day, we had another Discourse Analysis class. Have I already mentioned how much I enjoy these classes? Although they are not directly CS-related, they promote the same ideas. In discourse transcription/analysis, we "code" behavior into written form. In computer science, we code human intention (i.e. what we want the computer to do) into written form. Needless to say, Justine is an excellent teacher, and the class has taught me a lot about human interactions, particularly the unconscious side of human interactions.


Wednesday was a long day. Because our Discourse Analysis class was canceled on Monday, our class scheduled was altered, such that the Discourse Analysis class on Wednesday was moved to 6 PM.

Before I discuss the Discourse Analysis class, however, I should mention my interviews with Samantha, Calie, and Nikita. All three of them were very helpful - I mean, Samantha even lent me her book to help me prepare! In particular, Samantha (as the leader of the Alex Team) has very definite ideas regarding what she wants in an agent. She imagines an agent that can switch between different appearances and modules. Imagine, for instance, being able to choose whether you'd like an agent who is 8-10 years old, 12-15 years old, or 20-30 years old. While I certainly think Samantha's idea is doable, I'm not sure if I'll be able to figure out just how to do it in 7 weeks. Regardless, the difficulty of the task is invigorating - I like thinking through the complexities of a project, imagining how I might solve them.

Callie and Nikita also offered great input. Callie works mainly on the linguistics aspects of an agent, but her main piece of advice - to keep the agent as flexible as possible for future alterations - really stuck. Nikita had worked with the lab's virtual agents even less than Callie, but her insight as a casual observer were also great.

As for that night's Discourse Analysis class...well, I realized at the start of class that I hadn't done the homework in quite the right way. I was supposed to code for the correlation between a nonverbal behavior and a verbal behavior; instead, I had coded the correlation between two nonverbal behavior. Fortunately, I had noted how these two nonverbals interacted with the participants' verbal behaviors - I just hadn't made it the focus of my analysis. Disregarding my mistake, however, the rest of the Discourse Analysis class was as stimulating as always. Whenever Justine points out behaviors that people manifest during discourse as the result of internal feelings or motivations (anxiety, desire to make a certain point), I can't help but feel like I'm in the presence of Yoda/some other all-knowing, super powerful being. In retrospect, I can't imagine an internship at NetApp being quite as revelatory as my summer has been so far at the ArticuLab. Certainly, I would have gained work experience at NetApp, but there is 0% chance I would have gained any intellectual/academic stimulation like this.


Thursday always feels like the busiest day. This is in part because we have our weekly lab meetings on Thursday. This week, all the lab interns were expected to give a brief update before the meeting; each one of us was allotted approximatley 5 minutes. During my update, I gave a short demonstration of Alex in Unity (with his deformed animations) and showed a few of my favorite creature pictures. During the actual lab meeting, the 'hottest" topic was the results of a recent Rapport Team experiment. The team had run the data through two statistical methods, but were now having problems deciphering exactly what the results meant. Although I don't know the exact details of the experiment, I imagine (based on what I heard during the meeting) that the experiment had involved the rating of rapport across a series of discourses (involving preteens, maybe?). Despite the statistical models being over my head, I always enjoy listening to these discussions, inside glimpses as they are into the writing of scientific papers.

My interview with Justine was scheduled for 1:30 - immediately after the lab meeting. However, given Justine's importance at CMU and the HCI department, she had another appointment lined up. Even so, she was very helpful in clarifying my task during the meeting; as she saw it, I would be building an agent specifically for the Rapport Project - a peer tutoring agent. To build a multi-purpose agent would be, in Justine's words, the stuff of graduate dissertations (in other words, beyond my capabilities). Furthermore, Justine pointed out that Alex worked just fine; there was no need to drastically alter him.

This information helped me shape my interviews with Dave, Amy, and Evelyn - all of which took place after the lab meeting. Dave and Amy described the peer tutoring agent: it would help 12-13 year olds with algebraic math. Although I would only be responsible for the appearance of the agent (i.e. front-end development), it would help to also think about back-end development. They suggested I read existing literature regarding virtual agents to determine what the agent should look like; after all, the race/ethnicity and gender of an agent plays heavily into how certain demographics perceive them. Having worked on a virtual peer tutor before, Evelyn offered additional advice regarding the best kind of interface. Specifically, she advised against a "talking head" environment, in which the agent is a static (or barely moving) image at the bottom of the screen.

Given all this excellent advice, I was ready to really start writing my proposal at the end of the day. I found a few very informative papers regarding the effect of agent appearance on students. Here's the bibliography for my written proposal, which includes the titles I found most useful:

Baylor, A. L. (2009). Promoting motivation with virtual agents and avatars: role of visual presence and appearance . Philosophical Transactions of the Royal Society .
Baylor, A. L., & Kim, Y. (2004). Pedagogical Agent Design: The Impact of Agent Realism, Gender, Ethnicity, and Instructional Role. Intelligent Tutoring Systems , 5-22.
Baylor, A., Shen, E., & Huang, X. (2003). Which Pedagogical Agent do Learners Choose? The Effects of Gender and Ethnicity . Proceedings of World Conference on E-Learning in Corporate Government, Healthcare, and Higher Education 2003 , 1507-1510 .
Karacora, B., Dehghani, M., Kramer-Mertens, N., & Gratch, J. (2012). The Influence of Virtual Agents' Gender and Rapport on Enhancing Math Performance. In Proceedings of the 34th Annual Meeting of the Cognitive Science Society .
Silvervarg, A., Raukola, K., Haake, M., & Gulz, A. (2012). The Effect of Visual Gender on Abuse in Conversation with ECAs. Intelligent Virtual Agents , 153-160.

I spent the rest of the night creating my powerpoint and writing up my proposal, both of which I will post on this blog.


Worked on creatures. On Friday, I presented my research project proposal to Dave, Evelyn, and Amy. You can view my powerpoint below. It was a great deal of fun hogging the spotlight for the hour or so that I had. What's more, it felt great to know that no matter what kind of agent I designed, it would have some impact on the lab after I left. One of the big issues that came up during my presentation was the matter of environment. From my powerpoint, you can see that there are two options that stand out. One stresses immersion, the other stresses portability. I personally favor the immersive option, but mostly because I'm somewhat selfish and want to design a very "cool" environment (which would be less possible with the portable environment design). Since this is Dave's agent, we decided that he would talk to Justine about the two options, and my design would be based on what he ultimately decided.

Later that night, I went out to dinner with the other lab interns - Angie, Cameron, and Marissa. They're awesome people, and we had a lovely time at a Thai restaurant. My CS teacher from Duke, Professor Peck, stressed the importance of meeting other undergraduate and graduate students during my DREU experience. So far, that's something I've somewhat neglected, so I'm glad that I'm back on the right track.

View my written report here (link to come).