Home
About me
About my mentor
Research
Reflections on DMP
Weekly journal
A Day in the Life...

Final report

Research:

I'm tapping into one of Prof. McRoy's running projects, working with a data set that her group for Cross-modal Analysis Signal and Sense Data and Computational Resources for Gesture, Speech, and Gaze Research (otherwise known as GSG for people who can't remember the entire title!) has collected.  She and one of her grad students are working on a scheme to annotate the dialogs semantically, as well as using the DAMSL annotation scheme for higher-level labels.  This data set presents several challenges, including fine-tuning the annotation scheme and establishing inter-rater reliability, developing tools to analyze the output from the Praat program, as well as analyzing the data itself.  I will be working on various aspects of this project as they come up, as well as aspects of argumentation in general.

It has been interesting to learn about Praat, free software developed by Paul Boersma and David Weeninck at the University of Amsterdam.  It can be downloaded from www.praat.org.  I've learned a little bit about writing and installing Praat scripts.  However, most of Praat's functionality is geared towards finer-grained issues like phonetics and phonology (since that is what it was developed for in the first place).  I did some searching, but was unable to find tools that would work directly with Praat to do the kind of analysis we are interested in (namely comparing user-created tiers, counting bigrams within and across tiers).  Luckily, Praat makes it easy to produce plain text files that can serve as input to other programs, and can easily read in from properly formated text files.

Hence, my mission for the summer is to develop tools to work with the data that the GSG group has collected.  Most work in the NLKRRG lab is currently done in Java (advantage: write once, run anywhere; disadvantage: can be extremely slow), so I spend my days writing Java code.  I'm calling the project "Multi Tier Comparer" (MTC).

For a more comprehensive description of the operation of the tool, look at the User's Manual.

For anyone interested in the "how" part of the tool, or if you just want to see something that looks impressively professional, check out my documentation generated by Javadoc.  It looks just like the API on the Sun website!

Here's how my program might be used in the context of a research project (* indicates steps in which my program is involved):

• Data is collected (conversation between two people is recorded and captured to a digital sound file).
• Transcript of dialog is created and aligned with digital sound file in Praat.
• One or more people create Praat annotation tiers and annotate the dialogs using one or more annotation schemes
• Praat tiers of interest are exported to text files using a customized Praat script.
• *If more than one annotator was used, use MTC to read in the text files and display them in a table.  One or more people choose which annotation to accept in each case where there is a discrepency.
• *To find patterns in the data, use MTC to read in a text file of one label set and count bigrams within a specified window.  The results are sorted by frequency.
• *To find patterns in the data, use MTC to read in two text files and count co-occurances of labels between tiers.  For example, find out the frequency of repair points occuring at the same time as any given semantic label.
• Look for interesting patterns within the data set.
• Look for interesting patterns across data sets.
• Collect as much annotated data as possible and possibly use it as a corpus of examples to "teach" a system how humans argue.  Or extract descriptive "rules" of argumentation and build a system around these.