Journals
Here you can find weekly journal entries documenting my work and living experiences in Pittsburgh. I started on a Wednesday, but it's much easier to keep track of weeks starting with Sunday, at least for me! I'm actually only spending 11.5 weeks in Pittsburgh (after extending my internship by 1.5 weeks)--the numbering of weeks is not meant to be misleading. Finally, please be forewarned: I have a tendency to go off on tangents! Enjoy. :)
Week 1: 5/25 - 5/28
Just graduated (finally!) on Sunday. So happy. :) I kind of wish I had given myself more downtime before starting this internship... Packing in itself is pretty stressful, and having to check out and immediately head out on an 11-hour drive (it was supposed to be 8 hours, but you never know with traffic) is not my idea of fun. But I'm finally here in Pittsburgh and ready to start!
Everything was pretty easy to set up once I got here; our administrative assistant, Kelly, is great! However, I'm still having trouble setting up printers through the wireless network on my personal laptop. Fortunately, my labmates are all super nice (thanks, Laura!), so I have my readings for next week taken care of. :) They're not too difficult--thank goodness!--so I think I'm getting a handle on what kind of work it is that Prof Rosé and her students are involved in. The first set is about language and identity... reminds me a lot of my philosophy and sociology readings, which is not a bad thing at all! I'm very excited; there's a lot more to computing than sitting in front of a screen and coding, or reading and writing proofs/algorithms/etc. Of course, I knew this already, but it's really something else to see in the flesh the real power and scope of technology, and how it can be used for more than creating applications (and the business thereof). I'm very glad I'm here!
Prof Rosé is so nice too; she even took me out to lunch with a few other students in her lab group the very first day I got here! Wow. It's a little intimidating working around so many brilliant minds, but I hope I can rise to the challenge!
[Back to journals]
Week 2: 5/29 - 6/4
Really enjoying my time so far here in Pittsburgh! Upon graduating, I resolved to be more social and to get out more. I don't want to miss out on anything that CMU and Pittsburgh have to offer!
Work-wise, I've been assigned my project: implementing a phrase-based translation system for Xhosa-to-English. Apparently there's a dearth of Xhosa language/translation resources out there, and it's a bit difficult to determine what is being said in our codeswitching corpus when we don't have anyone on our team who is fluent in Xhosa... We're looking at Vivian de Klerk's codeswitching corpus right now; it's particularly interesting because English serves as the matrix language--interviewees were explicitly asked to converse in English as much as possible. There's not a whole lot of Xhosa in it, in comparison to English, especially in the "Legal Language" portion, so we're focusing predominantly on "Spontaneous Conversation" (pretty self-explanatory). I'm working with two other interns, Laura (another DREU participant!!!) and Kaitlyn, on the project. Fortunately they have a much stronger background in linguistics than I do... Kaitlyn will be working on a morphological analyzer for Xhosa, which I should be utilizing for my own part of the project, and Laura has already extracted the actual Xhosa portions of the corpus into a nice csv file. (I also learned that 'csv'=='comma-separated values'... whodathunkit? :D)
Prof Rosé has me reading African Voices, a volume on African sociolinguistics, which has been pretty useful. With singing, I always had some understanding of how sounds were produced, and I pride myself on my diction (in multiple languages), but it's pretty cool to learn about the real, formal terms for things. Also, need to work on learning how to actually pronounce "Xhosa"! Lateral clicks are hard...
Anyway, as I mentioned before, there aren't terribly many parallel Xhosa-English texts, so Prof Rosé suggested that I start with the Xhosa bible. Another student of hers, David, pointed me towards some plaintext bibles he found online--thank goodness, since I spent a little more time than I'd like to admit copying and pasting bits into a text file myself (hooray for low-tech, brute-force solutions, haha... :S). Just have to process the code now. I think I'll be using the MOSES toolkit; fortunately, there are scripts provided that handle most of the text-processing. It's mainly a matter of aligning sentences now. I had a hard time deciding which editions of the Bible to use, at least for English (I can't find the more recent edition of the Xhosa Bible, so I'm using one from 1975, which seems to be the standard, at least from what I can tell?). I originally thought of using a more contemporary English translation, such as the NIV bible, but I decided it would be best not to get into copyright issues with evangelical Protestants, so I settled for the World English Bible, which is one of the few (!!!) entirely free, open-source English bibles... There was also the Open English Bible, but that only has the New Testament so far.
On Saturday, there was an LTI social event: watching X-Men: First Class down at the Waterfront, and dinner (and drinks) afterward at Bar Louie! The LTI is so good to its students. I really respect that--it's important to get along with others, and I guess even grad school is still "just school" in some respects. It's important to make friends and good memories!
On a personal note, I finally got my housing situation resolved, thank goodness! I'm living in Shadyside now, which is a really nice area, and the rent is even cheaper than what I paid for living on Beeler St the last time I was in Pittsburgh! Plus, there's a (free) parking lot in the back (apparently street parking here requires a permit, and it's kind of hard to get one when you're subletting for only a few months). My new roomie Londen and her boyfriend James have been incredibly sweet... I don't really know anyone here, nor do I have any relatives nearby, so they took me to James' family's home for the Memorial Day weekend, where I got to feast on so much tasty cooking! Mmm... sure beats ramyeon, haha. I've probably been eating too much of that... I also went to a place called Belvedere's for 80s Night (woo, cheesy music!), where I danced for hours and hours. It was super fun! And I met a pretty cute guy, too. :3
[Back to journals]
Week 3: 6/5 - 6/11
Let this be known as the Summer of Ubuntu (which also happens to be a Xhosa word, heeeyyyyyy!). Finally made the switch today; completely wiped Windows Vista off my laptop and installed Natty Narwhal (I love these names!). So far, so good... I think. It took forever and a day for me to backup all my files, so the lesson learned is: don't use Windows Backup if you can avoid it. I hate saying things like that, but it's pretty true in this case. :S Anyway, at least it was a good excuse to pick up another external hard drive after my old roomie accidentally broke my last one (oh, Molly, it's okay! :B). It's so amazing how cheap they are now. Hooray for Kryder's Law!
Prof Rosé wants me to analyze the lexical overlap between the Xhosa Bible (XB) and our codeswitching corpus, to determine whether it's a feasible/not-completely-terrible training text... Reading up on corpora analysis. Terms to keep in mind: corpus homogeneity, corpora similarity. I had never thought of using an iterative approach to compare multiple segments of texts, either. Guess that's why I'm still in school, haha. :) The best-performing approaches seem to rely on WordNet, however, and that's of limited use to me in this case. But I did find a few papers on monolingual resources. In any case, I'm very glad I've become used to scripting in Python! (But Java will always be my greatest love! Though sometimes I wish I had more opportunities for programming in Prolog...) It's embarrassing because I had almost completely forgotten about this, since I was so busy fiddling with my bibles...
Speaking of which, I thought I had all my verses aligned, but fortunately the corpus-cleaning script provided by MOSES (still don't have all my MT tools installed, but I'm slowly taking care of it... I hate checking for dependencies D:<) brought to my attention that the two bibles don't have exactly the same content (which I knew, since the WEB has apocrypha, whereas the XB comprises only the Old and New Testaments). I must find this one mysterious line...
Prof Rosé had a dinner party at her house for her students on Sunday... I got to experience the famous Rosé Cheese Ball! I feel so spoiled. :) Took Londen and James as well; they seemed to have a good time. Nothing is better than good food and good friends/company! I was going to drive back home to Maryland last weekend, but went this weekend instead. I ended up buying an air mattress since the girl I was subletting from took back her furniture (not even going to go into this), and I needed to pick up some sheets and my chopsticks. I can't believe I forgot to pack any! But there's really no place like home--nothing beats my mom's cooking and free laundry!
In other news, went to lunch with the guy I met, Philippe. He's really nice and has offered to show me around Pittsburgh! We went to a Southeast Asian (I hate huge generalizations like that, but what can you do in this case?) restaurant in Oakland; I had really good spicy fried rice. Mmm, food... And I went back to 80s Night again! So fun. I'm totally on a New Order kick now. Went with Laura and Phil from the lab; what a great time! AND THEY PLAYED "TAKE ON ME"!!! I was so excited! I'm so lucky to have cool labmates--they're smart AND fun. :D
[Back to journals]
Week 4: 6/12 - 6/18
Let's get... biblical? At first, I thought I was off by one verse, so I spent some time hunting for it, until I realized it would be a whole lot more time-efficient to write a script to extract verses that are only present in both bibles. As it turns out, there's 24 verses in the WEB that aren't present in the XB--all of which are in the New Testament (I imagine that the Old Testament is pretty non-controversial/largely consistent across translations by now). I had really been hoping to have at least a baseline system trained on the bibles by the end of the week, but at least I've made some progress. And it wouldn't have been very good at all to have something trained on misaligned texts! Whether or not the Bible is a good training text for an MT system, at the very least I've written a script for extracting and aligning verses from any pair of plaintext bibles. I guess it's at least interesting to be able to see the discrepancies between different editions, and I'm sure it can be easily extended to other texts.
Still working on getting everything I need installed. I feel kind of embarrassed--sometimes I think I must be the worst CS major in the world. But Prof Rosé told me not to worry so much, and I'm going to do my best to be a little less self-conscious... Signed up for the MT group mailing list here at CMU. Gosh, it's so intimidating just walking to the bathroom, because I'm passing by the offices of some big names in NLP! But Carolyn is right, I need to take advantage of all the great resources available to me here while I still have a chance to!
Anyway went to a yoga class this week. Exciting stuff! Except it was bikram (hot) yoga, and it completely slipped my mind to bring a towel. So about a half-hour into the class, I was constantly slipping on my mat. :( As one would expect, this made transitioning from upward- to downward-facing dog very difficult. Lesson learned!
[Back to journals]
Week 5: 6/19 - 6/25
Finally got MOSES (and SRILM and IRSTLM) up and running! Maybe it's overkill to use two language model toolkits though... Anyway, figuring things out. Using the bash shell really takes me back; I haven't touched a command-line interface since elementary school when I used MS-DOS to play shareware games all day (ahhh, the days of Windows 3.11...). Worked through a few of the examples on the MOSES tutorial, but I'm not terribly familiar with all things Unix-flavored. So I'm trudging through. But the good news is that I have a very basic Xhosa-English system trained on those bibles. Something is better than nothing! And it took me forever, too, but it was a good learning experience. :) Now I'll be prepared for next year!
Nearly everyone in my lab group is at the big ACL conference this week, so it's been a little quiet around here lately. But I was still able to meet with some grad students whose research focuses on MT, so I have a bit of a better grasp of the overall MT pipeline and general issues. Of course, it wasn't too encouraging, since nearly everyone told me that the Bible makes for an AWFUL TRAINING TEXT, unless I'm okay with really funky output (like "john consecrated his drink unto his body"--I'm making this up, but a translation system is only as good as the data used...). Again, I tell myself that it's better than nothing... Although I do have a concise Xhosa-English dictionary that Carolyn kindly provided me with, and it's always good to have some monolingual resources on hand. Yeesh though, my project seems more and more daunting. :S
But still, it's been a good week! Had dinner at Philippe's (woah) earlier in the week, and went to the WYEP Summer Music Festival at Schenley Plaza with Londen and James. The weather kinda sucked, but I stayed for this band called Jukebox the Ghost--so good, so good! If Death Cab and Ben Folds had a baby, it'd sound like them. Gosh, it's been such a smashing summer so far, and it's not even July yet!
[Back to journals]
Week 6: 6/26 - 7/2
As Prof Farnsworth would say: "Good news, everyone!" Kaitlyn found a South African government website that provides "easy access to government information and services" (their words sum it up best!)--I think this is the corpus I've been dreaming of! Not only is it in Xhosa and English, but Afrikaans as well! It's located here; language options are towards the upper left-hand corner. Carolyn suggested I work on getting a three-way system implemented in our reading group meeting, but I kind of hope she was joking... :X Anyway, looking forward to scraping some data!
Going home for the Fourth of July weekend and bringing P with me. :) Slogging through the 200+-page MOSES manual... Now that I have a sense of how to work with it, I need to figure out how to incorporate the work Kaitlyn's been doing on her morphological analyzer to improve translation quality. Stemming to find probably roots is essential, given how highly agglutinative Xhosa is (and oh man, not only are there suffixes, but prefixes and even infixes! @_@;;). Other than that, not a whole lot to say. Looking forward to a delicious BBQ! (I'm proud to say that my mom makes the WORLD'S BEST TERIYAKI CHICKEN, and that's no lie!)
In all honesty, I don't think I've ever enjoyed any job as much as this one. I'm learning so much; my best friend Miki thinks I'm crazy for working late, but sometimes I have to anyway because I'm not exactly a morning person, haha... >_>; Anyway, I'm thoroughly enjoying life right now! Asked for a one-week extension for my internship last week and got it! So excited! Plus I ended up having to sign onto my apartment lease anyway, so I can stay through until mid-August. Grad student orientation at Wes doesn't start until 9/1. I wonder what I'm going to do with those few weeks before I have to go back? Probably will visit Miki in Boston...
[Back to journals]
Week 7: 7/3 - 7/9
The Fourth was great! I drove on the way to Maryland, and P drove back. It was a fun mini-roadtrip of sorts. :) Of course, the food was tooooo goooooood (I think I must have gained 5 pounds, no joke). My mom wasn't feeling so well, so P and I went to Baltimore and watched fireworks at the Inner Harbor. Getting out of Baltimore afterward was kind of a hassle, though. Police basically shut down the city because some people got shot. :( I understand partying hard, but not that hard... Got back on Tuesday; soooo tired. Mainly worked on my website, because I had been putting it off for a while, err... ~_____~;;
Anyway, I've been a little frustrated these days. Can't figure out why I can't check the phrase tables for my rudimentary system... At first I thought it was my lack of Linux/Ubuntu/bash finesse, but now I'm starting to worry that I didn't install everything right. Urgh. So yeah, working on my website, because at least I can see tangible results in a snap. :( At least it's fun dipping my toes into HTML again--it's been such a long time! I used to focus way too much on making everything pretty (granted, I'm a perfectionist), but this time I'm pretty happy with the minimal, non-graphics-heavy-for-once layout I've come up with. Hooray for CSS, and the Web 2.0 aesthetic! :D
I think Philip Resnik's STRAND approach is the way to go for getting those aligned sentences. A little intimidated, because even though I used to make a lot of websites back in middle school and early high school, I've never really understood networking or web architecture. Thank goodness a lot of other people have done this kind of thing already!
[Back to journals]
Week 8: 7/10 - 7/16
Finished doing a clean install of Ubuntu (I'm going to be honest here--not knowing a whole lot about Linux/Debian/Ubuntu's file system, I basically put everything into /bin, ugh, and I think that might have had something to do with my funky problems) and got everything back and working! Woohoo! Ultimately frustration pays off--now I have even more experience installing things in a non-windows environment (notes on installing SRILM, MOSES, GIZA++, etc. forthcoming, since I think they will serve as a handy reference for myself and hopefully others!) and my old build of MOSES didn't handle factored language models (FLMs) properly... which would have been a big problem, since I kind of need those for dealing with morphological data... (kind of important for a highly agglutinative language like Xhosa!)
Still need to scrape data, urgh. I've developed a ridiculous caffeine addiction this summer--I even drink more than I did at school! I think I must go through at least 2 12-oz bottles of Coke Zero and 3 cups of green tea on a daily basis. v_v; I'm kind of glad I asked for that extension, because I'm really going to need it!
I'm also kinda sad since Laura will be leaving soon... T_____T The lab will be so much quieter without her, and I'll have one less friend in Pittsburgh! But I promised her we'd go out to brunch at Pamela's (a very popular local chain that serves some famous, Obama-endorsed pancakes!) this coming Sunday. :)
[Back to journals]
Week 9: 7/17 - 7/23
Brunch with Laura was super! I love, love, LOVE the strawberry pancakes--they're topped with this awesome sour cream topping that adds a wonderful tang, and these crunchy brown sugar bits... mmmm. So much good stuff at Pamela's! But now Laura is gone, and the lab is so much quieter. :( At least Kaitlyn is still here!!!
This week I checked-in with Carolyn since she had some questions on integrating Kaitlyn's morphological analyzer with the baseline system(s) I have trained... It's amazing how big the MT pipeline is, but I think it's very good that I'm having to learn how everything works, and fast! That way I'll have a better idea of what's more feasible for me to research next (school) year, and better yet, what topics in particular spark my interest! I'm still kind of nervous about grad school and all that--I'm really lucky that I'll have an extra year at Wes to ease the transition. It's still up in the air whether I'll go straight away for my PhD after finishing the BA/MA program, but we'll see!
Actually, I met with Vamshi Ambati and Greg Hanneman this week, who are both PhD students in the LTI. Vamshi talked about his crowdsourcing research and gave me a few suggestions on how to utilize Amazon's Mechanical Turk service for the codeswitching project on a larger scale (like perhaps writing a classifier to detect instances of codeswitching). It could be interesting, but there's a lot of work involved in making sure that the people you use are reliable and knowledgable! At the very least, it's an idea worth sharing with Carolyn. :) And I spoke with Greg mainly on how he got to be where he is today, and it was pretty cool to meet someone who hasn't done NLP all their life! He was very encouraging, and I think speaking to him eased a lot of my anxieties about being what is essentially a professional student. It's possible to have a good work/life balance, and even PhD students are still people. It's easy to forget that sometimes. :) Haha.
Time goes by so quickly! Golly, it's almost August already...
[Back to journals]
Week 10: 7/24 - 7/30
So yeah, we're ditching the whole factored language models thing because Jon told me that it would require a whole lot more work than it's worth, at least given the amount of time I have left! Hmm, but I still can't get the morphological analyzer to work. It's times like these that I really, really wish I had a little more technological saavy... It must have something to do with my Linux distro, since it obviously works fine for Kaitlyn (on the Windows side)... Hmmm... I tried fiddling around with a few things, but I don't want to blindly destroy anything she's done! So I'm kind of stuck here, but in the meantime, I can tinker around with what I have and explore how different techniques in preprocessing affect performance with the codeswitching corpus we have. That part is pretty simple, thank goodness!
Having some problems aligning my data, though, since MOSES doesn't like that the English side of the corpus has 2000 more lines than the Xhosa side. I sat down for a while looking through both text files with my trusty Concise Xhosa-English Dictionary, but that's way too tedious. But lo and behold! Google has saved the day, and I stumbled upon hunalign, a bilingual sentence aligner! It seems so promising, but now I keep running out of memory. BAH! It already comes with a script to chunk data for batch processing, but even then, I'm still getting memory overflow error messages. :S If it's not something, there's always something else, or however that saying goes (I'm so bad with idioms, hahaha :<).
Other than that, summer's winding down, and I can feel it... I'm really happy that I met Philippe though--he's so wonderful! (I hope he doesn't mind me writing this and making it available to the public, haha.) His parents have graciously offered to let me stay for a week after my internship is over, so at the very least, I can have a fun week in Pittsburgh before heading back home and going back to school (it feels like I never have enough time anywhere...). Really, it's just a little over two weeks I have free between the end of my time here at the LTI and when I need to get back to Middletown. Phew!
[Back to journals]
Week 11: 7/31 - 8/6
FINALLY figured out a way to get hunalign to work so I can train decent baseline models on the *clean* web data! I really wanted to stick with hunalign since one of its features is to align the data through iterations while building up a dictionary, even if one isn't provided! That in itself is worth the pain, I think... Anyway, so the chunking script determines "good" places to segment the text, and I figured you shouldn't fix what's broke. I'm sure the algorithm it uses to segment the text is better than any one I could come up with, but the chunks themselves were still too big--so I simply wrote a shell script to rechunk chunks over a certain size (I determined that on my system, hunalign could only handle files less than 100 MB). The only problem is that hunalign seems to be sensitive about the format of the batch file, so it kept crashing just before finishing the job... But the chunked output was still there, so I went ahead and used it! Who knows why it kept crashing, but I looked through the code and--as far as I could tell--the output was still good and usable even though the script itself couldn't terminate properly. Hrmm.
Anyway, it takes forever for the script to run (I left my laptop on all night long all week long, pretty much). But now I have it! And better yet, hunalign also includes in its output alignment scores. So now I have two baseline systems: filtered-0, which just filters out anything with a score less than zero (meaning it's highly unlikely that the two sentences on both sides of the corpus are really aligned versions of one another), and filtered-1, which is slightly more selective about what it chooses as a "good" aligned pair. Now all I have to do is write a script to automate the actual translation process! :) Hooray for progress!
[Back to journals]
Week 12: 8/7 - 8/12
Unfortunately, the results are pretty disheartening... In short: there are over 2000 lines extracted from the codeswitching corpus that contain instances of codeswitching. Of those, only 1 line was translated using all versions of my system: both filtered-0, filtered-1, and the unfactored baseline model. :( I'm kind of embarrassed, because it's like I'll never produce something good, solid, and concrete. I guess I couldn't expect any better with a plain model without morphological features. I try to remind myself that all the grad students I spoke to told me I'd definitely have an ACL-worthy paper if I got anything substantial out of this! But considering the results I got, I don't even want to try to do an objective evaluation using BLEU or METEOR. :( Shucks.
At the very least, I learned a lot, and I'm very fortunate to have been able to spend my summer working at the LTI! I've had a great time--I've met so many awesome people, I've been to really fun places, I've eaten super tasty (!!!) food... I'm going to miss Pittsburgh so much. The independent life I've had here is pretty sweet. God knows I love my parents, but ever since I've started college, it's been a little dispiriting to go back home, since saying they're strict is kind of an understatement. Really. But it'll be nice to see them and my fluffy Kirby dog! :) And I think Lumi will appreciate being in a place with air-conditioning--hamsters really don't take heat well! (I pretty much have kept a fan on her cage all summer long, but she still sprawls out in a heat-induced stupor from time to time :S) But I'm really going to miss Philippe... At least I have another week with him. :) We're going to visit Hershey (WOOHOO, CHOCOLATETOWN, USA!!!) before I go back home. I'm so excited!
No matter what, there are always things to look forward to in life. Even if things don't turn out the way you'd hope (especially if you tend to have really high standards...), at least you did something. I try to tell myself this every now and then. :) Now... onto a new, exciting chapter of my life! So long, summer--you were the best!
[Back to journals]
[Back to top]
