My Research
The research project I am currently working on is WordsEye, a text-to-scene generation project that combines elements of both Natural Language Processing and Computer Graphics. I am working on the Twitter component, which involves extracting the necessary information from a tweet in order to generate a 3D scene of it.
WordsEye creates a 3D scene from descriptive English text. For example, the sentence ”The cat is on the chair” results in a scene with a cat sitting on a chair. Twitter provides a way to reach many potential WordsEye users, due to the increasing popularity of social media and the large number of Twitter users. For this reason, I worked on creating and uploading 3D WordsEye scenes from tweets. Generating descriptive sentences from tweets is an interesting challenge because tweets are limited to 140 characters and contain Internet slang, emoticons, misspellings, and Twitter lingo such as hashtags and retweets. For simplicity, I focused on filtering tweets for only four verbs (like, love, hate, and see), as well several different emotions such as happy or sad. I used part of speech tags to extract the subject and object of each verb/emotion, WordsEye attributes such as colors or textures, and whether or not the verb or emotion was negated (i.e. ”do not like”). Using this information, I generated text that is compatible with WordsEye. For example, the tweet ”Yay!” would become ”The person is happy.” The final result is a 3D WordsEye image uploaded to Twitpic and a tweet posted to the WordsEye Twitter account. I wrote several Python scripts to automate the entire process.
An example is the WordsEye scene below, generated from the sentence "The woman hates the platypus."
