| About Me | The Other People | The Research |
| Research Journal | What's So Great About Minnesota? | Final Products |
This is my research journal, made to catalog everything I'm doing this summer! Some entries were cowritten with Samantha Merritt.
| Skip to Week: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | Pictures |
On the first day, Samantha and I met our mentor, Victoria Interrante at her office at the University of Minnesota. She showed us the Digital Technology Center and then the lab where we spend most of our time working. There we were introduced to two of her PHD students, Lane and Loren. They proceeded to show us how to use the four main pieces of equipment we would be working with: the Head-Mounted Display (HMD), the motorized wheelchair, the Vicon Blade tracking system, and the HiBall tracking system. We proceeded to run several programs on the computer, explored the virtual environments previously constructed, and looked at the research from last summer.
We spent Tuesday reading relevant papers on subjects we would be referencing in our research and began to tentatively outline both the purpose and the methodology of the experiments we would be running this summer. They will focus on the use of rotational navigation in a virtual environment, and possibly also on the use of redirection. At the end of the day, we sat down with our mentor and finalized our first experiment to focus on the use of redirection and rotation in a box search. We also got the opportunity to meet Robert Bodenheimer, a professor at Vanderbilt, and discuss with him a paper of his that we had read earlier that morning.
On Wednesday we started by reprogramming the wheelchair joystick to only adhere to rotational input, and ignore translational. We then streamed data on the physical motion of the wheelchair while rotating in order to get a sense of how much translational motion was unavoidable, as well as to start formulating an equation for how to incorporate the wheelchair motion into the viewpoint algorithms. This would allow us to create a redirection scenario in the next stage of the experiment. This day was also useful as a refresher course in C++; Samantha had just finished a semester using the language in her introductory graphics class, and I had learned C++ in a summer course my junior year of high school.
Thursday and Friday were used to work our way through a stack of literature relevant to our proposed research. This was done for two reasons; one, that it was necessary to understand what had already been studied in order to keep our experiment from being redundant, and two, because one of the PhD students required full use of the lab for his own research. We decided that our experiment is relevant because it explores the popular concept of movement and cognition, as well as the creation of the illusion of self-motion. We found these concepts had surprisingly little published research. Samantha and I were excited that our topic may interest not only the virtual reality community, but also the psychology community. However, we did not have a chance to discuss our findings with our mentor, so we agreed to continue forward with the original outline, until such time that we could discuss possible expansions to our experiment. At the end of the day on Friday the genius PhD student, Loren, that we share a lab with asked us to be his test subjects. His work is focused on using an XBOX Kinect in coordination with the HMD to generate an avatar of the user wearing the HMD, allowing your real body to be visible in the virtual reality. Testing this was really fun. He also ran a 'vertigo room' simulation on me, which made it appear that I was high in the air walking on a narrow strip of floor. It made me so dizzy I almost fell over! I'm sure it was hilarious to watch.
This week, to date, has been comprised of being Loren's test subjects, planning for our experiment while waiting for use of the lab, as well as actually getting friendly and comfortable with the code. By midnight on Wednesday, Loren submitted his paper and we had managed to code visual translational movement and visual rotational movement (with much assistance from Lane in dealing with quaternions, nefarious little things). We were beginning to see what our final product would look like. We started to think about incorporating 'realism' factors, such as the acceleration of the chair and less arbitary rotation and translational factors, including a loud click the wheelchair makes before every movement. By Friday we were hoping to have the ability to do a run through of all three conditions of the box search task, in order to stay on schedule for our July 15 paper deadline. This would give us another two weeks to possibly code another environment, as well as fix any errors we have made thus far, and still leave time to test participants. On Wednesday afternoon we went out to lunch with Professor Maria Gini, her DREU intern Karen, and her graduate student Elizabeth. This was a great opportunity to talk about options in the field of computer science, as well as hear testimonials from women in various stages of their computer science careers. Karen is working with toddler-sized robots, and they were very adorable.
Thursday was spent mostly coding. Samantha and I hit a few of our more unique problems, such as the Z-axis flipping (making the ceiling become the floor and vice versa), and the equipment malfunctioning whenever the chair did not start facing directly north. We also were using the official HMD (freshly repaired) which still had a few problems with the visual wiring. We eventually isolated the problems to a section of code we had worked on with Lane, Victoria's PhD student, and realized we'd have to wait until he came back into the lab next week to solve it. On Friday, we continued with this in the morning, and then created our websites, requiring minor training in HTML and Putty. At lunch, Victoria took us to the Jewel of India Restaurant, and we had a chance to get to know each other and talk about both our lives and the experiment. It was nice to hear a little bit about Victoria's life outside of work, as well as her experiences in both the university system and the field. The rest of the afternoon was spent planning out a schedule to make our July 15 deadline (only a month away!), finishing up the remaining portion of experiment one that was within our expertise to fix, and discussing the logistics of experiment two.
Samantha and I started to work on experiment two, as experiment one was at a bit of a standstill. We started trying to work with the software, and quickly realized the license had expired. Samantha had a semester's worth of experience in Maya, and the software that was previously used was 3DS Max (still by Autodesk) and we were potentially transferring over to using Ogre. We hadn't clearly figured that out, which left us a bit in the lurch. We decided to go in search of materials to build the environment physically, and traveled to Home Depot (unhelpful) and Michael's (very helpful). Our current model is a square column made out of foam board with a wood base and top, and was surprisingly successful, considering our expectations. We then attempted to locate 3D objects in the Google Warehouse that would match the objects I had brought from home, including a Pikachu stuffed animal, a bowling pin and a tiara. Around one, we had finished the model and found the objects, so we decided to look back at experiment one, and managed to isolate some of the errors. We found that the flipping Z-axis in the visual only state, flipped based on where in the rotation the joystick was currently located, and that translation in the rotation only state was still stuck in one direction. The hand finally seemed to be moving somewhat correctly, which was exciting, and we spent the rest of the day trying to debug the code. At the end of the day, Victoria stopped in and after a few mild suggestions, she approved our design. Tuesday, while very busy, was spent collecting materials, creating the graphical representation of the room in SketchUp as opposed to G3D (a faster software, to prevent lags during the experiment), and putting together our columns. We found that Samantha is a hot-glue gun ninja, and I am an exacto-knife wizard. The day went so fast, it was 5:30 before we knew it, and we both agreed that it was a lot more fun having a clear plan for the day and working towards goals.
When we came in on Wednesday morning, we finished up our SketchUp model for experiment two. We decided not to use the tiara, as it was not a good use of our time to model it. After we find the last item, and place the 3D pillars in the 3D model of the room, the model will be complete. Loren came in this morning and helped us to rewrite the code to eliminate our novice use of quaternions. He explained how to use the CoordinateFrame class to create a 'virtual wheelchair' object that would control the user's perspective. While our quaternion method was in the right direction, it left too much room for error. We also determined the need for a third HiBall sensor to track the wand, in addition to the HMD and the wheelchair. Loren and I tested the HiBall system to make sure it was working properly (as there seemed to be a significant delay) and although we didn't find any problems, I learned a little bit more about how the HiBall works. After getting the three states of experiment one to work we collected data from the wheelchair so that Loren could build dynamic models of it in order to more accurately recreate its movement. Finally, the PhD students have a meeting at four on Wednesdays, and Victoria has Samantha and I sit in in order to learn more about their research as well as their experiences in the field. We also got to meet the fourth PhD student, Peng.
Thursday flew by. In the morning we tackled the sinking floor and the collision detection by determining the radius of the room and limiting visual movement beyond it. We also implemented the ability for physical rotation to be different than visual rotation, in order to test our theory. We went up to the DTC for banana bread and coffee, one of the wonderful weekly traditions that UMN has to show their appreciation for their researchers. On Friday, Vicki took us out to eat at a delicious Himalayan restaurant in Minneapolis. In the afternoon we researched how to import our SketchUp files into the program to make it run with HiBall.
On Monday we reworked the code to work around a wiring issue in the Arduino, a little box plugged into the wheelchair that allows us to take control of the joystick from the computer. We spent a fair amount of time unsuccessfully looking for the bug in the code that requires the HiBall to reacquire its sensors every single time our program is rerun. Then we imported our SketchUp model of the pillars and objects into the program. Unfortunately, when you looked around the room there was such a bad visual latency that it was obvious the polygon count of our model was too high, and would have to be redone.
Tuesday, Samantha and I went out for lunch at the Orchid Cafe, a delicous Thai/Vietnamese Restaurant in Stadium Village. Unfortunately, the restaurant is located next door to Dairy Queen, so we also had to buy ice cream. We spent the rest of the day reworking our model for experiment two, getting rid of polygons wherever we could. The shoe, water bottle, and clock were the ones we edited. I also spent a lot of time retexturing pikachu's face to make him look right. The next day Loren sent us the new dynamic model, but we quickly realized it was a little off. Vicki stopped in for the PhD check-in, and we tried to schedule a time to talk with Bernhard Riecke, a researcher from Simon Fraser University, to discuss our research.
On Thursday and Friday we ran a lot of tests on Loren's dynamic model to help diagnose what was going wrong. We measured that it was moving about 5% of what it should have. He was able to fix the issues successfully. On Friday, Vicki was unable to make it into the lab, so Samantha and I took ourselves out to the Indian buffet again (it was amazing). The really exciting part of the week was while we were eating lunch on the lawn outside our building, and a news crew interviewed us about the recent Supreme Court decision! Samantha, my boyfriend, and I were all in the final edits. (I'm not sure if this link will expire, but here it is: U of M Students React to Supreme Court Ruling.)
We had been noticing frequent, brief freezes while doing the box search experiment, and we decided to tackle the problem this week. We noticed that every time there was one of these freezes, the output in Visual Studio mentioned a problem with the joystick. It had "Loren can fix this" all over it, so we decided to wait for him to come in. we also did a demo of the experiment for one of Vicki's colleagues and his daughter. They both had complaints of cybersickness, which was not a good sign. The next day, Vicki and Loren came in and we ran Samantha and Vicki, and both felt mildly ill. I am apparently immune to cybersickness, and being the one who was usually in VR for testing, had not noticed the problem. We had a serious discussion about cancelling the experiment, but decided to test a few people on Thursday and see if anyone felt cybersick, then continue from there. Wednesday was a holiday.
On Thursday we had subjects 0 and 1. There were a multitude of technical issues: sensors unacquiring, wires getting unplugged, and our participants getting hit in the head with the umbilical. Most were errors that we thought we could fix with practice, and no one had reports of cybersickness. We went ahead and scheduled participants for the next day, and still planned on a finished paper by the fifteenth.
Friday morning seemed promising, participant 2 had eleven out of twelve perfect trials until the wheelchair turned off and didn't want to turn back on. We sent him home, and fixed it before subject 3 arrived. Her trials were interrupted with the same issue of the wheelchair just turning off. Subject 4 was just a mess, he ran into the wall, and almost ran into our expensive irreplacable equipment. The wheelchair turned off again, and we canceled the rest of our day. It was a disappointment, but at least nobody felt ill.
We emailed Loren and Vicki about the problems we thought were still in the Arduino, and Loren agreed to come in on Tuesday. Vicki seemed encouraged that the latency didn't seem to be causing most people and cybersickness problems. When Loren came in, he dissected the Arduino and fixed what we realized was a wiring issue. We agreed to continue testing now that everything seemed to be working fine. After Loren, Samantha, and I picked up our lunch specials from Shuang Cheng, I became subject 5. Everything went smoothly. Turns out I'm a box search expert.
On Wednesday we had subjects 6 through 11. Loren and Samantha were the first two, and it went perfectly. Then we had two outsider subjects, and they also seemed fine. Unfortunately, subjects 10 and 11 both felt extremely ill. 10 in her eleventh trial and 11 on his first. It would have been fine except for they were my parents, and so I got to watch my mother feel sick for 24 more hours, and my dad for an entire week. Oops... (I think there should be some more research done on cybersickness: my dad's 60 seconds in VR triggered a bout of vestibular neuritis.)
We figured our bad luck on Wednesday had simply been because my parents were older than the other subjects we had tested, so we continued on Thursday with subjects all under 22. Subject 12 reported cybersickness about a fourth of the way through the experiment. Subject 13 got a little more than halfway. Since our results had been so mixed, we thought it could be related to the amount of video game experience, since everyone who got sick did not consider themselves gamers. So subject 14 was a hardcore gamer, and toughed it out all the way until trial 10, when he chucked all over our lab. Uh oh. After he left, we gave a lab demo for some math REUs at the University, and helped Vicki move some boxes. Starving, we went to Shuang Cheng before helping Loren run some tests on the latency. We set up the HMD to swing like a pendulum in front of the computer screen, and video tape the movement to measure the latency.
Vicki took us out to lunch at a delicious Greek restaurant, and we talked over more testing procedures to see what was causing the cybersickness. We spent the rest of the afternoon running the HiBall with one, two, and three sensors plugged in, switching the order of the ports. Our beautiful spreadsheets and charts turned out to be pretty unhelpful.
When Samantha and I came in on Monday, the HiBall was dead. Completely dead. It was plugged in and on, but it refused to do anything. So, we spent the day working on the model of experiment two, changing up the objects we were using to reduce the polygon count further. The next day, Peng came in and taught us how to use the Vicon system instead of the HiBall. He looked at the HiBall and decided all it needed was a reboot, which hadn't happened in years. We decided to wait for Vicki to come in, since the computer was really sensitive and really old.
We spent the rest of the week working with our model in SketchUp, making new objects for the pillars. We finalized our decision to switch from 3DS to Ogre hoping it would speed things up. On Friday, after getting fries and malts with Vicki and Annie's, we rebooted the HiBall computer and our problems were all fixed! Peng gave us an excellent lesson in using Ogre, and we started setting up experiment two.
Monday and Tuesday, I had to leave early in the afternoon for pre-op doctor's appointments, since I was getting wrist surgery on Wednesday. While I was gone, Samantha worked with Peng to get familiar using Ogre instead of 3DS. We were having a lot of lighting issues in the scene, making some walls and objects too dark or too washed out to see. Samantha also worked on preparing the Maya version of our SketchUp file to have accurate floor layouts and angles between objects. I missed all of Wednesday for surgery, but while I was gone Loren came in and fixed the distortion correction problems, and also finally got the wiimote to work. Meanwhile, Samantha finalized the physical setup of experiment two with accurate angles and tape marks.
Being the super brave and amazing person that I am, I came into work the day after my surgery, only mildly loopy with narcotics. Samantha and I spent most of Thursday writing out a detailed experiment design and methodology, and realized there were quite a number of problems we hadn't thought through. Specifically, we were confused about how the participant would be informed about their current location, through audio recordings or visually. We brought up all the problems on Friday with Vicki over Shuang Cheng lunch specials, and we realized there were a number of major design flaws. Not only did our current plan seem unfocused, the necessary adjustments to fix logistical problems would cause the research to be redundant. We decided to halt all work on the experiment until we could talk about it with Bernhard Riecke, whose previous work had inspired our research.
Monday and Tuesday were both short days. Since we decided with Vicki to postpone experiment two, we spent the days working on our websites and left early with a stack of papers to read in preparation for our technical paper. We went out for Indian one last time, and worked on the poster abstract I was submitting to the Grace Hopper Conference. You can read the abstract here. On Wednesday we had a group of children from a U of M robotics camp come in, and we put all of them in the vertigo demo. I think they had a blast, and it was a really fun break for Samantha and I. We spent the rest of the time working on our technical paper. I missed Friday because I was in New Jersey for my grandmother's 80th birthday party.
On Monday, Samantha and I joined Vicki, Maria, and Elizabeth for lunch at Bona, a delicious Vietnamese restaurant. After lunch, we finished our websites! We spent much of the morning on Tuesday uploading them to UMN servers, with thanks to Elizabeth for her help. Then we started outlining and writing our technical paper/final report, interrupted only on Wednesday by another fun demo for robotic camp kids.
On Thursday we finished our paper. It took most of the day, but was really rewarding. We went to our final DTC researches coffee break, where they had donuts (which I noted they also had on our first week here, aww). Vicki and Loren were both in and we discussed what we had left to do: turn in our websites, turn in our keys, and turn over our project to Loren. Today is Friday. We are putting finishing touches on these websites of ours (I hope you appreciate this was all coded by hand), and then having our final lunch with Vicki at Loring Pasta Bar. Before we go, we'll sit down with Loren and talk about what he needs to know, then stop downstairs to say goodbye to Elizabeth and Karen. Then we'll be done.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| About Me | The Other People | The Research |
| Research Journal | What's So Great About Minnesota? | Final Products |