Week 1: June 4 - June 10

On Monday Barbara and I met with our mentor Dr. Lori Pollock and our research group at the University of Delaware. Introductions were brief and we got straight to the point of our research and what it our future work means to the research community and the world. Everyone was very welcoming. An overview of the project was explained to us and then what our specific part of the project for the summer was pointed out. We would be doing research to develop a 'tool' for the larger part of the web testing group's research project. With our goal in mind, I spent the rest of the afternoon and Tuesday learning JSP. From a programming perspective JSP is a combination of JAVA and HTML . Sara had presented us with a template for our research which was written in JSP and Servlet form. For me, it was like learning how to glue two different things together that you already know how to make.

By Tuesday afternoon I felt that I had a good grasp on JSP and I started working on my website for my DMP project. I got my own server space from the computer science department and received several different computer accounts to work on various networks for my research project.

On Wednesday we had another meeting with Lori in the morning about what we had learned the last couple of days, and we asked as many questions about the project as we could. Lori, Sara, Emily and Sreedevi did the best they could to get us on the right track with the project and helped us generate additional ideas of how to take the research project on. Sara gave us a template for the tool to start implementing our ideas on as well as several small programs to start completing that would be necessary for the tool later on.

Thursday and Friday were very frustrating days for me. I was constantly trying to implement different code to get one thing after another working. It never seemed to fail that once I got one thing working it wouldn't work when I placed it into the context that it was suppose to be used in. For example, I created buttons for our "tool" but when I went to make them functional with the rest of the code I was having trouble coding the differences between a variable and a string. Obviously my gluing skills in JSP weren't up to par. I continued to make a mock-up for the "tool" the reminder of the day. Barbara and I split up the work the best we could so that we each could work on parts and then implement them together later.

Week 2: June 11 - June 17

On the walk to work it was a beautiful day out and going to work seemed more exciting since the most difficult week, the first, was already over. All morning I worked on some java code for our "tool" while Barbara continued working on the mock-up. Later in the afternoon I once again attempted to merge them together. The implementation once again didn't go as hope and I spent most of the afternoon looking for additional ways I could possibly merge them together. After much frustration I decided to get caught up on my website and journal and then returned back to my java coding.

Tuesday was a very productive day for Barbara and me. We nearly finished our mock-up and different java wrappers that we had to write. We were done earlier than expected so decided to just keep perfecting it till the afternoon was over.

Wednesday was a very exciting day for me. We hammered out the final details of our mockup for our afternoon demo. I finished writing a highlighting tool and a file reader for implementation in the more final project. When we demo'ed our project everyone seemed fairly excited about our project and how well we were progressing on it. I felt really good about the work we had done and we got lots of ideas for improvements and additional features we should try to start implementing. It was a very successful day!

Thursday and Friday I started working on various lose ends that needed to be tied up. I would start on one project become stuck and start another. Although I can't say that I finished anything completely it at least showed us some of the potential problems in the various areas we need to work on extensions in.

Week 3: June 18 - June 24

Why are Mondays always Mondays? Sara spent a great amount of time today discussing our various issues with us and brainstorming solutions to them, but it was really a rough day realizing so many different things we created didn't really work right.

The reminder of the week brought never ending challenges. On Tuesday we went to Sreedevi's dissertation defense and she did an excellent job defending her work. It really gave me a picture of what it's like to do your research work for several years and finally get the chance to sum it all up in once. She passed and will now be graduating and everyone here is very happy for her. On Wednesday we had our meeting with Lori to discuss progress and the potential of what we are working on. After that we went to the web testing research group meeting to get an understanding for what the graduate students are working on and how they are making progress. There meeting helped to show us what our tool would be utilized in. After that we had Emily's presentation to go preview for the upcoming conference, which wasn't directly part of what we are doing, but it was important because it's a presentation for a testing conference. It was also important to note how she put together this presentation so that hopefully in the future I will be doing something similar.

Thursday was a rough day in terms of progress. The entire town of Newark lost power, which left our lab without power as well. While the power being out itself wasn't a big deal the fact that it blew the power to like 11 out of our 14 computers was. Even though one of the graduate students called for us right way in the morning the repair man never came till the end of the day. Friday was a normal work day for Barbara and I. We fixed things and broke others at the same time. I believe that was part of major planning downfall that we have going on, but we are still making progress and progress doesn't always come in large bounds forward.

Week 4: June 25- July 1

Monday and Tuesday really flew by for me. I checked into some new technology called Struts - specifically tree views. I decided it seemed like a very interesting technology but without strong documentation out it wasn't really for me. I finished up this file explorer and we implemented it. Monday was kind of long because Sara and I spent like three hours fixing an eclipse error and it was unbelievable how much SVN can do funny things. We realized it was committing our .project and .class files which were configured to individual machines not to the actual project itself. Oh well, we learned a lot from taking some time to experiment and work through the problems. Our 'tool' is coming together really well and my guess is that in a couple of weeks we will have it done and start implementing/writing additional oracles for our tool. Lori has mentioned that we will need to start using our tool to discover the benefits and downsides of our creation.

Wednesday was a very interesting day for me. It seemed like I had finally gotten used to my research surroundings and already I had to move once again. The entire HiperSpace lab was being moved to the 4th floor of our building after it had been here for several years. Everyone in the lab seemed a little disturbed by the move and by the short notice we got to get everything ready for it. So my research on Wednesday ended early because we had to pack and begin moving all the computers and then there will be none really done on Thursday because we weren't being given keys till Friday. This just leads me to believe that next week will be more productive than this one, but I still did get some good work done on Wednesday. My current focus is to work on integrating some error files we have.

Friday was a very productive day for Barbara and I. My list of things to finish up was getting a lot shorter every day. Fewer and fewer things needed to be changed or implemented. I started and finished up a java program that generates a HTML page for Sara's detailed reports. We even got it implemented in the larger tool.

Week 5: July 2 - July 8

Monday and Tuesday were very productive days for me. I started on several projects so that I really have none left on my list to get started. We had a special guest, Frank, come in and he showed me several different technologies that I could possibly implement as directory viewers. I also learned how to use the Firefox WebDeveloper toolbar as well as Firebug plug in. Both will be very helpful in finishing up our project in terms of making sure our tool looks good as well as functions that good. The only bad thing was that the AC in our lab is not working right since the power went out on Sunday. Thus it's really hard to focus on researching new topics without at least talking a short break from the room occasionally. I have to say though it's amazing to see how everything is coming together and how much progress we have actually made.

The rest of the week Barbara and I prepared for our demo with Lori. The battle of breaking something to finish something was never ending. Lori and Sara both empathized to us that that is part of research. I've started to realize the best products come not only from good planning and research but continuous change and updates. The week really took a lot of patience for us to get done what we needed to get done. When our meeting with Lori finally came on Friday we really hadn't left enough time for ourselves to put our two pieces together. This is not to say that we hadn't been implementing pieces as we went along but we had been working on so many different little pieces that they all took a little bit to merge together. During our meeting we discussed our progress with Lori and she said that we had implemented almost everything that was necessary to complete this step of our research. She said that it was important that we start gathering our thoughts for our tool paper and begin an outline at the very least. Everyone in our testing group said that we should start using our tool to view the several different test case suites the group has. I had started to look at the test data throughout the week but all the little bugs in our program made it nearly impossible. I left on Friday with high hopes to be all finished the beginning of next week.

Week 6: July 9 - July 15

Monday, Tuesday and Wednesday were exhausting days for me. One, we were doing extensive testing and debugging. Two, everyone in our research group was getting ready to go to a conference so Sara was getting us prepared for the next week of working alone. Three, we finally got to actually use our tool to its full potential so there was a lot of exploring of test data to do. By the time Barbara and I left on Wednesday nearly everything was finished or finished to the point it was a working version. I began to see the large amount of false positive faults that our various oracles were picking up from one of the sets of test data we have. It was amazing all the little things that you don't think about going into a webpage that can cause errors between different versions. The amount of errors and the different types of errors that I was starting to look at was nearly overwhelming.

To wrap up the week I started looking at all the Masplas and DSpace data we had using our newly finished tool. It actually was very serving of it's original purpose and held many new qualities that Barbara and I had implemented to call it our own. I started taking notes on all the various false positives I was seeing and how they progressed throughout DSpace. In the end I wrote up my observations and sent them off to Sara in an e-mail since everyone was at the testing conference. I also sent along various ideas I had for new oracles or how to improve the ones we have. By Friday afternoon, I was finishing up reading the remainder of the tool papers Sara had left for us and very content about starting on our own tool paper.

Week 7: July 16 - July 22

By Monday afternoon, Barbara and I carefully planned out our attack at writing our very first technical paper for our tool. It was very exciting to realize the hard work was over and writing the paper would be a great step to learning what another part of graduate school is like. The printer did slightly slow my progress when I went to print off some related papers to read and it refused to print more than one page every 2 minutes. In the end I had each of my parts of the paper outlined for writing the next day.

The rest of the week I spent looking for new ideas for my own oracle. While I had an idea of what I wanted to do I had to come up with a plan to execute it. I decided to narrow my work down to comparing links within a page and how they are structured. Since there was already an HTML parser out there written in java that one of our oracles used I figured it wouldn't be hard to adapt it to my needs. As well, one of our current oracles was already using that HTML parser library successfully to produce filtered results. It took a few hours of reading documentation before I outlined my plan of how I was going to create my oracle. I decided the output from our other oracle would work for mine and then I would simply read in that output and use the link information to create my own oracle. I spent a couple of days getting my java code working on the output but I realized that I was getting some strange results. My oracle was producing the same false positives that the other one was. It nearly blew my mind but not because I thought my oracle was perfect but because I didn't have a clue what would produce that kind of results. I had to start looking back into the HTML Parser documentation.

The HTML Parser documentation brought me good news and bad news. The good news was that a newer library was out that I could download and see if that fixed the problem with my oracle, but the bad news was that if the newer library didn't fix the problem then the problem I was facing wasn't a known bug or the HTML Parser was written to produce the results I was getting. In the end the HTML Parser wasn't doing what I thought should be correct and the new library didn't solve the problem. Since many of our oracle scripts were written in Perl I decided that maybe an HTML library in Perl could solve my problems. Even through I didn't know how to read the Perl scripts we execute now I decided to jump in and start looking at various Perl HTML modules. As it turned out it was like jumping in the deep end of a pool and sinking. By far, the Perl module had the features I had been looking for, but the strange structure for variables was making my life difficult. Since it was Friday Sara was in fairly early and she started helping me with my Perl code after I explained the issue with the Java HTML library. She was happy to help and suggested that I look into some Python HTML libraries as well. For over an hour we went back and forth between various documentation in both languages. In the end we decided that both languages could potentially solve my problem, but it was up to me to try to figure out which language in the long run would require less overhead. I started by looking at a few small examples online and before long I could see that I could understand the Perl HTML module documentation better. By late afternoon I had Perl doing a few small things and at least printing out the HTML tree correctly including implied end tags. I thought that this was a good sign for my work and I decided that I would invest in good Perl book over the weekend.

Week 8: July 23 - July 29

This week was a rather short for me. I took Monday off for working on the 4th of July, by choice of course. I had done a little bit of reading in my Perl book over the weekend, but not enough to simply jump into writing my Perl code. I spent the majority of Tuesday morning continuing the reading of my Perl book. By afternoon I had a good grasp on what needed to be done and how I might possibly be going about it. I literally spent the next two days trying desperately to develop my oracle. The resources all seemed to be there but my ability to put them together was not. The main problem was that I could get everything into the tree in the correct structure but I couldn't seem to force it to come out correctly or with all of the start and end tags present.

By Thursday I decided I couldn't just use the module functions that had been provided to me by others. While the documentation seemed to support what I was trying to do things just weren't working correctly so I took to writing my own Perl functions. I ran into the issue that I couldn't understand how the result of one function (that was written by someone else) could be passed into another one. Unlike in java, Perl accepts as many parameters as are passed into it if parameters for the function are not specified. If parameters are specified then other issues arise that I was trying hard to avoid. In the end one of the other problems that was my lack of understanding how true and false work in Perl. Sara had some free time again and she sat down to help me. While my misunderstanding of Perl variables was very great Sara had no problems explaining to me how to correctly set up the documentation I was using. She even showed me how to get the variables out once I had passed the returned variables from one function in to the next. I really only needed the first variable in the series of three that was passed but we went ahead and printed them to the file to check to make sure we were getting the correct results.

Once my challenge of understanding the basics of Perl were dissolved I spent the rest of the trying to finish up the Perl code. Not only did I start getting the code to do the dirty work for me, but Perl allowed me to put in various text in that would help me later when I read in the file with my Java code. Then the end I found it to be wise to write out to two files instead of one to help with the deciphering process. By the end of the week I was very happy with the progress I had made, but disappointed that once again that my oracle work would carry into another week. My disappointment didn't come from the overall time my oracle was taking but from the time it was taking me to put pieces together.

Week 9: July 30 - August 5

The week started off really well. My oracle had went from having no program that output the information it needed to having one that produced the correct output in the form of HTML tags and attribute values that I had been hoping for. While Perl was defiantly not the easiest language I have ever learn I do believe that it was worth my time and far more powerful than I has originally gave it credit for. The scary thought about the situation was the fact that I could actually read some of our other Perl scripts and make sense out of them more than some of the Java ones. I'm sure this is partly because it was fresh in my mind, but still I'm really happy I took the leap and jumped into learning it. After getting the output from the Perl script correctly I still had the challenge of actually going back and finishing my oracle code.

It took the next two days to rewrite my java code to interpret the output from my Perl script. One of the main problems was the recursion present in HTML pages if link tags were nested within each other. The nesting of tags in links was something that I wanted to preserve because broken links are a huge problem when it comes to HTML pages. I finished the oracle I had been working on for three weeks now. I wouldn't have ever thought that it would have taken me so long but the thing about research is that you really have to give yourself extra time incase things don't go the way you plan them. I'm glad that we did have a little extra time in our schedule so that I could get it done. As well, I had time to begin looking at some of the results it was producing to see if it was reducing false positives in our test suites, Masplas and DSpace. While I hoped that it would be a vast improvement over our other oracles I can only say it was a slight improvement, because there was still false positives it couldn't account for, like changes in table forms with the movement of links. By the end of the week Sara had run my oracle on Masplas so that I could see the results of it after it was run on an entire suite. This was very exciting because I had a suite of results to check for correctness in the tool we developed. Without the tool it would have taken days to check the results but within hours I had sorted through the oracles results and made a list of correct results and false ones. As it turned out, some of our server code wasn't executing correctly and the oracles were producing false positives. Sara ran the oracles again for us the next day and our results were more along the lines of what they should have been.

Thursday & Friday I worked on our poster for the UD Undergraduate Research Symposium. My goal really should have been to make a poster using poster 101 rules, but in the end it turned out more like a poster using paper 101 rules. I had virtually filled half of a two foot by three foot poster with text in size 14 font. Some people would have called this negative progress but to me it was a step in the right direction. All the thought and work that went into the massive amounts of text would be good to use in the paper later on. For the poster then I simply removed the text and inserted bullet points of my main points from the paragraphs. The discouraging part of the situation was no matter how much I was trying to limit what I put on the poster I still seemed to have too much, but with the help of Lori, Sara and Emily we got the poster shaped up and looking good. We only had minor changes to make when we returned to it on Monday.

Week 10: August 6 - August 11

Monday was a good day for putting the last of many things together. I spent the morning finalizing parts of the poster as well as taking a look at how my web page was coming. I decided that I need to devote a little time to it in the next couple of days to get it up to par. I continued ignoring my part of the paper. This was not because I thought it would be hard to work on but simply because it didn't sound like much fun to do right at that moment.

The reminder of the week Barbara and I spent trying to piece together our tool paper as well as putting finishing touches on our tool. It seemed like yesterday that Barbara and I had just got here, but I'm very excited to have a finished product and to be heading home. We had one last meeting with Lori on Thursday morning. We showed her our paper and we primarily analyzed its structure. Our paper is far from finished in terms of something to be published but content wise we believed that what need to be said was there. Impressively when our two parts were put together you couldn't really tell that two different people wrote them. Overall, we were pleased with our paper tried to tidy up some of the grammar issues we were having. By the end of the day we realized we couldn't get it all done for DMP submission but no big deal we said. We plan on finishing up our DMP work when we get back home and don't have to worry about packing and moving back home. DMP this summer was a great learning experience.