|The Research Process | My Weekly Log
|Week 1: 5 June - 9 June
My first day here began with a project-overview presentation by the graduate student mentors (Sreedevi Sampath, Sara Sprenkle, and Emily Gibson) who are heading up the project that Holly and I will be working on over the summer. We were given a glimpse of the project as a whole as well as an idea of what our part will be in it. Holly and I will primarily be responsible for developing an open-source tool that will fit into the Web Application Testing framework that Sreedevi, Sara, and Emily have been working to develop. This tool will be a visual interface that will allow a user to navigate through the expected and actual HTML results returned from the execution of logged user sessions on a web application.
Our tasks for the week were as follows:
|Week 2: 12 June - 16 June
We have resolved our issues with file-handling. We can now access files from the local hard drive by utilizing Java to read lines from a File object and print them out within an HTML page for viewing within a frame. With this challenge out of the way we were able to improve our mock-up by doing real-time analysis and display of two local HTML files when the tool is initialized.
|Week 3: 19 June - 23 June
This week did not see a lot of progress, due to intervening events, but the progress we did make was good. Both Holly and I got around a few of our problems by coming to a better understanding of how servlets can be used in the background of a web application to do the "heavy lifting." I have now successfully integrated the directory structure of test cases into our application, by communicating with a helper Java class through a Java servlet. Holly is also utilizing a servlet to organize the dynamic construction of a browser window which displays the levels of the directory and allows the user to navigate to any test case in the directory.
This week also saw a lot of time away from our computers. Emily Gibson is preparing for a talk she is given at a conference. Holly and I sat in on two practice runs to provide input on possible points of confusion and slide problems. We also had a group meeting with our three grad mentors and Lori about the progress we were making on our project. On Tuesday almost everyone from our lab sat in on Sreedevi Sampath's dissertation defense. I found that to be a very profitable experience. It was nice to see what that actually might look like. Sreedevi passed of course, then we went out to celebrate with her at a local restaurant.
The biggest reason why this week did not see more progress is that on Thursday the power went down. It was down all across campus for about half an hour. But even when it came up everywhere else, half of our lab was still dead. Holly and I got an impromptu day off because we had nowhere to work from.
|Week 4: 26 June - 30 June
The visualization tool as of Friday was in pretty good shape. I integrated the new directory browser into the main page by setting up a browsing option. If the user wishes to have the directory browser open, the other contents of the screen will shift to the right and the browse window will be displayed. However, if the user wants to have a larger view of the HTML pages being compared and does not wish to browse, they can also close the browse window and the contents of the page will shift back in to place. When first entering the main screen of the tool we chose to not have the browser window open because the contents of that window do not always generate quickly enough. By keeping the window hidden at first we ensure that when the user opens up the browser window they will actually have something to see.
Another new feature was a pane at the bottom of the screen which lists the actual URL requests that resulted in the HTML pages being displayed. So the user can not only see the comparison between pages, but can also keep track of the calls that returned those pages.
A significant amount of my time in the fourth week was spent reorganizing our code. I also need to spend a good bit of time doing Java documentation on the code that we have now. I have been very remiss in doing that. It will be a rather time consuming task.
|Week 5: 3 July - 7 July
This past week was very short, due to the 4th of July holiday. Holly came in on both the 3rd and 4th, but I was out of town visiting my brother, so I will now have to catch up some time.
I spent Wednesday, Thursday, and Friday doing a lot of tidying up. There were quite a few cosmetic, structural, and logical holes left to be sealed. I kept running through the different functionalities of the tool, finding discrepancies and errors as I combined different actions in different ways. It's not a glamorous job, but it has to be done. I have taken on most of these little tasks because Holly is working on our last big problem: getting the error report option in working order.
I say that the error report is our last big problem, but there are also two other significant issues to be mentioned:1.) The tool is still not as portable as we would like. We are using a context-variable in the web.xml document to stipulate where the Perl scripts can be found because we have not yet discovered a good way of making Java and Perl get along without utilizing the UNIX command line. Also, we have been using the UNIX diff command which is, obviously, platform dependent.
|Week 6: 10 July - 14 July
We have come to a satisfactory stopping point in our development of the tool. The needed functionality (along with some "nice-to-have" functionality is in place) and the bugs seem to have been worked out. There are surely more bugs to be found, but Holly and I expect to uncover those problems while actually using the tool for its intended purpose: viewing/analyzing the expected and actual results from the replay of user sessions on a test application (DSpace - electronic publications web app). By doing so we hope to not only strengthen our tool by uncovering quirks but also to gain new insight into more sophisticated oracles (algorithms for comparing the expected and actual results and locating potential faults).
While Holly and I hope to contribute to the development of a better oracle before our summer tenure is over, we must also devote time to writing a tool paper which describes and evaluates our visualization tool. We have hopes of submitting this paper to a conference with a fall deadline. We also hope that any work done on a new oracle could be the foundation of another paper, but we'll see how that goes.
The technical difficulties of this past week dealt mostly with transitioning our project from running within the Eclipse development environment, to running independently on Resin. We also had to deal with a performance issue when dynamically generating the browse window for navigation within the test suite. Holly re-organized the algorithms responsible for this maneuver and things are running much more smoothly now. In short, it works!
|Week 7: 17 July - 21 July
This week was spent analyzing data from replayed user sessions and brain-storming ideas for better oracles. So far most of our ideas are along the lines of decreasing the number of false-positives: detected "possible faults" that are in fact not faults. Holly and I have each developed some ideas and have begun to implement them. Hopefully, we will be able to run some sort of case study to compare the performance of our refined oracles with the original oracles in terms of false positives and false negatives.
While Holly was working on her improved oracle she discovered a problem with one of the original oracles: struct++. This algorithm utilizes a Java archive that was acquired from a third-party, but it turns out that the archive itself (which has classes that are designed to parse HTML documents) has an error. Holly is looking in to converting the algorithm from Java to Perl.
I too am dabbling with Perl in order to implement my new oracle. I first tried my ideas out in Java, but the running time was unacceptably slow. Having never written in Perl, it took a little while to figure things out, but I think I'm on the right track now.
|Week 8: 24 July - 28 July
Holly and I both wrapped up our new oracles this week. The heavy-lifting of each was done with Perl, but Holly's required a Java front-end to do a little pre-processing before the parse. These two oracles have been integrated into our tool. The integration process wasn't too hairy, but it does bring up the question of extensibility. It's very possible to add and take away oracles, but it can't be done simply at the drop of the hat. Adding a new oracle to the main functionality of the tool (visualization, navigation) only takes about ten minutes. However, there is the dynamically generated fault report to be considered. There are also two supporting documents for the tool which, presumably, should be generated before the tool is ever used for a particular test suite. These two documents fuel the fault report and allow requests in the navigation window to be highlighted if a potential fault has been detected in them. Every time a new oracle is added, these two files would have to be re-generated by running the test suite again.
Aside from producing a tool paper, Holly and I will also be presenting a poster on our tool at the Summer Undergraduate Research Symposium here at the University of Delaware. Work has begun on that, though I don't think it will take a terribly long time.
There is also a possibility of a second paper on our new oracles. In order to do this we will need to run several test suites through the various oracles and compare their results. We already have seen an improvement in the number of false positives avoided, but we don't know how well our new oracles will perform in terms of actual fault detection. Will they miss things that the other oracles picked up?
|Week 9: 31 July - 4 August
Our tool has grown up to this point in such an organic way, that it is very easy to look back and think, "Now why didn't I design it that way from the beginning?" One of my biggest jobs on this project has been putting all the pieces together. I designed the front-end and the interface between the front-end and the back-end. I also helped put some of the back-end together, though I think Holly really deserves more of the credit in that area. So, when it came time to start adding the oracles way back near the beginning of this project, I did the work of automating the oracle execution in tandem with the initialization of the tool and setup of the ability to navigate from oracle to oracle. What I didn't think of at the time was how easy it would be to add new oracles to my system. What I realized later was that, although it didn't take too terribly long to add a new oracle, you needed to change a lot of little things in a lot of different places to get it right. This wasn't difficult for me, since I designed the system and knew where everything was. But it wasn't going to be very fun for anyone else.
So, this brings me to my accomplishment of the week: greater ease of extensibility. The addition of a new oracle now only requires that a single line be added to a configuration file. This line includes information about the name of the oracle, the parameters it takes, how its button should be labeled, etc. Once this information has been added to the config file, the tool does the rest. The Rendered and Raw versions of the HTML documents will always be shown, but you can now add up to five other oracles to the tool. The only stipulation is that they must be written either in Java or Perl.
The rest of my time this week was spent on other minor improvements and creating our poster for the symposium. Holly and I have split up different sections of the poster to work on. What it essentially breaks down to is Holly is in charge of tool features and I am in charge of everything else (intro, background, evaluation). This division will probably roll over into the work on our paper as well.
|Week 10: 7 August - 11 August
On Monday and Tuesday Holly and I finished up the poster. We each had a copy printed for ourselves. It's big, it's blue, it's beautiful. Wednesday was the symposium. It went very well. I will always be grateful to my four years of competitive speech & debate in high school - it makes thinking on your feet a lot easier. Not that either Holly or I got grilled by anybody, but we did have a few people stop by with good questions.
The rest of the week, which only lasted through Thursday, was spent writing the paper. We were both hoping to completely finish the first draft by the time we left lab that afternoon, but after staying a little later than usual it still wasn't quite ready. This was partly due to my having to fight with Latex, which I have never used before. Oh well, it will get done.
It seems odd that the summer is over. It has all gone by very quickly. I feel very fortunate to have had this experience - not only because of the work, but because of the people. I think I'm going to miss Delaware. = )