natasha @ wpi:
the summer research experience
DMP Program
CAPE Project
natasha @ wpi

The Details:
   All About Me
   My Mentor
   The Projects

The Journal:
   Week 1: 5/17 - 5/21
   Week 2: 5/23 - 5/28
   Week 3: 6/1 - 6/4
   Week 4: 6/7 - 6/11
   Week 5: 6/14 - 6/18
   Week 6: 6/21 - 6/25
   Week 7: 6/28 - 7/2
   Week 8: 7/5 - 7/9
   Week 9: 7/12 - 7/16
   Week 10: 7/19 - 7/23
   Week 11: 7/26 - 7/30
   Week 12: 8/2 - 8/6
   Week 13: 8/9 - 8/13
   Week 14: 8/16 - 8/20
   Week 15: 8/23 - 8/27

The Pictures:

The Contact Info:
   Personal Website
   AIM: theoryofideas

The Acronyms:
   CS @ WPI

The Projects

Project 1: Implementing Persistent Storage in CAPE
Project 2: Enhance the CAPE GUI

Project 1: Implementing Persistent Storage in CAPE

My first project is to implement some sort of persistent storage in the CAPE system. This is necessary because right now the system assumes that there is unlimited memory available for its processes, which may not always be the case. Below is my current list of tasks. I will update this list pretty regularly while I am working on this project.

(Last updated: 09/09/2004)

 Tasks Completed? 
Look over previous projects from Databases II to see how other groups implemented persistent storage and which optimizations they chose. Also, look through existing code to see how actual system differs from the class project and how all the parts work together. Yes 
Read "Aurora: a new model and architecture for data stream management" for ideas of how a queue manager should work. Yes 
Look at how to do random access files in Java to see if this can be used to store tuples on disk. Yes 
Create a QueueManager class and package in CAPE to handle all persistent storage. Yes 
Get Nishant's storage manager to compile in Raindrop. Yes 
Look at how the storage manager works to see how it can be implemented with our persistent storage system. Specifically, look at how parsing is done and see if it can be directly used in the queue manager. Yes 
Get file I/O working by having the queue manager communicate with Nishant's storage manager, without worrying about timing or multiple operators accessing the same queue. Yes 
Get file I/O to work with multiple operators accessing the same queue concurrently. Yes 
Work out the timing - enable the QueueManager to decide when tuples need to be written to disk. Yes 
Make the QueueManager optional to run by creating a superclass for queues, which can be either a memory or a persistent queue. Yes 
Get the operators to communicate with the QueueManager so that all tuples that have been enqueued can be written to file once after the operator finishes. No 
Get the QueueManager to communicate with the Scheduler and Execution Controller to prefetch tuples from file so they are in main memory when the next operator needs them. Note: this will only be done if we decide to implement multi-threading in the entire system. No 
Make sure the QueueManager is seamlessly integrated into the actual system and that it remains unnoticed as streams are processed. Yes 

Project 2: Enhancing the CAPE GUI

My second project is to add various enhancements to the existing CAPE GUI. Below is my list of tasks for this project, and it reflects my progress on these tasks.

(Last updated: 09/09/2004)

 Tasks Completed? 
Change button icons to make them look more attractive. Yes 
Add ability to plot statistics from several files. Yes 
Add a front GUI that starts up before the system is running to set up some initial options. Yes 
Have sliders to adjust configuration before running the system:
  • Migration: How often to re-optimize query plan
  • Distribution Mgr: How often to re-distribute
  • Scheduler: How often to adapt
Fix migration issues so that the tree refreshes after migration occurs and migrated nodes are colored yellow for one second. Yes 
Alter queue labels so they only show number of tuples in queue. Yes 
Maintain the ability to choose whether GUI should be run or not. Yes 
Show current configuration stats in the bottom panel. Yes 
Add counters for how many times each optimization was attempted and how many times it was optimized. Yes 
Show more info about each operator when clicked. Yes 
Fix bug where clicking on a queue makes a copy of the label and arrow. No 
Make more descriptive labels for operators (i.e. J for join, S for select, etc.). No 
Find way to tell GUI when system is done running so it can stop too. No 
Make GUI run faster. No