Tasks |
Completed? |
Look over previous projects from Databases II to see how other groups implemented persistent storage
and which optimizations they chose. Also, look through existing code to see how actual system differs
from the class project and how all the parts work together. |
Yes |
Read "Aurora: a new model and architecture for data stream management" for ideas of how a queue manager
should work. |
Yes |
Look at how to do random access files in Java to see if this can be used to store tuples on disk. |
Yes |
Create a QueueManager class and package in CAPE to handle all persistent storage. |
Yes |
Get Nishant's storage manager to compile in Raindrop. |
Yes |
Look at how the storage manager works to see how it can be implemented with our persistent
storage system. Specifically, look at how parsing is done and see if it can be directly
used in the queue manager. |
Yes |
Get file I/O working by having the queue manager communicate with Nishant's storage manager,
without worrying about timing or multiple operators accessing the same queue. |
Yes |
Get file I/O to work with multiple operators accessing the same queue concurrently. |
Yes |
Work out the timing - enable the QueueManager to decide when tuples need to be written to disk. |
Yes |
Make the QueueManager optional to run by creating a superclass for queues, which can be either a
memory or a persistent queue. |
Yes |
Get the operators to communicate with the QueueManager so that all tuples that have been enqueued
can be written to file once after the operator finishes. |
No |
Get the QueueManager to communicate with the Scheduler and Execution Controller to prefetch
tuples from file so they are in main memory when the next operator needs them. Note: this will
only be done if we decide to implement multi-threading in the entire system. |
No |
Make sure the QueueManager is seamlessly integrated into the actual system and that it remains
unnoticed as streams are processed. |
Yes |