CIMAP: Progress Journal | |
|
Home CIMAP(My Project) Mentor The Lab Life at UMass |
Week 1: May 19-23 I have met all the members of the lab, and Karim Mattar, the previous owner of the project, has given me a crash course in networking and a grand tour of his code. I have many GUI improvement ideas, as I have a hard time operating the application and I see many places where a few lines of code would make a big difference. I spend the week getting to know the package, as well as adding a toolbar, automatic "ok-button-press" on pressing enter, as well as experimenting with creating small JInternalFrames a la Photoshop that would update on-click with live information about a selected node/edge, etc. I spend a whole day trying to get the frame to refresh and Teng fixes my bug in 5 minutes. I feel like I am learning 500 pages of Java Swing technique per day. Karim and I also discuss that the application is very fragile, crashing on bad input, and it doesn't have a lot of redundancy or error checking. We discuss ways to prevent people from opening the wrong types of files as BGP files, including creating our own file type, but that idea is soon abandoned. We still cannot get the application to open in a maximized state. Week 2: May 27-30 Karim has left for Egypt for the summer, leaving me alone in a room with CIMAP. Memorial Day has taken a bite out of progress and Teng and decide that we should check the consistency of the drawings the application makes against the BGP table, the relationships file, and data available on fixedorbit.com. After many ad hoc tests, and by-hand drawings, we conclude that the mapping is accurate enough for now and we should concentrate on adding features not dependent on mapping, like the Perl script that runs the algorithm Professor Gao developed to create relationships files from BGP tables. After a day's work, I incorporate the Perl script into the application, and everything works smoothly, except that we seem to be making defective Relationships tables. Teng and I also discuss the types of data that should be made available to the user in the InternalFrames i mentioned above, as well as incorporating live WHOIS calls, meaning that when the user wants to know which company an AS is, instead of accessing the out-of-date text file stored in the project file, we would run a script that would get the name live from an internet database. We decide this would be an unfair feature to add to someone who doesn't have an internet connection, but it is a good idea to put it into a web version of the application. At the group meeting, Professor Gao stresses that we should solve the OutOfMemoryError problem before proceeding with our ambitious GUI plans. Week 3: June 2-6 Jianghong and Teng determine that the perl script is not broken and successfully create functioning relatnionships files, which is a relief. I spend some time consolidating and reorganizing the menu, completely redrawing the menu that allows the user to specify fields for drawing AS-to-AS paths. The menus seem less confusing and not as redundant. It allows entry of IP addresses, lots of formats for entry of as names, i.e. 80, AS80, as80, etc. I also add fields to the InternalFrame "Status", which now tells you which BGP and Relationships files you are working with, what process you just did to which node(s), which node you have currently selected by clicking on it, and the node's company. The memory problem seems to be our biggest task now; this problem happens when we try to find direct and indirect customers of large providers like AT&T at large depths, like 50, and large path counts, like 1250. The application finds all of the nodes and edges, and draws them on the panel, but then it outputs and memory error and dies. Karim confirms through email that this is the error he was getting, and he doesn't think it is coming from the Depth First Search that finds all the customers at depth x. I decide to download a trial version of JProfiler, a program that monitors memory usage, instance counts of classes, garbage collection, and lots of other useful data. JProfiler slows the runtime of the program significantly, but it reveals lots of useful information about what the application is doing. We run the memory-causing problem on it, and we discover that the memory request surge happens after everything is drawn. We debug-trace it to EventDispatcher, but we don't know what event it crashes on. I suspect it may be JScrollPane trying to calculate the dimensions of the scrollbars, but i later turn them off, and the program still crashed. I also discover that loading the relationships file into memory for the duration of running the program is a huge overhead, which creates approximately 58000 LinkedLists and 119000 LinkedList Entries. I discover that the loader creates linkedlists before it knows whether a field for a particular AS (its customers, providers, etc.) has even 1 AS listed in it. Thus, a new LinkedList is created (an expensive process), and never used. I fix the code, reducing the LinkedList creation to approximately 17000 and the LinkedList Entries to 78000, which JProfiler estimates to be saving 3MB in free memory. Finally, I find that the algorithm responsible for the process that creates the memory problem is very inefficient. When the user specifies the AS s/he wants to see, and the maximum depth and paths, the program finds ALL the paths, and then discards the ones that are too deep, but not after creating a vector of all the paths, and a hashtable of all the paths. After i eliminate this total waste, JProfiler doesn't show any noticeable gain in performance, but an unscientific analysis of the time it takes to do this process is about 4 times faster than before. Week 4: June 9-13 This week Meghan Emilio from Trinity College arrived and will be working with me in the lab also due to the CRA program. We spent a good part of the week setting her up with computers and software, and we have decided that she will do the applet development while I continue with the application development. We began migrating some data from the BGP, Relationships, and AS Names files to the MySQL database. The queries slow the database slightly, but the meomry load is not as great as loading everything into direct memory. Week 5: June 16-20 The MySQL migration has continued, and we have successfully loaded 2 full BGP tables, a relationships file, and a AS Names table. This allows the applet to have much more free memory because the Relationships file is not loaded into a LinkedList of LinkedLists anymore. Meghan has also continued to clean up the applet code, which came with no documentation and a lot of unused classes. I have developed database access files, and we have been trying to keep the applet code and application code consistent. We have also discovered the source of the elusive memory problem. In short, after running tests with various depths and maximum numbers of paths, we concluded that the JScrollPane is stretched too far down by some queries, which makes it throw an exception. We have no adequate solution because we need to develop and new visualization scheme, which is currently not our primary objective. We have also developed a longest-prefix match algorithm, which allows the user to input not just AS number, but also any IP address into any field in the program that used to require one or the other. We have also consolidated the mapping functions, allowing the user to retrieve lots of information(degree, commercial name, neighbors, etc.) about an AS from one window. Weeks 6-11: June 23-August 3 Meghan and I rewrote the algorithm for drawing relationships between a non-Tier One AS and a Tier-One AS. It is much more efficient now, and draws on the screen, instead of going off the screen. I've also developed:
Weeks 12-14: In these last 2 weeks, Meghan and I began standardizing all the user interface popups and GUIs, as well as eliminating input bugs, and making the program run as smooth as possible. We also created a downloadable version of the application, and documented every class file. br< |