Introduction

Welcome to Sean Mealin's DREU blog. This summer, I will be working with Professor Balik at North Carolina State University. We will be working on creating a tool that allows blind and sighted people to collaborate in the creation of graphs. I will use this space to chronicle everything that happens as we move forward with this project.

As the summer starts, I realize that I don't know much about what we're going to do. The only information that I know for sure, is that we're going to create a tool for graph creation that is accessible to screen readers, and intuitive enough so that people can use it with little or no training. I think that one of the first things I need to do is to talk to Professor Balik, and see what direction the project is going to be heading. I am super excited for a couple of reasons; the first is that I can't wait to work with Professor Balik. I've met her before, and she seemed very receptive to new ideas. I'm sure that between the two of us, we'll be able to come up with creative solutions to any problems that arise. The other reason I am excited is that I think that the project will not only be an interesting challenge, but it will have practical uses both in college and in a professional setting.

I can't wait until it begins.

Week 1: (May 23 through May 27)

Finally, Monday, my first day rolled around. We spent most of our day setting up our programming environment. We decided to use Eclipse as an IDE, and SVN to enable us to collaborate. It took a little time to setup because finding and installing the SVN plug-in for Eclipse was a bit confusing at first. Also Eclipse itself is a very large download.
Professor Balik also took me on a virtual tour of the network. She showed me where I can host this web site, and where I can save files to the network, if I need to.

Tuesday we spent brainstorming the requirements for our program. I think we came up with some really good ideas. We also discovered how to import a .docx file into Eclipse, and how to add that to the SVN repository.

Wednesday, we continued to work on the requirements document. We went over what we had come up with on Tuesday, and simplified it as much as possible. We changed it a bit, to meet one of our goals: the program should be useable with little to no training ahead of time.

Thursday, we worked on creating use cases in the hopes that it would help us to simplify the work-flow. We came up with examples across a wide range of disciplines, such as computer science and physics. I think that this tool is going to be much more useful than I originally imagined.

Friday, we took a break from documentation. We focused on designing layouts for various screens. Because I wasn't up-to-date on creating GUIs in java, I ended up spending most of the day reading tutorials and a book or two on GUI programming.

I think that the project is coming together really well. This week we spent most of our time on documentation and paperwork, but I think it will definitely be worth it when we actually start to program the tool.

Week 2: (May 30 through June 3)

I was ready to continue the project after week 1, but Monday was Memorial Day, so I didn't go into work. I mainly took the day off, but I spent some time creating my DREU web site.

I started off Tuesday by fixing a few small errors on my site, and posting it to the space that Professor Balik set up for me. I then continued working on getting a handle on GUIs. When I was playing around with the program's main screen, I found that when I pressed the alt key, focus did not switch to the menu bar like it is supposed to. We also spent some time working with NCSU's Disability Service Office, arranging to borrow some equipment that we needed.

On Wednesday, I came back to the alt key bug. After doing some googleing, and trying different things, I found that I could get it to set focus on the menu bar by changing to a system-specific look and feel in Java. What that means, is by default, Java GUI programs have a computer platform (Windows/Linux/Mac) independent style, which also controls how it handles input from the keyboard. The Java scheme didn't put focus on the menu bar, so I had to set it to emulate the user interface of the host system. So on a Windows computer, it has a Windows UI look and feel, while on a Linux computer, it has a Linux look and feel. I also spent time creating some of the model classes, such as Node, Edge, and the Graph class.
Professor Balik and I took a trip around lunch time to NC State's other campus, so we could stop in at the Disability Service Office. We picked up a Tactile Drawing Pad from them.

On Thursday, we spent some time going through our code to make sure that our comments were detailed enough where we could come back at a later time to expand on our work. We also discussed the various screens of our application, and how they would be laid out. The tactile Drawing Pad from the DSO that we picked up the day before was very useful. What it allowed us to do was draw what we thought the screens should look like. When you draw on the device, (which looks like a clipboard with a piece of thin plastic on it), the lines raise so I am able to feel them. After you draw an image, you remove the plastic, and put on a fresh one. When we were working on the layout, we realized that we had to be careful when switching screens, so we wouldn't lose any data about our graph. That brought up a discussion about how to handle moving from a continuous space to a discrete, precision-limited space without losing too much positional information.

On Friday, we continued talking about moving information between different spaces, and specifically how our program would handle it. We also talked about how the program would represent graphs and their components to the user. After doing some research, we came to the decision that Java itself didn't have anything useful, so we would have to draw everything ourselves. From there, I started learning about the Java 2D drawing API, and how I could use it in our program. Most of the afternoon was spent creating a proof-of-concept program for drawing images. That lead to the question, what happens if the graph is so large, that all of it won't fit on the user's screen at the same time? Professor Balik and I weren't too sure, so we decided to do some research over the weekend, and solve the problem on Monday.

This week was spent on more concrete things for our program. Now that we have an idea about the final layout, and we know how we can represent graphs on the computer, we are going to start doing some of the hardcore coding next week. I'm definitely looking forward to that.

Vacation week 1: (June 6 through June 10)

While I did some work this week, officially it was a vacation week, because of the absence of Professor Balik. But that did not stop me from getting a few things done.

On Monday, Professor Balik and I spent the first part of the day addressing the screen size issue. We decided to take the approach of implementing scrollbars that would allow the user to scroll the image as needed. After creating a proof of concept program, we decided to focus on the program's general architecture; for example, we needed to figure out what classes should inherit from abstract classes. It would have been convenient to have an accessible UML diagramming tool, which is ironic because that is one of the uses that we have identified for our graph program.

Tuesday, we spent some time creating tools and references that might help us as we continue to write the program. We created classes that would update text on the screen, but also notify screen readers of the change. We also put all of our notes into a single location so they would be easier to find. We also spent some time incorporating the official Java graphics into our program, for example the icons on the toolbar. Near the end of the day, we found some accessibility oddities in our program, but we're not sure how to fix them.

Wednesday, Professor Balik was not around, and because it's an unofficial week, I did a bit of work from my apartment. I spent some time reading up on accessibility best practices in Java. While it was somewhat helpful, it didn't give me any leads on how to solve our accessibility problems.

On Thursday, I did a little work remotely. I spent some time focusing on how the Java 2D drawing API could assist us in drawing edges. I think that's going to be one of the more difficult parts, because neither Professor Balik nor I have more than passing familiarity with that specific API.

On Friday, I didn't do any work on the project. I spent the day pursuing other activities, in the hope that returning to do more work next week may give me a fresh perspective.

For this week, we didn't get as much done as previous weeks, but that's OK considering that this and next week are vacation weeks. The way I look at it, any work that I get done now, is just putting me ahead for when I officially come back to work on June 20. Sometimes, I feel a bit frustrated by Java, because accessibility for GUIs isn't as straight forward as some other languages, such as C#. I think that if I just keep on pushing through though, we'll be able to get everything working at the end of the day.

Vacation week 2: (June 13 through June 17)

I'm still on vacation, but I still managed to get a bit done. This week, I spent a bit of time working each day, but I didn't do too much. I had the thought of making a custom control so I would have finer control over both accessibility and drawing, but I decided that there was too much overhead coding to make it worth it. Instead, I made JPanels focusable (which they typically aren't), so Jaws can find them. Once I did that, I was able to use the accessibility API to post messages to Jaws and other screen readers.

I had put in a toolbar for shortcuts for sighted people, but Jaws consistently got lost in it for some reason, so I decided to make it invisible to Jaws and other screen readers, so the application feels more like other applications. I also spent some time refactoring the code in a few classes to make it more readable. I think that it will help in the future as we add more features.

Week 3: (June 20 through June 24)

Things are finally back to normal. Vacation was fun, but it's nice to be back working fulltime. Our primary triumph this week is that we were able to get the drawing system working. Now we are able to draw undirected graphs accurately. We'll deal with directed edges later.

Another thing that we spent a lot of time on, is fixing a bug that kept on cropping up when we tried to close the program. Whenever the program was about to close, it would execute a specific method that would cancel the closing process. So effectively, there was no way to terminate the program short of killing the process in Task Manager. The problem was something in the code that wasn't setting a keyboard shortcut correctly. The code in question was from following an old tutorial for a previous version of Java. So I guess I really need to keep a closer eye on when tutorials I read were published, and for what version of Java they were meant for. We also figured out how to prevent one of our views from scolling when the arrow keys were used to move around it.

Week 4: (June 25 through July 1)

This week ended on a very happy note. Our Graph Sketching program finally contains enough functionality to allow me to draw and access fairly complex graphs. The screenshots below show our program's interface. The first screenshot contains my recreation of a graph from my discrete math final exam. During my exam, my discrete math professor had to draw it using a tactile drawing kit in order to make it accessible to me. I still had to ask him for the name of each node as I examined it. With our program the graph is now easily created by and completely accessible to both blind and sighted people. Professor Balik told me about the Petersen graph, which is often used as a counter-example in graph theory. Based on her description, I was able to draw this quite complicated graph using our program as shown in the second screenshot below.

Graph Sketching Program interface with graph from discrete math final exam

Petersen graph rendered using the Graph Sketching Program

Week 5: (July 4 through July 8)

This week we added the ability to edit and remove nodes and edges. We also added a Jump to Node dialog so it's possible to get to every node even in an unconnected graph. We spent a lot of time on the point and click interface making it possible to select edges with the mouse and drag nodes around the screen. One of the challenges we faced was to insure that the currently focused object is always visible on the screen. We ended up having to use some advanced math to continuosly set the upper lefthand corner of the Java ViewPort everytime the focused node or edge changed in one of our views. We will extend this to the other view soon. We also spent a day refactoring and removing old code to optimize our program as much as possible. Because of this there is a noticeable decrease in the time it takes to switch between views.

This week brought our time of working on Centennial Campus to an end. We had to move offices to main campus, but it's pretty exciting because it's a nicer office and there's a lot more to do on main campus. Professor Balik and I are looking forward to sampling all the different restaurants around the campus. Below is a picture of me standing in front of Engineering Building II, the main Computer Science building where we used to work. The other picture was taken in the Innovation Cafe where we normally went for lunch.

Week 6: (July 11 through July 15)

Last week was a very busy week, but I didn't get as much coding done as I wanted. Last Sunday, I woke up to find that my laptop's hard drive died sometime in the night. So I had to call Dell tech support, and they sent out a new HD. So Monday and Tuesday I didn't go into work (without my laptop, I wouldn't have been able to do much), but I did setup the development environment on my desktop in my apartment. Monday I did a bit of coding, but Tuesday I spent the entire day restoring my laptop from an image, and updating all of my software. On Wednesday, Professor Balik and I spent the day writing progress reports for various people. Reports aren't my favorite thing to do, but I understand why they're necessary. Finally on Thursday and Friday I was able to get quite a bit of programming done. I spent both days on writing code that would allow me to load and save graphs. After taking some time to investigate the various standards available for storing graphs (GraphML, etc), I decided to create my own XML based format. I made the choice to come up with my own format because learning the various ins and outs of other formats would have required a significant investment in time. So while I worked on reading and writing the XML, Professor Balik worked on the frontend (prompting the user to save, etc). We're to the point where we can save graphs, but I'm having a problem loading the color data for the various elements in the graph. I think that I should be able to solve it at the start of next week.

Week 7: (July 18 through July 22)

Another week has gone by, and it occurs to me that my internship is almost over. I am really disappointed by this because I have really enjoyed working with Professor Balik, and working on the project. This week, I put on the final touches on the saving and loading subsystem of the program. Now I am able to create, edit, save and load graphs with no problems. What is really interesting is that I was able to use the GSK tool to design a flowchart for saving. I was looking to do the minimum amount of coding (the most efficient), so I was able to design a chart that laid out the interaction of everything that would write a graph to disk ("Save", "Save As", closing the program, etc). That was a fun experience for me; I've never really been able to organize something that complex before in a format that I can share with others (i.e. not in my head). Now that Professor Balik and I have that done, I think our next step is to put in the code to draw directed edges. From the research that I've done, I don't see an easy way to do that with Java. There are several problems that need to be overcome; I think I'm going to have to dust off my old calculus and trigonometry notes.

Week 8: (July 25 through July 29)

I am happy to announce that GSK is finally in a state where Professor Balik and I are happy to start putting it through some real world tests. At the start of the week, we spent a couple of days figuring out how to draw directed edges. After using some clever math, we have everything working how it should. There's some room for improvement, but we can always revisit the topic if it becomes an issue. The rest of the week was dedicated to refactoring the drawing code in the program to make it more extensible. The idea that we had is to allow the user to add objects to the graph from different domains, without forcing the user to make sure that the graphics are correct. For example, we may have a "finite state machine" category that has objects such as "start state", "state", "end state", and the like. In order to do that, we had to use several design patterns that we found in various books. After implementing them and rewriting a minor part of the program's source code so that it would use the new drawing code, everything is working perfectly. While I was working on that, Professor Balik was working ahead and creating all kinds of fun objects that you can add to the graph. We're pushing to add enough objects in a few key domains so that I am able to use GSK in a course that I have in the fall. That will be an excellent case study, and really allow us to get a sense of just how useful the tool actually is.

Week 9: (August 1 through August 5)

This week, I spent most of my time on my final report. In order to prepare me for graduate school, Professor Balik showed me the basics of the LaTeX typesetting language. With this neat tool, I can create professional reports in a plaintext editor like Notepad. Everything is represented as text, so it is completely accessible to screen readers. I spent a day or two learning the basics, and memorizing some of the most often used commands. I then started working on my final report. I think with all that I've learned this summer, it's going to be really good.

Week 10: (August 8 through August 12)

I can't believe it, but the last week has come and gone. I finished my final report on GSK today, so I'm really happy about that. In the paper is a lot more detail of how the program works than was put in this blog, so I think it's a fun read. I also spent some time this week squashing a few bugs in GSK. I think it's to the point where all the major bugs are gone.

I've had a blast working on GSK this summer with Professor Balik. I can't believe that it was only ten weeks ago when we started to plan the tool. Now, it is a fully working program that I think will be very useful both for myself and others.

I want to take this opportunity to thank Professor Balik for allowing me to work on this great project with her. I've learned so much about Java, and so many tips on how to properly design a program from scratch. Also, I want to thank her about all the random subjects that I've pestered her with questions about. She's helped me to understand quite a few things about the field of Computer Science in general.

Even though GSK has reached version 1.0, there are still more features that need to be added to make it even more useful. Though this summer is coming to an end, I hope that I'll have the chance to work on it more in the future. To those who are reading this blog, I give this one bit of advice: find a project, even if it's just you and your friends, and create something. Whether it's a game, or a tool to help others, you will learn things you didn't even know you didn't know, and have a blast while doing it.

Sean Mealin's DREU Experience

Navigation