Kathleen Creel Williams College Class of 2010 Email Me Home

Week 1: June 1 - 5.

I arrived on the first of many hot Knoxville days, excited about my summer of robotics and of exploring a new city. I met my wonderful my roommate and fellow intern Jeanette Booher, and together we set up our apartment. On Monday, we had our first meeting with Dr. Lynne Parker. Dr. Parker was very kind and accommodating. After helping us get set up with administrative issues and taking us out to lunch, she explained our project. She sent us home with two of her published papers on the topic for background reading.

We spent the week refining both our understanding of the project and our pseudo-code mockup, adding detail and refining structure. Dr. Parker met with us every day and looked over our drafts, explaining what parts needed expansion or refinement. At the end of the week we felt ready to write code and begin the meat of the project.

Dr. Parker's papers described her graduate thesis work on the autonomous decision making procedure ALLIANCE which allowed heterogeneous robot teams to work together with reduced explicit communication and high fault tolerance. Our role in the project, expanded over the course of the week, was to re-write ALLIANCE since she no longer had her original code. Once we had ALLIANCE working, we would expand it to incorporate robot-human teams. The robot would analyze the human's behavior and select an appropriate action from a set of possible tasks just as it had done when observing its fellow robots. The decision procedure would be tested using standard box-pushing tasks.

Papers: "Peer-to-Peer Human-Robot Teaming for Shared Workspace Interactions: Proposed UTK work for 2009," Lynne Parker, 2009. "Implicit Coordination IR&D: Peer-to-Peer Human-Robot teaming for Shared Workspace Interactions: Progress Report for August-December 2008," Lynne Parker, John Hoare, Sudarshan Srinivasan, 2008. "ALLIANCE: An Architecture for Fault Tolerant Multirobot Cooperation," Lynne Parker, 1998.

Week 2: June 8 - 12.

This week we took our detailed outline and began to write methods and procedures. Jeanette focused on deciphering the box-pushing code which graduate students in the lab had written previously so that we could call it when that task was chosen. I began implementing the ALLIANCE procedure. ALLIANCE functions in part by calculating which behavior of the available choices the robot has the greatest motivation to perform. Motivation to perform each task is calculated with a formula that combines factors such as the time that the task has been left undone (or that another robot has been working on the task), whether or not the task has previously been attempted, and how recently the task currently being performed was started. So my first step was to implement the calculation of each of these sub-factors and of the final comparative formula.

We met with Dr. Parker twice. We updated her on our progress, especially our difficulties in understanding the box-pushing code. In the second meeting, she explained the code. We realized that the code relied on ASyMTRe, a different system for robot cooperation and data-sharing developed by a former graduate student Fang Tang, and Dr. Parker gave us one of her papers to read. By the second meeting I had finished many of the key ALLIANCE methods, allowing us to spend the next few days debugging it and assimilating the rudimentary box pushing methods.

We met the graduate students, who were very friendly and helpful. They worked in the lab next door to us. Both of the graduate students in our room were elsewhere for the summer, so we had the office to ourselves (including its giant 1970s style microwave).

We also had our first group paper discussion, which I found very intellectually exciting. We evaluated merits of the paper's presentation and content and put it in the context of other contemporary work. Each student in the group was assigned a subsection of the paper to summarize and present to the group. I was very grateful that Dr. Parker thought to institute this discussion series, as it was wonderful to get a broader view of the field.

Paper: "Building Multirobot Coalitions Through Automated Task Solution Synthesis," Lynne Parker and Fang Tang, IEEE July 2006.

"What Must a World Be That a Humanlike Intelligence May Develop In It?" Ben Goertzel, 2009.

Week 3: June 15 - 19.

This week my code was complete enough to test on the robot in simulation. The simulation environment which interfaces with our robot, Player/Stage, provides proxies and schemas which mediate between the robot's more rudimentary interface and the user's control systems. We learned how to parse the configuration (config) file which Player/Stage uses and how to access its provided drivers. Jeanette had finished modifying the box-pushing code, and we collaborated on code that provided a go-to functionality.

In order to do this, we read about the algorithms used for path planning, localization, and motion, as well as the interaction between the perceptual and the motor schemas. Perceptual schemas extract information from sensors, and motor schemas are self explanatory. ASyMTRe adds a communication schema, which allows the perceptual schemas to be connected across the robots. The three most useful drivers we identified were as follows. Wave front is a path planning and localization driver which extrapolates from the current location and the final destination to plan a path using way points. Each way point is then locally navigated using vfh (vector field histogram), which is for obstacle avoidance and position control. Finally the amcl (adaptive Monte-Carlo localization) algorithm is used to localize the robot by continually updating a probability distribution over the set of possible robot positions.

Week 4: June 22 - 26.

This week we focused on ensuring that both the go-to code and the decision procedures functioned properly and consistently in simulation. Using an existing configuration file based on wave-front, we wrote a go-to function which used the three drivers described last week. By Friday the goto and the ALLIANCE code were both operational.

Paper: "Towards a Universal Theory of Artificial Intelligence Based on Algorithmic Probability And Sequential Decisions," Marcus Hutter, 2002.

Week 5 : June 29 - July 2.

We next began a section of the project which Dr. Parker added once she saw that we were making good progress: an overhead surveillance camera system which would identify boxes. Once identified, boxes could be labelled as potential tasks for the decision algorithm to evaluate.

We considered two image processing packages, and picked OpenCV over CMVision due to its increased functionality and fantastic documentation. It even had sample code which identified rectangles in images. We set up a camera and took our own still images of boxes in the hallway to test the code with. We soon realized that the code would have to be significantly modified to deal with messy, real world environments.

Paper: "Where to Go: Interpreting Natural Directions Using Global Inference," Yuan Wei, Emma Brunskill, Thomas Kollar, Nicholas Roy, 2009.

Week 6: July 6 - 10.

Once the box-finding code identified boxes reliably, we set it up to process images directly from a live camera feed. In order to reduce noise, we established a training period in which the camera was focused on the empty hallway. Squares found in this period were discarded. I experimented with various image processing techniques to improve detection rates, such as color correction and Gaussian blur, but determined that the error rate was not improved. Jeanette focused on implementing a "box" to organize the data regarding points attributes of the box. By Friday the live camera feed was also quite reliable.

The paper we read this week was voted Best Paper at the IEEE Robotics Conference, so in our analysis of it we focused on its superlative virtues, such as clarity, focus, interest, and conciseness. We also greatly envied the very cool autonomous flying vehicles.

Paper: "Towards a Navigation System for Autonomous Indoor Flying," Slawomir Grzonka, Giorgio Grisetti, Wolfram Burgard, 2009.

Week 7: July 13 - 17.

This week was focused on improvements to the box-finding code. As a temporary measure, we modified it to store the box information in a file which is sent to the robot. Last week the square finding code had been taking frames from the camera twice a second. Dr. Parker suggested that we modify it to only communicate every 3-5 seconds. We tested the code on real boxes in the hallway. We fixed the way boxes were drawn, and slightly improved the way they were found. We also reviewed the localization algorithms in preparation for converting pixel references into map relevant locations so that the robot will be able to use the box collection data. We also experimented with different camera angles and conversion of pixel data into physical distance using camera focal length. I also updated the main robot control code to take in information about boxes from a pipe, processes the data, and incorporate that data into the decision procedure. In order set up the pipe in the correct way, I worked through assignments focusing on inter-robot communication that Dr. Parker had given her students.

Week 8: July 20 - 24.

Things became at once more exciting and much slower when we moved to testing our code on the physical robots rather than in simulation. The robots were having network connection problems, so we worked with the graduate students for a few days to resolve these issues. They had recently been modified to connect only through an ad-hoc network, and the network was still quite finicky. But eventually we were able to reliably connect to the robots and begin testing and modifying our code and integrating its pieces.

We also set up the camera to point straight down on the box area, allowing us to dispense with focal length issues. Although this allowed us to reliable convert pixel-diistances to real distances such that the robot could locate boxes in its own co-ordiante system, the camera's field of vision was somewhat limited, so we considered adding a second camera.

The square finding code was working reliable enough that we were able to start testing the robot's ability to play "box tag," or tag the boxes in the order determined by the decision procedure, as a preliminary to the box pushing task.

Week 9: July 27 - 31.

As we reached the end of our internship, our time was split between three goals. We worked to prepare a forty minute presentation for the graduate students, both to help us practice presenting our work and to inform the graduate students returning from his own summer internship to pick up our work of what we had done. We spent part of the first half of the week evaluating the reliability of our code, preparing slides, writing our presentations, and practicing. We discovered that both ALLIANCE and the square finding algorithm were quite reliable. The former correctly identified the proper task in simulation, and box-tagging tests showed promising results in tests. The latter detected no false negatives and rare false positives, most of which were eliminated since they were not found repeatedly and therefore did not persist.

I also finished the communications socket which passes box information from the camera to the robot. And Jeanette added a second camera to the camera system. We did not yet stitch the two images together, which would help to speed processing time. Instead, the images are processed separately and then found boxes are merged and sent in one file.

Week 10: August 3 - 6.

Our last week was a flurry of activity. Although all of the parts which we had planned to create existed at the beginning of the week, they were not all yet fully integrated. We also needed to write documents for and talk to the graduate student who would be taking over and continuing our project. Explaining our summer of work to him was bitter sweet. As I wrote out the procedure for connecting to the robot and the diagram of our code connections, I felt proud of our work and sad to be leaving it. We made some critical changes to the code during this week as well. We fixed the both ends of the communications socket, allowing for more reliable communications. We found a piece of the image processing code which was slowing down the square finding and sped it up, which led to dramatic performance increases. We also decreased the size of the captured camera images. Although we were not able to stitch the two images together, the speed increase meant that the images were processed within the specified sending period. These combined changes helped the camera code to work efficiently and smoothly. We were able to leave John a functioning robot which used camera data to find boxes and chose between tasks based on them.

Back