Linda's explanation of the project

Paralleling Sal's work with the bunnies and clustering of 3D regions that were specified by hand, we are going to use the regions from the color, structure, and texture files to cluster together sets of regions. We will classify each region and presumably use both overlap and known semantic characteristics to then correctly label each region. For the clustering, we will probably use k-means and maybe the EM (expectation maximization) algorithm.

Sal's idea

Sal thinks that this clustering will not be as clean as we might hope, and is anticipating that by introducing the idea of semantic detectors, such as window dectectors, to recognize various classes of objects. Detectors that we would use would need to be robust against variance in :

• scale
• brightness
• perspective
.

Linda has pointed out that scale might not matter--as long as line segments are visible, Yi's structural output should have detected and outputted the sides of any structure that contains segments. Also, Yi's structural algorithms already seem to handle perspective by using the vanishing point of certain segments and then trying to make them parallel, or something along those lines.

Consensus of the approach for the group

For now, we are going to focus on continuing with Yi's work. His approach has not yet been fully explored and the way he is trying to go about clustering and classifying objects is one that no one else is taking right now. Sal's idea can be worked on in parallel, but is not going to be the focus of the group until we have at least run a preliminary round of experiments on what Yi's processing yields.

Immediate tasks

Our next goal is to run a test on a subset of the images collected by Clifford and Jenny (as well as using the images that we already had in the database) to determine how well the structural output is able to distinguish the presence and absense of structure. The thought is to use 500 images with structure and 500 without, then calculating the number of false positives, false negatives, and accurate readings, so that we can determine the confidence that we can have in the structural output. Clearly, we need to keep in mind while running this test that the density of structure detected, as well as color and textural features, will also be taken into consideration when we are doing the final classification of each type of object.