Week 7 Accomplishments:
- SpooNN is detecting multiple objects (that are actually there) and correctly classifying them! There are still entire objects the network misses, but it's still major progress!
- Implemented AP/mAP code to analyze the accuracy of the network--with iou_threshold = 0, mAP is aobut 90%*! Multi-SpooNN is showing some promise!
- Submitted my abstract and poster title to the DREU to be considered for the Grace Hopper Celebration!
*See next week's blog posts for some sad news :(
Monday, July 8:
- Researched non-maximal suppression algorithms (NMS). This will definitely be useful in combining bounding boxes that encompass the same object, but I will have to do more research to see which algorithm is the most effective.
- I fixed a bounding box error--when I left on Friday, all of the bounding boxes were limited to the bottom lefthand corner. Evidently I had switched an 'h' and a 'w' in scaling the bounding boxes back to the original image size, which was the culprit. When I fixed this, the bounding boxes were definitely more accurate.
- I have been playing around with how I choose which bounding boxes to keep (I have 14x14=196 different bounding boxes, and I only want the ones that are around objects in my image). For example, I have tried changing the loss function for objdetect as well as eliminating an activation layer for the classify stage. So far I haven't had any luck, but I will keep trying.
- Because I was feeling a little frustrated at the MOD implementation, I spent the last hour or so updating this website (blog, places I've visted, etc.). I'm hoping that tomorrow I have a few more ideas to try.
Tuesday, July 9:
- I spent the day tweaking various parts of the model to see if they had a positive impact on the bounding box accuracy or classification accuracy. For example, I tried different loss functions, I added confidence thresholds for objdetect and classify and changed the kernel shape to 3 on the last conv layer. Changing the kernel shape seemed to be the most effective
- I later tried to vary the iou_threshold in the training stage and found that with a very low iou_threshold, the bounding boxes are very accurate but the labels are incorrect, and with a higher iou_threshold (like 0.1), the labels are correct but the bounding boxes are way off. Tomorrow I will try to see if there is a value in between 0.0 and 0.1 that produces good bounding boxes as well as accurate labels
- It was a little frustrating/almost boring to have to retrain the network after making each little change. Even though it takes less than 5 minutes to train, when you train the network many times in one day, that time adds up
Wednesday, July 10:
- After not having much success by changing the iou_threshold this morning, I decided to try using the hinge loss function for objdetect (I had not tried it since I changed the kernel shape to 3)
- My network actually detects multiple objects per image with decent bounding boxes and correct labels!! There are still several images where it doesn't detect any objects (??) but there are also significantly more images where it detects 3+ objects! Definitely a step in the right direction
- I wrote an abstract draft for this project (to be submitted to the DREU and Brown Summer Research Symposium); Professor Bahar and Jasmine made many helpful suggestions, which I appreciated!
Thursday, July 11:
- Today Jasmine said that maybe the best approach is to analyze all of the bounding boxes based on their class probabilities and not try to use multiple objects to try to find the "best" bounding boxes.
- Jasmine sent me some code she thinks I will be able to use parts of to do this type of analysis. I spent the day trying to understand the code and figure out how parts would fit into Multi-SpooNN
- We also had our meeting with Michigan since Professor Bahar will be out of town tomorrow.
Friday, July 12:
- I worked to implement the Average Precision (AP) and mean Average Precision (mAP) analysis in Multi-SpooNN. When I first got it working, the mAP was only about 11%, which was disappointing and a little frustrating. Jasmine thought it might just be the accuracy of the network, but I wanted to try changing the iou_threshold back to 0.0 to see if there would be improvements. I'm glad I did, because the new mAP is about 90%!
- Afterwards, I began working to generate heatmaps based on the classification probabilities. I've run into some problems with dimensions not aligning though, so I'll have to continue to look at this next week.