July 23 - July 27

The TAC SCM Prediction Challenge final round was this Tuesday. Before it started, I tried to add more features to the support vector regression component prices predictor. The first feature I added was future customer demand. Amy argued that the future customer demand predicted by the Bayes' model might affect current component prices, because agents could conceivably use the same predictions in their own procurement decisions. For example, they might request more of a certain component now if they think that customers will request more of a computer type that requires that component in the future. This made sense, so I added several features to the SVR predictor that reflected the future customer demand. These features were determined by two parameters: the number of different intervals in the future, and the length of those intervals. For each interval in the future, I found the predicted customer demand on each day during that interval and took the average.

One other feature I managed to add to the predictor before the final round began was my first version of the future predicted supplier actual capacity, added in the same way as future customer demand, in different intervals. I used a nearest-neighbors approach to predict the actual capacity in each interval given the current market conditions, which at this time were just the current and past component prices for a number of days. The training data consisted of (market conditions, actual capacity) pairs, but of course the suppliers' actual capacities were never known. I just used the mean actual capacity from the most recent market report, which is received every 20 game days. I implemented a general k-NN algorithm using simple Euclidean distance between the feature vectors, and at first I wasn't sure how to weight the neighbors. I had the idea that the weights should be inversely proportional to the neighbors' distances, but I didn't know how to make them add up to 1 no matter how many neighbors there were until I realized that I could normalize them by dividing each weight by the sum of all weights.

Before the competition started, I had trouble combining different current and future component price predictors. Due to the object-oriented hierarchy I had set up, I couldn't choose different predictor classes for current and future component prices. This is something I would eventually like to fix, but I didn't have time before the competition, so I just inlined the code for the two different predictors--yuck.

Then came the actual competition. Our agent ran smoothly, but I wasn't too happy with the results. The computer prices predictions weren't very good--third place out of four teams--but we didn't expect them to be. We had gotten first place during the qualifying round on current component prices, so I was disappointed when we got second place this time on both current and future component prices. Amy tried to comfort me by saying that we had beaten TacTex, who came in third place. Still, I think none of the agents beat any other by a great margin. Amy talked to me about attending the Women in Machine Learning Workshop. I'm kind of interested, but we haven't obtained very good results, so I don't really want to talk about my own work.

After the final round ended, I continued trying to improve the component price predictions. I implemented a k-NN predictor that uses the same features that the SVR predictor does. It turned out to perform a lot better than the SVR predictor, but that's not surprising, since Deep Maize uses a k-NN approach and beat us in the final round. Still, the NN predictor that I implemented was worse than the simplest method from Botticelli, so it's even further from beating Deep Maize's approach.

After playing around with the NN predictor for a while with different features, I added more features to the SVR predictor. For one thing, I realized that perhaps I should use the NN future capacity predictor even for the "current" actual capacity, because when making future price predictions, the "current" is a time in the future. Since the NN future capacity predictor needs the "current" market conditions, I would also need some way to predict future component prices. I was thinking of adding future component prices as a feature anyway, so I implemented a simple predictor for future component prices by using supplier offers from past games. Each offer's price is stored into an array indexed by the component, supplier, and day on which the offer was received. I added the future component prices on different intervals in the future, again taking the average price in each interval.

After adding all these features, I wrote another script to test the SVR predictor and the NN predictor using different combinations of features. I'll look at the results after the weekend, but I'm not very hopeful, since I've looked at some preliminary ones.