The idea this week will be to get Sarsa and Q-Learning agents that select moves according to

Boltzmann selection, and find a few values for the
temperature constant so that we get a good range of possible behaviour. Then we can compare the Sarsa and
Q-Learning agents with Boltzmann and Epsilon Greedy, with various values for epsilon and the temperature.

First things first. Since the agents will need to learn over many episodes, it may take a good long time to
train the agent, and if something crashes halfway through, a lot of data is lost. So I have now implemented save and
retrieve functions for the Q-values. I also made a few other additions to the code, mostly saving various results
to disk rather than just print them to screen, and making it possible to seed the random number generator (for
probabilities). I also tried to make a main function that took input from standard input, but it turns out that I've
only done this in C before, and searched high and low for a way to read an int (just an int, that's all!), but it
seems it's rather complicated in Java. I gave up on this since it is not really essential.

Now for the real work. After a little bit of coding and a little bit of reorganization, I created a surprisingly
bug-free Boltzmann move selector for Sarsa and Q agents. I started off testing it for a Sarsa Learning agent. As a
general rule, as T (temperature) increases, the agent learns to take a 'safer' path around the cliff. When the values
for T get too high, the agent can't seem to stick to its path though, there's too much exploration, but it does believe
in a good path (that is, if it selected greedily, it would find a good path). Something I found particularly interesting
is that at the beginning of its learning session, in the first 100 episodes or so, it believes the 'dangerous' path
is optimal, but as it goes on it panics and even takes the furthest path possible. For example, at T=40, after 200
episodes, the agent thinks the dangerous path is optimal ~60% of the time, and takes a path one square away ~32% of the
time. But after 400 episodse, it prefers the path two squares away (33%) and one square away (31%).

I finished the first tests, and made a few graphs to show the results and compare the various methods. All the
results below are averages from 100 runs of agents learning over 500 episodes.