Seeing this code went a long way to help me understand Q-learning. Thanks for writing up and sharing such a simple example.