Charlie Meyer's Blog

Chess, Free Will, and LLMs

As of this writing, I have an elo rating of 899 on chess.com ten minute games, which puts me in the 73rd percentile of players on the site. As I’ve climbed the rating ladder, I’ve experienced intermittent bouts of chess-induced anxiety. Even though my opponents are randomly chosen chess.com users like “deleri82”, “filipp63”, and “zaxy889”, losing feels bad. Based on a combination of chess.com’s own analysis and my expertly crafted “chess_games” Google Sheet, losing a game often comes due to a blunder – a move that objectively lowers one's chances by a large degree (basically, losing a piece or getting yourself checkmated).

I could have done better

In any given chess position, there are many available legal moves and the task at hand is to pick the move that best improves your chances of winning the game. When I sit down for a chess game, there is no one forcing me to lose my knight on the 8th move. In my life there are hardly any choices that are more “free” than how to move my chess pieces, yet I make at least one horrible move in almost every single game I sit down to play. I know the rules of chess, and in the vast majority of cases, it seems as though I could have done better.

This feeling of I could have done better is the source of my chess-induced anxiety. When a position becomes sufficiently complicated and time is ticking off the clock, a move eventually occurs to me as “best” and I make it. While nobody is forcing me to make my moves, there is really no agency in this process. Either the correct move will occur to me in the ~20 seconds I have to make a move, or it won’t.

I couldn’t do better but I can improve

For years, I thought that chess was a pointless pursuit as at the highest levels, preparation and memorization of opening lines and algorithms for forced checkmate in endgames have so much importance. But a game of chess needs to be thought of in the context of a larger system of training, where players engage in economic decisions about what preparation to prioritize given limited time and cognitive resources. Optimizing the training process is what matters – how can I train now so that the right move occurs to me during the game?

The idea that “practice makes perfect” isn’t exactly a groundbreaking theory, but the idea that the purpose of practice is to help you force yourself to play well is to some degree freeing. Once the training process is done, I should enter matches without anxiety, knowing that the right move will now occur to me more often than not.

Large Language Models (LLMs) like GPT-4 are often (correctly) described as “autocomplete on steroids”. Given their training data and prompt, they will merely produce the most likely next token until their response is complete. While playing a game of chess, GPT-4 is forced to respond with what it has calculated to be the most likely correct move in the situation. [1] The remarkable thing about GPT-4 is that the responses it is forced to produce are so often correct in so many diverse situations. This is a testament to GPT-4’s training, which was optimized for producing good results for human users on so many topics.

It seems as though GPT-4 and I play chess in a very similar way, so what’s the difference between us? I can choose what I train for. For me, that is chess, programming, teaching, and playing the clarinet [2]. GPT-N will eventually be a better chess player, programmer, teacher, and clarinet player than I am, but I at least get to choose where I’d like to dedicate my time. [3]





[1] Yes there is some stochasticity involved with the temperature of the model but this doesn’t change GPT-4’s freedom here

[2] Yes I was popular in high school

[3] If you recursively apply the idea that there is no choice in the chess game you might arrive at the conclusion that there is no choice to train for chess either. You’re probably correct.