Interesting work, and here are a few thoughts (in no particular order).
10 seconds seems very high for what's being done here. We can get realtime facial recognition (at like 30-60fps). Recognizing chess pieces ought to be trivial by comparison.
If you're taking the rules seriously, there should be a lot more rules: there are many (many) invalid board states. It seems strange focusing on just a couple. In fact, it seems strange looking at rules altogether. Why not simply have your algorithm figure out what piece is at what position?
SURF or SIFT might be useful here; where you're looking for features that identify specific pieces. In other words, I'm not totally sure if ML is necessary. I would consider processing the image with something like a canny filter and then running a shape detection on the contours.
Either way, really cool project, and great write-up!
The chess pieces don't have to be recognized via a neural network. You just need to track the delta of fields with chess pieces between moves and then you know which piece moved to which field.
So, just an optical recognition of empty field vs used field is required. This works since the starting position of chess pieces is always the same.
This way old inductive chess computer from Mephisto worked (the high-end models had inductive fields).
I don't think this is being built to follow along a game in progress (though that would be an interesting application) but rather capture a current position and transfer it to a digital representation, for example for computer analysis. So there is no delta to track.
Rated standard tournament chess involves a tedious process of annotating moves on a carbon copy as you play. One to give to the tournament director to enter the game into the database and one to review the game after. So there is a niche market for this.
Annotating each move manually seems a lot faster than the MVP app in the article, which seems to take some 10 seconds to recognize the whole game state.
It also seems suited towards an altogether different use case, where you don't already know the previous game state or the moves that resulted in it. This recognizes the entire game state over and over again, which is a lot of redundant work if you already know the previous game state and only consider valid moves from that state.
For the purpose of recording an entire game, a much simpler computer vision approach would be possible based on the previous state. Just find the empty square where there was previously a piece, consider where it might have ended up according to its possible moves and look for changes in those squares. The hardest CV issue then is figuring out what a pawn was promoted to.
This makes no difference. Players are required to keep notation in tournament games, and the use of external software (like what is used at the top boards at some tournaments) does not absolve the players of this responsibility. Writing down the moves of the game as it is being played is a major part of playing chess, and it is one of the most important things for players to learn because it directly improves play (even disregarding reviewing the games afterwards!).
Players are required to keep notation in tournament games, and the use of external software ... does not absolve the players of this responsibility
New technology can change the rules. I'm not sure about FIDE, I'm a former USCF TD. The USCF began allowing digital scoresheets many years ago. No doubt sponsorship from Monroi played a roll in the rule change, but it happened none the less. I can easily see the use of digital chessboards eventually eliminating the requirement to manually write moves down.
There are also many instances where players are not required to notate (at least under USCF rules). Like when a player's time remaining drops below 5min in sudden death time controls. When a player can't read or write. Blind, or other handicapped players. And blitz games.
At many of my tournaments it was pretty routine for parents to set up video cameras to record the games because the player's were too young to read or write.
I am a chess professional, although more as a teacher than a player, so I have a lot of experience in this field. As caro_douglos pointed out, it slows down the initial reaction to opponent's moves, which is one reason why it instantly improves performance for new players. It forces the player to settle into a certain rhythm of play which prevents silly mistakes.
It is also permitted (and encouraged) to write down the time on the clock after each move, and to record all draw offers, both of which are valuable information during and after the game. If one of my goals is to grind down my opponent's clock in a complex position where I have the initiative, seeing how much more time he is using on average gives me an idea of how long I need to maintain the tension to gain a sufficient time advantage. For example, in a game where both players start with two hours, entering the endgame with 1 hour versus my opponent's 15 minutes is an enormous advantage, even in a position which should otherwise be drawn with perfect play.
Also, in some time controls, players gain extra time after a certain move (generally the 40th), so it can be very important to know exactly when you're going to get that extra hour. These time controls are less common than they used to be.
I have students as young as 5 years old learn how to keep notation after only a few weeks of learning chess. It is a very important skill.
When it comes to studying for an upcoming tournament match you will know who your opponent is so you have the ability to study up on their previous play. Maybe they play a certain style against your mainline go-to that you can exploit.
i think your excessively opposed to this. a simple solution that records all the games has obvious benefits. market is probably too small anyway though, i don't see carbon copies going away any time soon.
To the contrary, many chess players are opposed to the increasing computerization of the game. Computers have already devastated opening theory (by finding solutions to difficult lines and forced draws in others) and made cheating 100x easier, requiring draconian monitoring at tournaments just to keep them fair. Even if a surveillance camera could track every game and create a scoresheet for every game in a tournament (which I strongly doubt is possible), this is the kind of thing I want out of the playing hall.
This isn't a computerization of the game, its a computerization of the recording of the game. Big difference and has nothing to do with computer chess engines. It could be used to detect cheating by comparing played moves to engine lines. Also a board camera isn't the the only way this could be implemented, rfid's in the pieces or a pressure sensitive board would both work. Players could still take their own game notes. It would also eliminate errors in the recording of the game and provide records for blitz and time pressure situations.
If you're going the ML way, I recommend an end-to-end approach to get significantly more performance (at a data preparation and training cost). Simply generate a dataset of chess boards, and then render them using a 3D software of choice (pretty easy job in blender/etc.). For robustness then you can add various kinds of perturbations, like different camera angles, small displacements for each piece, etc.
10 seconds seems very high for what's being done here. We can get realtime facial recognition (at like 30-60fps). Recognizing chess pieces ought to be trivial by comparison.
If you're taking the rules seriously, there should be a lot more rules: there are many (many) invalid board states. It seems strange focusing on just a couple. In fact, it seems strange looking at rules altogether. Why not simply have your algorithm figure out what piece is at what position?
SURF or SIFT might be useful here; where you're looking for features that identify specific pieces. In other words, I'm not totally sure if ML is necessary. I would consider processing the image with something like a canny filter and then running a shape detection on the contours.
Either way, really cool project, and great write-up!