Image by Matthew Scroggs, who also built the pictured machine
There’s been a lot of fuss made about artificial intelligence in the last twelve months, and some of it has been about artificial intelligence and intellectual property.
It’s all been rumbling on for a while of course, but 2023 was the year that the Supreme Court said a final “no” to a patent for an invention allegedly invented by a machine[1], the High Court said “yes” to an invention which trains a neural network to “reflect emotional perception” of music[2] (an appeal is now pending), and the opening salvos were filed in a copyright infringement claim saying that images were used “unlawfully as input to train and develop [synthetic image generation software] Stable Diffusion”[3]. Meanwhile in China, the Courts have recognised a copyright claim to an image created using Stable Diffusion[4].
These cases contain just some of the IP issues presented by the rise of AI. None of them are simple. This is all about what is ostensibly quite advanced technology, and that can make it difficult to understand and communicate the IP issues. When I have the time I will try to get my head around what Emotional Perception say they have invented and then I might know what to think about the High Court’s decision…
In the meantime though I have been reminded about some less-advanced technology. Made out of matchboxes by Donald Michie in 1961, MENACE is a machine that can play noughts-and-crosses. More importantly, MENACE is a machine that can learn to play noughts and crosses well. If you haven’t heard of it (or if like me you have a vague memory of someone telling you about it once, but you can’t remember exactly how it worked), I will try to explain.
MENACE uses a total of 304 matchboxes. There is one matchbox for each possible state of the game during play. To find out which move to make, the “operator” of MENACE finds the matchbox corresponding to what the board looks like at the moment. For example, it is MENACE’s turn to move, playing crosses, and at the moment the game looks like this:
Somewhere among the 304 matchboxes, one box will have a label corresponding with this state of play. The operator finds the right box, and opens it. Inside the box there are coloured beads. The operator closes his eyes and chooses a bead at random. The colour of the bead drawn out determines what move MENACE will make next, according to a colour key.
So, if the bead drawn out of the box is a blue bead, then MENACE will put a cross in the bottom middle.
It is easy to see that this is a pretty stupid move. Noughts will easily win on the next turn by playing in the bottom left square. MENACE is not very clever just yet. The machine is playing basically at random, and will lose quite a lot against a good human player.
But now comes the trick. Because MENACE has lost the game, the beads drawn out during play are not put back in the boxes. From a starting point where everything is possible, losing strategies become less and less likely as beads are removed. After enough games, eventually there will be no blue (or red, green or yellow) beads left in our example box.
If MENACE should win a game, not only are the beads put back, but three additional “bonus” beads of the same colour are added to each box, reinforcing the winning path through the game. If the result is a draw, then MENACE gets one additional bead in each box. Eventually the distribution of beads in the boxes makes MENACE quite an expert noughts-and-crosses player. Against another expert player, of course the result will always be a draw.
Take note that nobody is expressly giving MENACE any advice about strategy for playing a good game of noughts and crosses. The “teacher” is being very unhelpful. MENACE is never told “if your opponent has two noughts in a row, make sure you block him”. MENACE is never even told “the object of the game is to get three crosses in a row”. The only feedback MENACE gets is the “reward” or “punishment” (extra beads or beads taken away), reflecting the win, lose or draw outcome of the game. And yet the machine “learns”.
MENACE’s designer has however made some careful decisions about how many beads to put in for a win or a draw, and how many to take out for a loss. In machine learning, this is called the reward function, and coming up with a good one is often critical to a machine that efficiently learns a good strategy.
Michie’s low-tech machine illustrates the principle of reinforcement learning. This remains an important technique in AI today, and I have worked on patent applications myself that rely on it. The basic idea that a machine can learn by being “rewarded” for good moves and “punished” for bad ones finds applications in all sorts of difficult decision problems. How long should a traffic light stay green at a busy junction? What is the best medicine to treat this patient? What objects appear in this picture? Reinforcement learning has been used to do all of this and more.
If you want to make a machine which can play a more complicated game, like chess, then you might suspect that you will need more matchboxes. You will find that unfortunately there are not enough matchboxes in the entire known universe. You can use a lot of virtual matchboxes on your computer, but you still won’t have anywhere near enough. Working around this universe-wide matchbox shortage is one good reason that developing AI systems still needs clever people making clever inventions, not just faster computers. I hope to meet some more of them this year.
[1] https://www.supremecourt.uk/cases/uksc-2021-0201.html [2] https://www.bailii.org/ew/cases/EWHC/Ch/2023/2948.html [3] https://www.bailii.org/ew/cases/EWHC/Ch/2023/3090.html [4] https://ipkitten.blogspot.com/2023/12/chinese-court-deems-ai-generated-image.html
ASK AN ATTORNEY
Author
