Programmers Attack Go With Brute Force

Last June an article by Jonathan Schaeffer, Martin Müller & Akihiro Kishimoto, AIs Have Mastered Chess. Will Go Be Next? was published. “Randomness could trump expertise in this ancient game of strategy,” followed. “Jonathan Schaeffer, a computer science professor at the University of Alberta, in Canada, had been creating game-playing artificial intelligence programs for 15 years when Martin Müller and Akihiro Kishimoto came to the university in 1999 as a professor and graduate student, respectively. Kishimoto has since left for IBM Research–Ireland, but the work goes on—and Schaeffer now finds it plausible that a computer will beat Go’s grand masters soon. “Ten years ago, I thought that wouldn’t happen in my lifetime,” he says.” (http://spectrum.ieee.org/robotics/artificial-intelligence/ais-have-mastered-chess-will-go-be-next)

Jonathan Schaeffer is the man behind Chinook, the computer program that solved Checkers. You can find the paper, Checkers is Solved, to learn about the proof here: (http://webdocs.cs.ualberta.ca/~chinook/)
He has also revised his book first published in 1997, One Jump Ahead: Computer Perfection at Checkers, which I read years ago. Jonathan Schaeffer is like E. F. Hutton in that when he talks about a computer game program, you listen.

For years I have followed news of computer Go programs. Before sitting down to punch & poke I searched for the latest news, coming up empty. This as good news for humans because Go is the last board game bastion holding against machine power. It is also the world’s oldest, and most complicated, board game. It “originated in ancient China more than 2,500 years ago. It was considered one of the four essential arts of a cultured Chinese scholar in antiquity. Its earliest written reference dates back to the Confucian Analects.” (https://en.wikipedia.org/wiki/Go_%28game%29)

Schaeffer and his group have developed a Go-playing computer program, Fuego, an open-source program that was developed at the University of Alberta. From the article, “For decades, researchers have taught computers to play games in order to test their cognitive abilities against those of humans. In 1997, when an IBM computer called Deep Blue beat Garry Kasparov, the reigning world champion, at chess, many people assumed that computer scientists would eventually develop artificial intelligences that could triumph at any game. Go, however, with its dizzying array of possible moves, continued to stymie the best efforts of AI researchers.”

In 2009 Fuego “…defeated a world-class human Go player in a no-handicap game for the first time in history. Although that game was played on a small board, not the board used in official tournaments, Fuego’s win was seen as a major milestone.”

They write, “Remarkably, the Fuego program didn’t triumph because it had a better grasp of Go strategy. And although it considered millions of possible moves during each turn, it didn’t come close to performing an exhaustive search of all the possible game paths. Instead, Fuego was a know-nothing machine that based its decisions on random choices and statistics.”

I like the part about it being a “know-nothing machine.” I have often wondered if humans, like Jonathan Schaeffer, who are devoting their lives to the development of “thinking” machines, will be reviled by future generations of humans as is the case in the Terminator movies. It could be that in the future humans will say, “Hitler was nothing compared to the evil SCHAEFFER!” If I were supreme world controller a command would be issued ending the attempts to crack Go, leaving my subjects one beautiful game not consigned to the dustbin of history, as has been the fate of checkers. I fear it is only a matter of time before chess meets the same fate. GM Parimarjan Negi was asked in the “Just Checking” Q&A of the best chess magazine in the history of the universe, New In Chess 2014/6, “What will be the nationality of the 2050 World Champion?” He answered the question by posing one of his own, “Will we still have a world championship?” Good question. I would have to live to one hundred to see that question answered. Only former President of the GCA, and Georgia Senior Champion, Scott Parker will live that long, possibly still be pushing wood in 2050, if wood is still being pushed…

The article continues, “The recipe for building a superhuman chess program is now well established. You start by listing all possible moves, the responses to the moves, and the responses to the responses, generating a branching tree that grows as big as computational resources allow. To evaluate the game positions at the end of the branches, the program needs some chess knowledge, such as the value of each piece and the utility of its location on the board. Then you refine the algorithm, say by “pruning” away branches that obviously involve bad play on either side, so that the program can search the remaining branches more deeply. Set the program to run as fast as possible on one or more computers and voilà, you have a grand master chess player. This recipe has proven successful not only for chess but also for such games as checkers and Othello. It is one of the great success stories of AI research.”

Voilà, indeed.

“Go is another matter entirely,” they write, “The game has changed little since it was invented in China thousands of years ago, and millions around the world still enjoy playing it.”

But for how long?

“Game play sounds simple in theory: Two players take turns placing stones on the board to occupy territories and surround the opponent’s stones, earning points for their successes. Yet the scope of Go makes it extremely difficult—perhaps impossible—for a program to master the game with the traditional search-and-evaluate approach.”

This is because, “For starters, the complexity of the search algorithm depends in large part on the branching factor—the number of possible moves at every turn. For chess, that factor is roughly 40, and a typical chess game lasts for about 50 moves. In Go, the branching factor can be more than 250, and a game goes on for about 350 moves. The proliferation of options in Go quickly becomes too much for a standard search algorithm.”

Hooray! That is the good news, and there is more…”There’s also a bigger problem: While it’s fairly easy to define the value of positions in chess, it’s enormously difficult to do so on a Go board. In chess-playing programs, a relatively simple evaluation function adds up the material value of pieces (a queen, for example, has a higher value than a pawn) and computes the value of their locations on the board based on their potential to attack or be attacked. Compared with that of chess pieces, the value of individual Go stones is much lower. Therefore the evaluation of a Go position is based on all the stones’ locations, and on judgments about which of them will eventually be captured and which will stay safe during the shifting course of a long game. To make this assessment, human players rely on both a deep tactical understanding of the game and a clear-eyed appraisal of the overall board situation. Go masters consider the strength of various groups of stones and look at the potential to create, expand, or conquer territories across the board.”

This sounds good so far, but then they continue, “Rather than try to teach a Go-playing program how to perform this complex assessment, we’ve found that the best solution is to skip the evaluation process entirely.”

Oh no, Mr. Bill!

“Over the past decade, several research groups have pioneered a new search paradigm for games, and the technique actually has a chance at cracking Go. Surprisingly, it’s based on sequences of random moves. In its simplest form, this approach, called Monte Carlo tree search (MCTS), eschews all knowledge of the desirability of game positions. A program that uses MCTS need only know the rules of the game.”

I do not know about you, but I am hoping, “What happens in Monte Carlo stays in Monte Carlo.” Do you get the feeling we are about to be Three Card Monte Carloed?

“From the current configuration of stones on the board, the program simulates a random sequence of legal moves (playing moves for both opponents) until the end of the game is reached, resulting in a win or loss. It automatically does this over and over. The magic comes from the use of statistics. The evaluation of a position can be defined as the frequency with which random move sequences originating in that position lead to a win. For instance, the program might determine that when move A is played, random sequences of moves result in a win 73 percent of the time, while move B leads to a win only 54 percent of the time. It’s a shockingly simple metric.”

“Shockingly simple,” my jackass. There is much more to the article, including this, “The best policies for expanding the tree also rely on a decision-making shortcut called rapid action value estimation (RAVE). The RAVE component tells the program to collect another set of statistics during each simulation.”

As in “Raving lunatic.” The article provides a list of what current computer programs have done to games, and how they rate in “…two-player games without chance or hidden information…”

TIC-TAC-TOE (Game positions, 10 to the 4th power) = Toast

OWARE (Game positions, 10 to the 11th power) = Fried

CHECKERS (Game positions, 10 to the 20th power)= Cooked

OTHELLO (Game positions, 10 to the 28th power)= Superhuman

CHESS (Game positions, 10 to the 45th power) = Superhuman

XIANGQI (CHINESE CHESS) (Game positions, 10 to the 48th power) = Best Professional

SHOGI (JAPANESE CHESS) (Game positions, 10 to the 70th power) = Strong Professional

GO = (Game positions, 10 to the 172th power) = Strong Amateur

They end the article by writing, “But there may come a day soon when an AI will be able to conquer any game we set it to, without a bit of knowledge to its name. If that day comes, we will raise a wry cheer for the triumph of ignorance.”

I would much prefer to raise a stein and drown my sorrows to that…

Garry Kasparov Tangled Up in Deep Blue

When world human chess champion Garry Kasparov lost the second match with Deep Blue in 1997, I said, and have continued to say, and write, that Garry Kasparov will be remembered only for losing to the chess program known as Deep Blue. Many find this unpalatable, but, as Walter Cronkite used to say to end his CBS news broadcast, “That’s the way it is.”
Proof can be found on This Day in History under May 11:
“On May 11, 1997, chess grandmaster Garry Kasparov resigns after 19 moves in a game against Deep Blue, a chess-playing computer developed by scientists at IBM. This was the sixth and final game of their match, which Kasparov lost two games to one, with three draws.” (http://www.history.com/this-day-in-history/deep-blue-defeats-garry-kasparov-in-chess-match)
This is the only listing for Kasparov. There is absolutely nothing concerning any of his other chess accomplishments. It may be unfortunate for Garry, but this is how he leaves his make on history, as a loser.
The above mentioned article ends with, “The last game of the 1997 Kasparov v. Deep Blue match lasted only an hour. Deep Blue traded its bishop and rook for Kasparov’s queen, after sacrificing a knight to gain position on the board. The position left Kasparov defensive, but not helpless, and though he still had a playable position, Kasparov resigned–the first time in his career that he had conceded defeat. Grandmaster John Fedorowicz later gave voice to the chess community’s shock at Kasparov’s loss: “Everybody was surprised that he resigned because it didn’t seem lost. We’ve all played this position before. It’s a known position.” Kasparov said of his decision, “I lost my fighting spirit.”

Many have called Garry Kasparov the greatest chess player in the history of the game. I have always wondered why. I mean, if a player loses the biggest match of his life, a match in which he was fighting for the honor of the human race, how can anyone in their right mind consider him to be the greatest? Garry Kasparov will always be considered a loser by the public.

The chessgames.com website provides the final game of the match, naming it, “Tangled Up in Blue.”
Deep Blue (Computer) vs Garry Kasparov
“Tangled Up in Blue” (game of the day Sep-12-05)
IBM Man-Machine, New York USA (1997) · Caro-Kann Defense: Karpov. Modern Variation (B17) · 1-0
1. e4 c6 2. d4 d5 3. Nc3 de4 4. Ne4 Nd7 5. Ng5 Ngf6 6. Bd3 e6 7. N1f3 h6 8. Ne6 Qe7 9. O-O fe6 10. Bg6 Kd8 11. Bf4 b5 12. a4 Bb7 13. Re1 Nd5 14. Bg3 Kc8 15. ab5 cb5 16. Qd3 Bc6 17. Bf5 ef5 18. Re7 Be7 19. c4 1-0
http://www.chessgames.com/perl/chessgame?gid=1070917

Garry Kasparov lost with the Karpov variation. Cogitate on that one for a moment. Consider why Kasparov would even consider playing a variation named for the previous World Champion, whom he had dethroned. The variation was totally out of character for Kasparov. The only way this makes any sense to me is that Garry Kasparov took a dive. The term “take a dive” means to lose intentionally, as when a prize fighter loses because the fix is in, like Sonny Liston did when he hit the mat against Cassius Clay, later known as Muhammad Ali, in their 1964 title fight. The match meant a great deal to IBM, especially in winning the match. How much was it worth to IBM? Jonathan Schaeffer, of the Department of Computing Science at the University of Alberta, the man behind the program of the now World Checkers Champion, Chinook, that cannot lose (Computer Checkers Program Is Invincible (http://www.nytimes.com/2007/07/19/science/19cnd-checkers.html?_r=0) see also (http://phys.org/news104073048.html) had this to say:
“The victory went around the world. IBM estimated it received $500 million of free publicity from the match, and IBM stock prices went up over $10 to reach a new high for the company. (http://askeplaat.wordpress.com/534-2/deep-blue-vs-garry-kasparov/)

NPR featured a story August 8, 2014, “Kasparov vs. Deep Blue.” It can be heard here: (http://www.npr.org/2014/08/08/338850323/kasparov-vs-deep-blue)
A transcript is also provided:
In 1997, Deep Blue, a computer designed by IBM, took on the undefeated world chess champion, Garry Kasparov. Kasparov lost. Some argued that computers had progressed to be “smarter” than humans.

GLYNN WASHINGTON, HOST:
And speaking of Stephanie Foo, she likes to take things literally. And I told her, I said Stephanie, don’t be such a drag. You’ve been smoking the company line, you got to loosen up, come on, the rules are meant to be broken – it’s time to rage against machine – lady rage, come on. Well, Stephanie – Stephanie promptly brought me a story about raging against a machine. The real machine and someone raging against it. Stephanie Foo, take it away.
STEPHANIE FOO, BYLINE: OK, yes. This story is about chess, but not just any chess game – one of the most famous ever.
(SOUNDBITE OF ARCHIVED RECORDING)
UNIDENTIFIED MAN: Deep Blue and Garry Kasparov, the world’s chess champions…
FOO: It’s 1997 – world chess champion Garry Kasparov versus Deep Blue, a computer designed by IBM. And for people who wanted to believe that the human brain was still stronger than computers, this was a huge deal.
(SOUNDBITE OF ARCHIVED RECORDING)
MAURICE ASHLEY: This is international chess match, I’m Maurice Ashley. The future of humanity is one the line. Now the weather.
FOO: Now Kasparov has never lost a match – ever. He was destroying all the grandmasters at the age of 22. He’s even beaten Deep Blue once before, so he is going into this rematch totally confident, and true enough – bam – Kasparov wins game one easy.
(Applause)
FOO: But then game two is where everything starts to go wrong. In this match, Deep Blue is dominating. Kasparov is visibly frustrated. He’s is rubbing his face, sighing, and then abruptly Kasparov just walks off the stage and quits – forfeits the game. The night after the game, his fans analyze the match and figured something out – something Kasparov, an undefeated grandmaster should have seen. If he had not stormed off the stage and just played his normal game, he could’ve tied Deep Blue.
(SOUNDBITE OF ARCHIVED RECORDING)
UNIDENTIFIED FEMALE: The match now stands at one game apiece.
FOO: Now, the match was best of five games, with Kasparov eventually losing the whole thing, but the turning point was when he forfeited that match. So since 1997, people have always speculated – what happened in game two? Did he quit because the computer was really so much smarter than he was? Then recently this book by Nate Silver came out called “The Signal And The Noise.” In it Murray Campbell, one of the engineers who created Deep Blue and who was at the match, comes out and says that he thinks he knows what really happened, and he says it starts in the first game – the game Kasparov won.
MURRAY CAMPBELL: Near the end of game one Kasparov had reached a very strong position. It was clearly to any chess expert in the audience, that Deep Blue was going to lose in the long run.
FOO: But here’s where it’s interesting. At the end of the game Deep Blue did something weird – it committed suicide.
CAMPBELL: Deep Blue was calculating a particular move that it could make that would prolong the game as long as possible. And then at the last second, it switched to a completely different move and played it.
UNIDENTIFIED MALE #2: Rook to D1.
CAMPBELL: And this particular move was really bad, and so it caused us to give up the game right away.
FOO: This really bad move confused Kasparov. Murray says he heard Kasparov’s team stayed up that night trying to analyze the logic behind that move – what it meant. The only thing was – there was no logic.
CAMPBELL: The more obvious explanation is that there was a bug.
UNIDENTIFIED MALE #2: Uh-oh.
FOO: A glitch – the kind of plot twist only a nerd could love.
CAMPBELL: Due to a bug in the program, unfortunately, it had played a random move.
FOO: But Kasparov didn’t know that, and Murray guesses that Kasparov was so caught up thinking the machine do something that he didn’t that he lost it, and the whole rest of the match was a landslide.
CAMPBELL: My theory is that Kasparov might have seen the drying opportunity but didn’t, because he was overestimating Deep Blue’s capability and assuming that it was incapable of making a mistake that would allow a draw. Deep Blue was very strong but wasn’t that strong. And I don’t know if this is true or not – I think we’ll never know unless Kasparov says himself, but you probably won’t get to talk to him because he doesn’t like to talk about the subject.
FOO: Yeah, Kasparov spent suggesting that IBM cheated, and he hasn’t really talked about the game for many years – until now.
MIG GREENGARD: You have to understand, he’s a little frustrated talking about this stuff over and over again sometimes.
FOO: That’s Mig Greengard. He’s been Kasparov’s aid, publicist and confidant for 14 years. And he’s here to speak on Kasparov’s behalf.
GREENGARD: He’s authorized me to talk with you about it. I talked with Garry about it…
FOO: It being the glitch.
GREENGARD: …And what he said to me – he said it’s ridiculous that move had no impact on his subsequent play – and had no impact on him – that’s it, move on. So that’s all really that I can – that I can go with, is the horse’s mouth.
FOO: So maybe Murray is wrong about the glitch but Mig says, he’s not wrong about Kasparov having a sort of mental breakdown – it just happened a little later. Mig told me that Kasparov was used to playing with computers. He thought he had them all figured out. Kasparov had certain traps that he would set, lures for computers, and computers would always fall for them. So in game two, Kasparov set his trap and waited.
GREENGARD: Because he had these assumptions that of course being a computer, it’s probably going to play this, this and this.
FOO: But it didn’t – it didn’t take the bait.
UNIDENTIFIED MAN #2: I see what you’re up to.
GREENGARD: The played something else.
FOO: Something good.
GREENGARD: Something that not only is not the predicted computer move – but a very, very strong move.
FOO: So you’re saying that this is the moment where basically he was psyched out.
GREENGARD: Right. It was just very – I think a very confusing, very disorienting experience to have to then sit down at the board not really knowing what you’re facing. Can I still try to trick it? Does is still play like a computer? Does it make mistakes at all? So psychologically damaging to Garry in that he realized this was a whole new animal.
FOO: And then after that really awesome move, Deep Blue actually makes another bad move.
UNIDENTIFIED MALE #2: I guess I’ll play this.
FOO: This bad move is the one that allows Kasparov to tie. But Kasparov is too convinced he’s going to lose to see the fault.
GREENGARD: Like well, no way the computer would allow that – that can’t be there. Whereas against a human you think why not, maybe he made a mistake in his calculations – I’ll give it a shot. Against the computer you get – the computer gets the benefit of the doubt. How could something play like God, then play like an idiot in the same game?
FOO: In a way that’s like a total machine mistake though, right? Because since the machine doesn’t have a specific style or personality like, each different move that it makes could be brilliant and idiotic.
GREENGARD: Sure, sure – of course when he resigned he didn’t know any of this – which itself was demoralizing and humiliating.
FOO: So essentially what Mig’s saying is that Deep Blue wasn’t necessarily as smart as we all thought. Deep Blue didn’t have this magnificent triumph over Kasparov, it was more that Kasparov forced himself to fail.
GREENGARD: In actually turned out to be a bit of a red herring as far as artificial intelligence goes. It turned out it didn’t have emulate human thought to beat the world champion. It didn’t even have to play great chess, but it mostly revealed that humans aren’t perfect – humans make mistakes. They certainly – it turned out to be less complicated than we’d hoped. Deep Blue could calculate 200 million possible moves per second, but it was Kasparov who is overthinking it.
WASHINGTON: Thanks so much to Mig and Murray for helping us out on that piece. And of course, you’ve got to check out the almighty Nate Silver’s book “The Signal And The Noise.” And yes, that piece was produced by Stephanie Foo. We’ve got issues against the machines today on SNAP. And when we return, the man tries to corrupt me with all the free food I can stuff into my mouth. And we’re going to illegally destroy private property just because we can. On SNAP JUDGMENT the “Rage Against The Machine” episode continues. Rock on and stay tuned.
Copyright © 2014 NPR. All rights reserved. No quotes from the materials contained herein may be used in any media without attribution to NPR.

Amazing Game: Kasparov’s quickest defeat: IBM’s Deeper Blue (Computer) vs Garry Kasparov 1997

Bob Dylan – Tangled Up In Blue

Bob Dylan – Tangled Up In Blue – Live Oslo 2013