I read the March 27, 2015 edition of the Mechanics’ Institute Chess Club Newsletter, #703, by John Donaldson, the day it was published (http://www.chessclub.org/news.php?n=703). Of particular interest was this, “The Chess Club and Scholastic Center of Saint Louis has done the chess world a favor by conducting a literature review to answer the question, “Does chess provide educational benefits?” (http://news.stlpublicradio.org/post/chess-literature-review-gives-base-claim-chess-educational-benefits)
I clicked on the link to find the headline: On Chess: Literature Review Gives Base To Claim Of Chess’ Educational Benefits. The article was written by Brian Jerauld, who “is the 2014 Chess Journalist of the Year, and the communications specialist for the Chess Club and Scholastic Center of St. Louis. He is a 2001 graduate of the University of Missouri-Columbia School of Journalism and has more than a decade of experience writing about boats, sports and other ways to relax.” It was dated Feb 12, 2015. Since I had posted a series of articles on this very subject during the latter part of February I could not help but wonder how this had been missed. Further pondering brought forth the question of why no one had commented on this study on the blog or on the USCF forum thread concerning my blog posts on the subject. (http://www.uschess.org/forums/viewtopic.php?f=23&t=21185&sid=95eabe87ba3a5d3ce890f4237d794c88)
Mr. Jerauld writes, “By now, the claim that chess comes packaged with hidden educational perks is a hype certainly heard around the world. And how could it not be believed? Just find some random piece of research that supports such big talk, tie it together with obvious, awesome-sounding hyperbole — like “decision-making skills” and “higher-order thinking” — and boom: You’ve got yourself some Grade-A propaganda.
Over the years, all this talk has given a rather rosy-colored narrative that always ends in support of chess curriculum implementation. But recently, the Chess Club and Scholastic Center of Saint Louis, whose scholastic program branched out to more than 3,000 students and hundreds of area schools last year, dropped the rhetoric and set out to discover if chess actually has an effect on its students.
Empirically, can all the talk walk the walk?”
Good question. Brian continues, “A year ago, the CCSCSL set out to apply a rigorous and critical eye of existing chess studies by commissioning Basis Policy Research, an independent research firm that focuses on K-12 educational exploration. The goal was to survey the entire landscape of existing chess research, digging back through more than four decades of random studies, and compile a literature review of what was actually known about chess’ impact on student outcomes.”
This is exactly the kind of study I wrote about in my post of February 27, 2015, Does Playing Chess Make You Smarter? (https://xpertchesslessons.wordpress.com/2015/02/27/does-playing-chess-make-you-smarter/) The study mentioned, Educational benefits of chess instruction: A critical review, by Fernand Gobet & Guillermo Campitelli, (http://www.brunel.ac.uk/~hsstffg/preprints/chess_and_education.PDF) is dated now, having been published a decade ago, so I looked forward with interest to new literature on the subject. In trying to be scientifically objective, and also being a chess player, I relish a refutation. If new information refutes previous thinking, humans advance. For example, studies were done in the middle of the last century on the amount of radiation harmful to humans. During the course of my life the amount needed has been constantly lowered until now it is an accepted fact that even the lowest amount of radiation detected is deleterious to a human being. Decades ago there were many studies done on the effects of smoking cigarettes, funded by the tobacco industry, which concluded smoking caused no problems whatsoever to a human being. Over the years I have learned how difficult it is for old(er) people to change their preconceived ideas. For example, when I was young it was an accepted fact that an ulcer was caused by stress. It is now known that an ulcer is caused by a virus. My father was unable to wrap his mind around that fact, continuing to believe stress was the cause. There are many examples in the scientific community of a scientist with stature in the community refusing to accept evidence contrary to that with which his career was built. Change is difficult, and some people will stubbornly cling to the old ways “come hell or high water.” I try not to be one of those people. I do not care who is right, or wrong, but what is right, and what is wrong.
During the telecast of the third round of the US Championships yesterday the new study was mentioned by Jennifer Shahade when she said, “St.Louis commissioned a study that showed…they didn’t know what the study was going to say. They wanted to find out what kind of connections chess had to academic performance, and surprisingly, the main connection was math and chess.” GM Yasser Seirawan said, “Really?” Jen continued, “Yes, specifically math and chess.” To which Yasser responded, “I remember the Margolis study where it was about reading.” This comes at the 3:42:30 mark of the broadcast.
I clicked on the link and downloaded the PDF in order to read the new study, Literature Review of Chess Studies, By
Anna Nicotera, and David Stuit, dated November 2014. (http://saintlouischessclub.org/sites/default/files/CCSCSL%20Literature%20Review%20of%20Chess%20Studies%20-%20November%202014.pdf)
“This literature review identified 51 studies of chess. Twenty-four of the 51 studies met a set of pre-determined criteria for eligibility and were included in analyses. Results from the literature review were categorized by the quality of the study design and organized by whether the studies examined after-school or in-school chess programs. The main findings from this literature review are:
1. After-school chess programs had a positive and statistically significant impact on student mathematics outcomes.
2. In-school chess interventions had a positive and statistically significant impact on student mathematics and cognitive outcomes.
Although the findings are interesting, they do come with this caveat, “While the two primary outcomes listed above are based on studies that used rigorous research design methodologies, the results should be interpreted cautiously given the small number of eligible studies that the pooled results encompass.”
This is a caveat huge as the Grand Canyon. It is called a “small sample size.” If a baseball player goes one for three during a single game his batting average is .333, which is outstanding. This does not mean he will finish the season with a batting average of .333. Even if the batter hits .333 for a week, or even a month, it does not mean he will finish the long season hitting .333. If a chess player wins one tournament, even a so-called “Super tournament,” it does not mean he will become World Champion. Sofia Polgar had one super outstanding result with what I seem to recall a performance rating that was higher than any her sister Judit obtained in any one tournament. Yet Sophia did not attain the status that did Judit, because that one tournament is considered to be a limited sample size.
The authors studied the same studies as did Fernand Gobet and Guillermo Campitelli. The discredited Ferguson study is included among those studied, as is the Margolis study mentioned by GM Yasser Seirawan. While considering this post and thinking about IM John Donaldson, GM Yasser Seirawan, and to an extent, NM Jennifer Shahade, I kept thinking about something Upton Sinclair wrote – “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”
My thoughts also keep going back to something else written by Gobet and Campitelli: “In spite of these disagreements about the nature of transfer, some results are clear. In particular, recent research into expertise has clearly indicated that, the higher the level of expertise in a domain, the more limited the transfer will be (Ericsson & Charness, 1994). Moreover, reaching a high level of skill in domains such as chess, music or mathematics requires large amounts of practice to acquire the domain specific knowledge which determines expert performance. Inevitably, the time spent in developing such skills will impair the acquisition of other skills.”
Mr. Jerauld ends his article with, “The literature review is a huge first step for the CCSCSL, acknowledgment of the research that exists and laying the groundwork for future research that may be implemented through the club’s expansive scholastic initiative. The Basis Policy Research chess literature review has kicked off a new Research Portal, meant to serve as a repository to all global research — and to keep St. Louis on yet another forefront of chess.”
If this is truly “laying the groundwork for future research that may be implemented through the club’s expansive scholastic initiative,” I would suggest not only a rigorous, controlled, scientific, type study which would include a control group, but also a study of the same size to study the game of Go, as a counter balance to the chess study. This would obviously double the amount of money needed to fund these studies, but we are talking about a man who has a BILLION DOLLARS, so money is no object if Rex Sinquefield actually wants a fair and objective study, not one in which the scientists aim to please the man with the deep pockets.
Armchair Warrior 