Chess.com has been under the gun recently for the way the people who make decisions at the website have handled the long running and ongoing problem with cheating. Two day ago, Nov 8, 2022, Chess.com published the results of what they call their Fair Play Survey Results (https://www.chess.com/blog/CHESScom/fair-play-survey-results?page=4).
This is found at Chess.com:
“Ensuring fair play and protecting the integrity of the game is a priority for Chess.com. We believe that our members should have a voice in how we handle cheating in chess, and in an effort to understand the community’s sentiments better, we shared a fair play survey with three groups: members, titled players, and top players (the top 100 players overall as well as the top 20 women players by FIDE rating). We received 11,383 member responses, 166 titled-player responses, and 61 responses from top players.
Perhaps the most immediate insight is that on many important questions, the community is very divided. Cheating in chess is a complex problem, and there are no easy answers. We have tried to identify some useful insights from the responses and have shared them below. The full survey results are included here, and we welcome further comments and insights.”
The problem was illustrated immediately with the first few comments:
Duckfest
Around 11-12k respondents? Considering the player base on chess.com one could argue cheating is not as big a concern as the heated discussions on the forum would have you believe.
The next commentator hit the nail on the head:
Arullu
Well bearing in mind how many people are on this site the sample size is not indicative of much. It is just far too small a percentage of overall membership to draw any valid conclusions from. It is still useful but you should not draw any definitive conclusions from it. My two cents worth…
Duckfest wrote:
Maybe I could have phrased it better. I’m not questioning the validity of the sample size. The data for sure will be useful.
What I meant is that if members were concerned about cheating more people would have filled in the survey. The data shows most members don’t care enough to complete a 5 minute survey. That is assuming all members have been invited to respond. (https://www.chess.com/blog/CHESScom/fair-play-survey-results?page=4)
Because of having spent far too much time pouring over the Baseball numbers with Sabermetrics this writer knows far more than the average human when it comes to sample size. After perusing the article this writer knew immediately it was so flawed as to be worthless. Although I am not the most “mathy” kinda guy, a lifetime of analyzing Baseball statistics, with more hours spent at “The reason for the internet,” Baseball Reference, or more commonly, “B-Ref”, (https://www.baseball-reference.com/) than you would believe or even imagine, I do know about sample size. For example, a rookie begins the season with a MLB team and during the first week of the season rips the cover offa the ball, and the announcers have all but given him a berth in the Hall of Fame. Then he goes zero for the second week because the MLB pitchers learn he cannot hit the curve ball, so they give him a steady diet of curve balls he cannot hit, until he is relegated to the minor leagues. Here is an article concerning sample size for your amusement (https://www.baseballprospectus.com/news/article/17659/baseball-therapy-its-a-small-sample-size-after-all/).
In an email exchange with one former Baseball player who also knows and understands sample size, who also happens to play Chess, both over the board and at Chess.com, this was received:
“I was one of the 166 titled players who responded to this survey. That’s a small sample size, no matter how you look at it.”
One of the many problems with the Chess.com article is that “titled player” is not defined. For example, I was under the impression that a “titled player” was a player who had received a “title” from FIDE. Years ago a “titled” player was someone with at least an International Master title, but that changed when FIDE, in its wisdom, foisted the “FM” (FIDE Master) title on the Chess world. With the rise of female participation in Chess tournaments there are now FIDE titles beginning with a “W”, as in WGM, which is a “Woman Grandmaster.” This is not to be confused with being a “Grandmaster,” because a “WGM” is less than a real “Grandmaster.” Hence the “W”. Confused? It is really far more complicated than that to the extent it has become comical. An example would be the recent US Chess Championships, where there were a plethora of different titles showing after the names of the female participants. One female student wanted to know if the “f” after the name of some of the female players at The Week In Chess was for “female.” I had to be honest and inform the young one that I had no idea, but did mention it could be for “FIDE Master.” She said, “Then it should be a “FM”, right?” What could I say? Teachers do not have all the answers. Is an “Expert” a titled player? The “Expert” class is the first class with a crooked number and begins at 2000 rating points, but it is not considered a “title,” yet it is considered a major step on the road to Master, and even with rating inflation, is still a goal for many players.
Foisting an immediately discredited explanation upon the Chess world by Chess.com surely is an indication of something amiss at the website. The people at Chess.com should know about limited sample size and the worthlessness of publishing such a flawed study. Nevertheless I will cut them some slack because it could be they are completely ignorant of sample size and meant well when depositing such a load of crap. The article shows Chess.com is in damage control mode and will continue to do whatever it takes to cover their collective asses. Then again, when there are so many firing salvos at you maybe it is best to “duck and cover.”
Armchair Warrior 