I played chess for a while earlier this year.  I never played or watched any sports when I was younger, so sport forums were weird to me.  One thing that kept coming up led to a long meandering stream of thought I tried in vain to edit together into a coherent narrative.  It’s probably easy to guess what the most frequent and hotly debated question would be: who’s the greatest to ever play the sport?

My standard response to this kind of question is to say it’s a question of semantics, or specific definitions.  But this is a place where we reward players for greatness anyway, what with World Champions and player rankings, so there's something being incentivized.

I don't think many try to think about what they mean when they talk about the quality of a player - probably because it's more fun to not.  But I'm under no such obligation.  For most people, what it means is a varying combination of three factors - natural talent, dedicated effort, and actual skill level.

Depending on your sensibilities, it may seem strange at first to claim that people try to use these to defend their favourite choice.  Going back to chess, there are many names that come up in these arguments, but the three most prominent serve as a good microcosm: Bobby Fischer, eleventh World Chess Champion, who single-handedly ended Soviet chess supremacy and had an unparalleled dominance over his opponents at the time; Garry Kasparov, thirteenth World Chess Champion, who was the highest rated player in the world for over 21 years and had an unparalleled dominance over his opponents in his time; and Magnus Carlsen, sixteenth and current World Chess Champion, the highest rated player in history, and who doesn't have unparalleled dominance over his opponents, but for defensible reasons.

To me at first glance, it seemed obvious that Magnus fits the title of best player the most - ELO ratings being a metric of relative playing level, and the quality of the average GM increasing over time.  Proponents of Fischer or Kasparov mainly sound the argument that Magnus reigns in an era with advanced chess AI that any player can use to train and prepare with.  That's true, professional chess players make heavy use of engines to create and test their strategies.  This is also partly why Magnus doesn't dominate this age like his predecessors - the improvement of engines and other freely available advantages reduce the extent to which the game hinges on natural talent.  Added, at this point in the game, I think we're reaching diminishing returns on how well a human can actually play chess.

But why does this matter more than any externalities about the player that gives them an advantage?  Magnus also makes more use of strong physical training regimens to better his stamina during games, but that isn't used as an argument against his relative skill to the other two (more so Fischer than Kasparov, the latter of which did employ physical training).  Maybe dissidents realize it's not as defensible an argument; someone could refute with that the others could have done the same in their time.  But could they have?  It isn't that Fischer didn't care enough to try a physical training regime - you don't reach the level of World Champion without inordinate levels of dedication to your craft - but more likely that cumulative game knowledge hadn't reached the point where stuff like this was obvious to the top players as something to even consider.

You could try and remove all externalities, to try and compare the raw, untempered talent of a player against all others.  It's still a difficult question then, but you'd have a stronger argument for Bobby (or even older names like Capablanca).  Or you could remove all the factors a player seemingly doesn't have control over, and focus solely on their dedication to getting stronger.  It's a more difficult question to answer then, but not more difficult to think about - it's just that we don't have a good and reliable way of measuring which top players spend the most time or care the hardest about beating their opponents.  The argument would lie dead there.

Or maybe it's about simply entertaining the viewers.  In the current state of competitive chess, formats often incentivize the best players to play for safe draws to preserve rankings.  This usually leads to a lot of criticism and calls from even the players themselves to change the format, on grounds that it's not entertaining enough.  But then why even have human players?  AlphaZero and Stockfish dwarf the best players in skill and unorthodox moves.  Maybe what people really want is the thrill of watching dominance, but then that raises the further question, why chess?

It's possible none of this matters, because it's only relevant to a question no one really wants answered.  But there are other questions it's relevant to.  For example, Magnus famously decried the current system for choosing a new World Champion, a round-robin between the candidates to decide who gets to duke it out one-on-one against the world champion.  In his opinion, it unfairly privileges the current champion - most other sports don't send the previous winner all the way to the final by default.  As alternatives, he suggests anything from annual knock-out events to moving away from the classical time format.

Is a round-robin a fair evaluation at all?  The Candidates tournament is held over the course of 18 days in 2018.  Sure, you'd have to be one of the best players in the world to win regardless, but in many years the field is pretty even, and it comes down to which player is in best condition that month, who has the strongest stamina-conditional skill, and who has the most coverage in their play.  

You could make a good argument as to why these are necessary criteria for the world champion, but I don't think you could as easily claim they're the most important ones.

The criteria for eligibility to this Candidates tournament is arbitrarily handpicked by FIDE, as the winners of specific high-level tournaments, carryovers, even wild cards by the top sponsor, a practice many top players dislike, including the 2020-21 wild card.  I've always been fascinated by the way so many systems today make use of very qualitative guesses by officials in an attempt to formalize (school syllabi, for example, or informally-but-used-formally-often, the h-index metric in academia), often with just enough degrees of correlation to maintain deniability.  It feels like we should have better ways to do this.

Some super-GMs think that the title of World Champion is being taken too seriously; that owing to the format, some day someone clearly not the strongest player will win, and all the mythos around it will disappear.  

The contender for the 2021 World Championship, Ian Nepomniachtchi, was described by Magnus as someone who has very high highs, and very low lows, so does that make him a better player than someone whose best is slightly below Ian's, but is consistent at that level?  Ian even has a lifetime positive classical score against the World Champion (though some of those games were played when they were younger).

Magnus' dominance can be attributed to a variety of factors in his actual style, from playing multiple openings to reduce the effect of computer preparation (which is arguably more about lowering his opponents' skill than raising his own, from a purely chess-theoretic standpoint), to his ability to grind out endgames while his opponents wilt in stamina (again an idea that wins, but lowers the level of chess play), to just playing the kind of moves that nettles his opponent and makes them blunder.  

One of his rivals Anish Giri, on the other hand, is a player with a very different approach - one that some say makes him harder to defeat than Magnus himself.  Yet he lacks in the ability to convert positions to victory, giving him an infamous reputation for drawing games.

Of all these varying styles, I'd find it weird if we hand-picked one of them to say, "This is the best."  But we don't think about it, we just let the natural incentives at play relieve us of all responsibility.  I think people unconsciously give too much leeway to the null action, to not doing anything.  It’s when you start doing something that criticism comes in full force, because you’re accepting the mantle of responsibility.  (Anecdotal example: when I first heard of Effective Altruism, I was more disparaging toward people who gave to random charities than toward people who didn’t donate to charity.  Not that being disparaging at all is good, but that was a long time ago, sue me).

Given the balancing act organizations have to maintain with placating players, maintaining optics, and still having a decent following at the end of it all, I don't think I could do better than FIDE.  Maybe I could, but the counterweight of having to actually deal with the blame inherent to not taking the null action is heavy.

But what if you're not FIDE or FIFA or the ICC?  Sports is too entrenched in our society for its problems to remain internal.  

There is, for example, the controversial issue of trans people in sports, specifically the supposed advantage transgender women have in women's sports, because of their higher testosterone levels and muscle-to-fat ratio.  The object issue doesn't pertain to this though, so I'm blocking out that discussion and focusing on whether the question matters at all. 

People are born with different genetic makeup presaging different levels of testosterone or strength or flexibility or stamina or relative effort to maintain equal performance.  We don't call for segregating sports based on how much a person lucked out genetically, probably because no one would watch FIFA: Category 2, or because many ignore or don't realize the extent to which genetics plays a factor.

So if what you want to test is natural talent, you'd try to regulate sportspeople's training regimen and nutrition to level the playing field in that regard, or something better.  If you cared about dedication, you'd segregate sports on natural talent (which would need an increasing number of segregations as you reach diminishing returns on increasingly effective training, since genetics isn't comfortably discrete).  If you didn't care about any of this and only wanted to see good quality of play, the best AI in your sport is over 700 ELO ahead of the best current human, and you’re left to wonder whether you need the humans at all (this doesn’t apply well to physical sports, but I think this suggests that the enjoyment of a sport is largely independent of its nature).

It may be that there's no good answer, or that competitive sports is too riddled with these kinds of epistemic problems for it to last in the long run.  Maybe like I mentioned earlier, the core incentive is simply the joy of watching your favourite player dominate.  That’s my best guess, at least.  In that case though, sports would be far better off if we actually optimized for that while setting formats, or even creating new games.