All of simplegeometry's Comments + Replies

Oh, my bad, yeah. When I was writing the comment, I flipped the direction of advantage for longer time controls (longer time controls are actually better for humans in odds matches of course), but this way I agree it's unclear a priori whether 200 elo drop would be enough to account for longer time controls.

Thanks for this update! I find that an odd prediction by the IM because Awonder is around 2670 FIDE and Joel is around 2470 FIDE, 200 elo is huge.

1Lorenzo
I think it's because 10+5 is very different from 60+30

Hissha from the Lc0 server reports 19 wins, 3 draws, and 2 losses against Lenderman (currently ~2500 FIDE) at 15+10 from a knight odds match 2 months ago -- with the caveat that Lenderman started playing too fast after 10 games. I haven't run the numbers but suspect this would be enough to go even against a 2750, if not Magnus?

I was surprised too. I think it's an exciting development :)

3[anonymous]
Hmm, that sounds about right based on the usual human-vs-human transfer from Elo difference to performance... but I am still not sure if that holds up when you have odds games, which feel qualitatively different to me than regular games. Based on my current chess intuition, I would expect the ability to win odds games to scale better than ELO near the top level, but I could be wrong about this.

At rapid time controls, it seems like we could maybe go even against Magnus with knight odds? If not Magnus, perhaps other high-rated GMs.

There was a match with the most recently updated LeelaKnightOdds and GM Alex Lenderman but I don't recall the score exactly. EDIT: which was 19-3-2 win draw loss.

1[anonymous]
I am very skeptical of this on priors, for the record. I think this statement could be true for superblitz time controls and whatnot, but I would be shocked if knight odds would be enough to beat Magnus in a 10+0 or 15+0 game. That being said, I have no inside knowledge, and I would update a lot of my beliefs significantly if your statement as currently written actually ends up being true.

This is something lc and gwern discussed in the comments here, but now we have clear evidence this is only true for Nash solvers (all typical engines like SF, Lc0, etc.). LeelaQueenOdds, which trained exploitatively against a model of top human players (FM+), is around 2k to 2.9k lichess elo depending on the time controls, so it completely trounces 1.6k elo players (especially 1.2k elo players as another commenter has suggested the author actually is). See: https://marcogio9.github.io/LeelaQueenOdds-Leaderboard/

Nash solvers are far too conservative and exp... (read more)

Reply1111
9Lorenzo
A grandmaster just lost a classical game (60''+30'') against Leela Knight Odds https://lichess.org/broadcast/leela-knight-odds-vs-gm-joel-benjamin/game-5/MbKHEbdb/7Tnz8uBj    3 days ago an international master gave Leela "very slim chances" of winning a game, based on the results of a match played by a previous version of the engine
5Thomas Kwa
Maybe we'll see the Go version of Leela give nine stones to pros soon? Or 20 stones to normal players?

I found it interesting to play against LeelaQueenOdds. My experiences:

  • I got absolutely crushed on 1+1 time controls (took me 50+ games to win one), but I'm competitive at 3+2 if I play seriously.
  • The model is really good at exploiting human blind spots and playing aggressively. I could feel it striking in my weak spots, but not being able to do much about it. (I now better acknowledge the existence of adversarial attacks for humans on a gut level.)
  • I found it really addictive to play against it: You know the trick that casinos use, where they make you feel l
... (read more)
4cata
I was playing this bot lately myself and one thing it made me wonder is, how much better would it be at beating me if it was trained against a model of me in particular, rather than how it actually was trained? I feel I have no idea.