All of polytope's Comments + Replies

Circling back to this with a thing I was thinking about - suppose one wanted to figure out just one additional degree of freedom to the Elo rating a player had (at a given point in time, if you also allow evolution over time) that would add as much improvement as possible. Almost certainly you need more dimensions than that to properly fit real idiosyncratic nonlinearities/nontransitivities (i.e. if you had a playing population with specific pairs of players that were especially strong/weak only against specific other players, or cycles of players where A ... (read more)

polytope109

I might be misunderstanding, but it looks to me like your proposed extension is essentially just the Elo model with some degrees of freedom that don't yet appear to matter? 

The dot product has the property that <theta_A-theta_B,w> = <theta_A,w> - <theta_B,w>, so the only thing that matters is the <theta_P,w> for each player P, which is just a single scalar.  So we are on a one-dimensional scale again where predictions are based on taking a sigmoid of the difference between a single scalar associated with each player.

As far... (read more)

Answer by polytope32

I assume you're familiar with the case of the parallel postulate in classical geometry as being independent of other axioms? Where that independence corresponds with the existence of spherical/hyperbolic geometries (i.e. actual models in which the axiom is false) versus normal flat Euclidean geometry (i.e. actual models in which it is true).

To me, this is a clear example of there being no such thing as an "objective" truth about the the validity of the parallel postulate - you are entirely free to assume either it or incompatible alternatives. You end up w... (read more)

3lbThingrb
This is true, but there's an important caveat: Mathematicians accepted Euclidean geometry long before they accepted non-Euclidean geometry, because they took it to be intuitively evident that a model of Euclid's axioms existed, whereas the existence of models of non-Euclidean geometry was AFAIK regarded as non-obvious until such models were constructed within a metatheory assuming Euclidean space. From the perspective of modern foundations, it's not so important to pick one kind of geometry as fundamental and use it to construct models of other geometries, because we now know how to construct models of all the classical geometries within more fundamental foundational theories such as arithmetic or set theory. But OP was asking about incompatible variants of the axioms of set theory. We don't have a more fundamental theory than set theory in which to construct models of different set theories, so we instead assume a model of set theory and then construct models of other set theories within it. For example, one can replace the axiom of foundation of ZFC with axioms of anti-foundation postulating the existence of all sorts of circular or infinitely regressing chains of membership relations between sets. One can construct models of non-well-founded set theories within well-founded set theories and vice versa, but I don't know of anyone who claims that therefore both kinds of set theory are equally valid as foundations. The existence of models of well-founded set theories is natural to assume as a foundation, whereas the existence of models satisfying strong anti-foundation axioms is not intuitively obvious and is therefore treated as a theorem rather than an axiom, the same way non-Euclidean geometry was historically. Yes, there are ways of interpreting ZFC in a theory of natural numbers or other finite objects. What there is not, however, is any known system of intuitively obvious axioms about natural numbers or other finite objects, which makes no appeal to intuiti
polytope142

This thread analyzes what is going on under the hood with the chess transformer. It is a stronger player than the Stockfish version it was distilling, at the cost of more compute but only by a fixed multiplier, it remains O(1).


I found this claim suspect because this basically is not a thing that happens in board games. In complex strategy board games like Chess, practical amounts of search on top of a good prior policy and/or eval function (which Stockfish has), almost always outperforms any pure forward pass policy model that doesn't do explicit search, e... (read more)

polytope10

Do you think a vision transformer trained on 2-dimensional images of the board state would also come up with a bag of heuristics or would it naturally learn a translation invariant algorithm taking advantage of the uniform way the architecture could process the board? (Let's say that there are 64 1 pixel by 1 pixel patches, perfectly aligned with the 64 board locations of an 8x8 pixel image, to make it maximally "easy" for both the model and for interpretability work.)

And would it differ based on whether one used an explicit 2D positional embedding, or a l... (read more)

1Adam Karvonen
I would guess that it would learn an exact algorithm rather than heuristics. The challenging part for OthelloGPT is that the naive algorithm to calculate board state from input tokens requires up to 60 sequential steps, and it only has 8 layers to calculate the board state and convert this to a probability distribution over legal moves.
polytope214

(KataGo dev here, I also provided a bit of feedback with the authors on an earlier draft.)

@gwern - The "atari" attack is still a cyclic group attack, and the ViT attack is also still a cyclic group attack. I suspect it's not so meaningful to put much weight on the particular variant that one specific adversary happens to converge to. 

This is because the space of "kinds" of different cyclic group fighting situations is combinatorically large and it's sort of arbitrary what local minimum the adversary ends it because it doesn't have much pressure to fin... (read more)

Apparently not a writeup (yet?), but there appears to be a twitter post here from LC0 with an comparison plot of accuracy on tactics puzzles:  https://x.com/LeelaChessZero/status/1757502430495859103?s=20

Yes, rather than resolving the surprise of "the exact sequence HHTHTTHTTH" by declaring that it shouldn't be part of the set of events, I would prefer to resolve it via something like:

  • It should be part of the set of events I'm allowed to consider just like any other subset of all 10-flip sequences. 
  • We do observe events (or outcomes that if constructed as singleton events) all the time that would we would have predicted to be exceedingly improbable (while they may be improbable individually, a union of them may not be).
  • Observing some particular unlikel
... (read more)

Here's my intuition-driving example/derivation.

Fix a reference frame and suppose you are on a frictionless surface standing next to a heavy box equal to your own mass, and you and the box always start at rest relative to one another. In every example, you will push the box leftward, adding 1 m/s leftward velocity to the box, and adding 1 m/s rightward velocity to yourself. 

Let's suppose we didn't know what "kinetic energy" is, but let's suppose such a concept exists, and that whatever it is, an object of your mass has 0 units of it when at rest, and i... (read more)

> lack of sufficient evidence. 

Perhaps more specifically, evidence that is independent from the person that is to be trusted or not. Presumably when trusting someone else that something is true, often one does so due to believing that the other person is being honest and reliable enough such that that their word is sufficient evidence to then take some action. It's just that there isn't sufficient evidence without that person's word.

polytope*110

I am also curious why the zero-shot transfer is so close to 0% but not 0%. Why do those agents differ so much, and what do the exploits for them look like?

The exploits for the other agents are pretty much the same exploit, they aren't really different. From what I can tell as an experienced Go player watching the adversary and other human players use the exploit, the zero shot transfer is not so high because the adversarial policy overfits to memorize specific sequences that let you set up the cyclic pattern and learns to do so in a relatively non-robust w... (read more)

(I'm the main KataGo dev/researcher)

Just some notes about KataGo - the degree to which KataGo has been trained to play well vs weaker players is relatively minor. The only notable thing KataGo does is in some self-play games to give up to an 8x advantage in how many playouts one side has over the other side, where each side knows this. (Also KataGo does initialize some games with handicap stones to make them in-distribution and/or adjust komi to make the game fair). So the strong side learns to prefer positions that elicit higher chance of mistakes by the ... (read more)

polytopeΩ450

There's (a pair of) binary channels that indicate whether the acting player is receiving komi or paying it. (You can also think of this as a "player is black" versus "player is white" indicator, but interpreting it as komi indicators is equivalent and is the natural way you would extend Leela Zero to operate on different komi without having to make any changes to the architecture or input encoding).

In fact, you can set the channels to fractional values strictly between 0 and 1 to see what the model thinks of a board state given reduced komi or no-komi cond... (read more)

polytopeΩ10213

This is very cool, thanks for this post.

Some remarks:

Playing at an intersection with no liberty is forbidden, unless the play results in capture

This is true, but the capture can be of your own stones. That is, Leela Zero is trained under Tromp-Taylor rules where self-capture is legal. So there isn't any forbidding of moves due to just liberties. Single stone suicide is still illegal, but only by virtue of the fact that self-capture of a single stone would repeat the board position, but you can suicide multiple stones.

However, there is still of course a que... (read more)

I think there are two "simple" abstractions possible here, where the amount of data to distinguish which one is right is minuscule under natural play in Go, and therefore can easily be overwhelmed by small constant factors in the "ease" of converging to either one due to inductive bias.

  • Abstraction 1: the size of the set of all empty spaces adjacent to a group
  • Abstraction 2: the sum of the number of empty spaces next to a given stone on the group, plus, recursively, this same sum for each of the neighboring stones of the same player to the north, south, east
... (read more)

Keep in mind that the adversary was specifically trained against KataGo, whereas the performance against LeelaZero and ELF is basically zero-shot. It's likely the case that an adversary trained against LeelaZero and ELF would also win consistently.

I've run LeelaZero and ELF and MiniGo (yet another independent AlphaZero replication in Go) by hand in particular test positions to see what their policy and value predictions are, and they all very massively misevaluate cyclic group situations just like KataGo. Perhaps by pure happenstance different bots could "... (read more)

polytope*3013

In the paper, KataGo creator David Wu theorizes that KataGo learned a method for counting liberties that works on groups of stones with a tree structure, but fails when the stones form a cycle.  I feel like that can't be the whole story because KataGo can and does solve simpler life-or-death problems with interior liberties, but I don't have a more precise theory to put in its place.

KataGo author here: why do you feel this can't be the whole story? The very fact that it solves other life and death problems with interior liberties when no large cycle i... (read more)

Couldn't you have also made the exact same argument for the word "vaccination" some number of generations ago, for almost exactly the same reason? It too derives from root words about a practice intended for protecting specifically against smallpox. (Namely, infecting someone with cowpox).

https://www.etymonline.com/word/vaccination

When words are so overly specific so as to almost completely fall out of usefulness for their original meaning (as in the case of both vaccination and variolation, since smallpox is not in circulation any more), ... (read more)

0leggi
No it doesn't seem "pretty natural to see people re-purpose" variolation for something that would be labelled in standard and accepted medical terms as vaccination with a live virus. Find some people in the medical profession that think it's a good idea then I may reconsider my stance, otherwise I've made my point and don't intend to post any more comments on the subject.