Sure! I was including "setting up a system that bans other people" in my definition here. I am not that familiar with how DSL works, but given that it bans people, and it was set up by Said, felt confident that thereby somehow Said chose to build a system that does ban people.

Though if Said opposes DSL banning people (and he thinks the moderators are making a mistake when doing so) then I would want to be corrected!

Has some things going for it, but probably too vulgar for most occasions, and also IMO implies too much malintent. I like "diagonalization" because it's kind of a thing you can imagine doing. 

I think my current favorite choice is "Leveling" and "starting a leveling war" from poker. Sentences like "man, I feel like this is too much trying to level them" and "this feels like it's starting a leveling war" are decent-ish pointers.

I don't know about Scott. Him being personally active on the site was long before my tenure as admin, and I am not even fully sure how moderation or deletion at the time worked.

I don't think Said ever banned anyone, though he also wrote only a very small number of top-level posts, so there wasn't much opportunity. My guess is he wouldn't have even if he had been writing a lot of top-level posts.

I don't think the central-case valuable PhDs can be bought or sold so I'm not sure what you mean by market value here. If you can clarify, I'll have a better idea whether it's something I'd bet against you on.

I was thinking of the salary premium that having a PhD provides (i.e. how much more people with PhDs make compared to people without PhDs), which of course is measuring a mixture of real signaling value, and simply just measuring correlations in aptitude, but I feel like it would serve as a good enough proxy here at least directionally.

I would bet a fair amount at even odds that Stanford academics won't decline >1 sigma in collective publication impact score like h-index, Stanford funding won't decrease >1 sigma vs Ivy League + MIT + Chicago, Stanford new-PhD aggregate income won't decline >1 sigma vs overall aggregate PhD income, and overall aggregate US PhD income won't decline >1 sigma. I think 1 sigma is a reasonable threshold for signal vs noise.

What's the sigma here? Like, what population are we measuring the variance over? Top 20 universities? All universities? I certainly agree that Stanford won't lose one sigma of status/credibility/etc. as measured in all universities, that would require dropping Stanford completely from the list of top universities. I think losing 1 sigma of standing among top 20 universities, i.e. Stanford moving from something like "top 3" to "top 8" seems plausible to me, though my guess is a bit too intense. 

To be clear, my offered bet was more about you saying that academia at large is "too big to fail". I do think Stanford will experience costs from this, but at that scale I do think noise will drown out almost any signal. 

TBTF institutions usually don't collapse outside strong outside conquest or civilizational collapse, or Maoist Cultural Revolution levels of violence directed at such change, since they specialize in creating loyalty to the institution. So academia losing value would look more like the Mandarin exam losing value, than like Dell Computer losing value.

Hmm, I don't currently believe this, but it's plausible enough that I would want to engage with it in more detail. Do you have arguments for this? I currently expect more of a gradual devaluing of the importance of academic status in society, together with more competition about the relevant signifiers of status creating more noise, resulting in a relationship to academia somewhat more similar (though definitely not all the way there) as pre-WW2 society had to academia (which to my understanding was a much less central role in government and societal decision-making).

I understand that you don't! But almost everyone else who I do think has those attributes does not have those criteria. Like, Scott Alexander routinely bans people from ACX, even Said bans people from datasecretslox. I am also confident that the only reason why you would not ban people here on LW, is because the moderators are toiling for like 2 hours a day to filter out the people obviously ill-suited for LessWrong.

They, being authors themselves, see the author's pain firsthand, but the commenter's feelings are merely an abstract report at most.

I do think there is a bunch of truth to this, but I am active on many other forums, and have e.g. been issued moderation warnings on the EA Forum, so I do experience moderation in other contexts (and of course get blocked on Twitter from time to time). Also, I... think authors are not that much less likely to ban moderators from their posts than other users. Of the maybe 30 users who have ever been banned from other user posts, one of them is a moderator: 

I am sure that if Eliezer was more active on the site, my guess is people would be a bunch more likely to ban him from their posts than they would other people for the same behavior. In general, in my experience, tallest-poppy dynamics are stronger in the rationality community than leadership-deference dynamics.

While I think building safety-adjacent RL envs is worse than most kinds of technical safety work for people who are very high context in AGI safety, I think it's net positive.

I think it's a pretty high-variance activity! It's not that I can't imagine any kind of RL environment that might make things better, but most of them will just be used to make AIs "more helpful" and serve as generic training data to ascend the capabilities frontier. 

Like, yes, there are some more interesting monitor-shaped RL environments, and I would actually be interested in digging into the details of how good or bad some of them would be, but the thing I am expecting here are more like "oh, we made a Wikipedia navigation environment, which reduces hallucinations in AI, which is totally helpful for safety I promise", when really, I think that is just a straightforward capabilities push.

I would bet that the average research impact of SPAR participants is significantly lower than that of MATS

I mean, sure? I am not saying your selection is worse than useless and it would be better for you to literally accept all of them, that would clearly also be bad for MATS.

I think you are prioritizing a different skillset than most mentors that our mentor selection committee rates highly. Interestingly, most of the technical mentors that you rate highly seem to primarily care about object-level research ability and think that strategy/research taste can be learned on the job!

I mean, there are obvious coordination problems here. In as much as someone is modeling MATS as a hiring pipeline, and not necessarily the one most likely to produce executive-level talent, you will have huge amounts of pressure to produce line-worker talent. This doesn't mean the ecosystem doesn't need executive-level talent (indeed, this post is partially about how we need more), but of course large scaling organizations create more pressure for line-working talent. 

Two other issues with this paragraph: 

• Yes, I don't think strategic judgement generally commutes. Most MATS mentors who I think are doing good research don't necessarily themselves know what's most important for the field.
• I agree with the purported opinion that strategy/research taste often be learned on the job. But I do feel very doomy about recruiting people who don't seem to care deeply about x-risk. I would be kind of surprised if the mentors I am most excited about don't have the same opinion, but it would be an interesting update if so!

Note that I think the pendulum might start to swing back towards mentors valuing high-level AI safety strategy knowledge as the Iterator archetype is increasingly replaced/supplemented by AI. The Amplifier archetype seems increasingly in-demand as orgs scale, and we might see a surge in Connectors as AI agents improve to the point that their theoretical ideas are more testable. Also note that we might have different opinions on the optimal ratio of "visionaries" vs. "experimenters" in an emerging research field.

I don't particularly think these "archetypes" are real or track much of the important dimensions, so I am not really sure what you are saying here.

Ok, having more time today and thinking more about it, I have updated the description of the proof in the infobox! Curious whether it seems better/more accurate to you now.

Only if one of them is diagonalizing the other (acting contrary to what the other would've predicted about its actions). If this isn't happening, maybe there is no problem.

Ah, yes, of course. I'll update the description.

Agree on all the rest, I think. I didn't intent to establish a strict ordering of agents (though my usage of bigger and smaller in the strict case of adversarial diagonalizing agents sure suggested it). In those case I find it a useful visualization to think about bigger and smaller. 

It's from Cantor's diagonal argument.

I agree that "diagonalization" is a fine term for the specific narrow thing where you choose actions contrary to what the other agent would have predicted you would do, in the way described here, but I am more talking about the broader phenomenon of "simulating other agents adversarially in order to circumvent their predictions". "Leveling" is apparently a term from poker that means something kind of similar and more general: 

Levelling

• Leveling in poker is the process of anticipating what your opponent thinks you are thinking, often leading to deeper layers of strategic decision-making.
• Its purpose is to outthink opponents by operating on a higher mental “level” than they are.

Like, I would like a term for this kind of thing that is less opinionated about the exact setup, and technical limitations. Like, I am pretty sure there is a more general phenomenon here.