I asked an LLM to do the math explicitly, and I think it shows that it's pretty infeasible - you need a large portion of total global power output, and even then you need to know who's receiving the message, you can't do a broad transmission.
 
I also think this plan preserves almost nothing I care about. At the same time, at least it's realistic about our current trajectory, so I think planning along these lines and making the case for doing it clearly and publicly is on net good, even if I'm skeptical of the specific details you suggested, and don't think it's particularly great even if we succeed.

I mostly agree, but the word gentile is a medieval translation of "goyim," so it's a bit weird to differentiate between them. (And the idea that non-jews are ritually impure is both confused, and an frequent antisemitic trope. In fact, idol worshippers were deemed impure, based on verses in the bible, specifically Leviticus 18:24, and there were much later rabbinic decrees to discourage intermingling with even non-idol worshippers.) 

Also, both Judaism and LDS (with the latter obviously more proselytizing) have a route for such excluded individuals to join, so calling this "a state of being which outsiders cannot attain" is also a bit strange to claim.

Your dismissive view of "conservatism" as a general movement is noted, and not even unreasonable - but it seems basically irrelevant to what we were discussing in the post, both in terms of what we called conservatism, and the way you tied it to 'Hostile to AGI." And the latter seems deeply confused, or at least needs much more background explanation.

I'd be more interested in tools that detected downvotes that occur before people started reading, on the basis of the title - because I'd give even odds that more than half of downvotes on this post were within 1 minute of opening it, on the basis of the title or reacting the the first paragraph - not due to  the discussion of CEV.

I agree that Eliezer has made different points different places, and don't think that the Fun Theory series makes this clear, and CEV as described seems to not say it. (I can't try to resolve all the internal tensions between the multiple bookshelves woth of content he's produced, so I referred to "fun theory, as written.")

And I certainly don't think conflict as such is good! (I've written about the benefits of avoiding conflict at some length on my substack about cooperation.) My point here was subtly different, and more specific to CEV; I think that solutions for eliminating conflict which route around humans themselves solving the problems might be fundamentally destructive of our values.

I don't think this is true in the important sense; yes, we'll plausibly get material abundance, but we will still have just as much conflict because humans want scarcity, and they want conflict. So which resources are "important" will shift. (I should note that Eliezer made something like this point in a tweet, where he said "And yet somehow there is a Poverty Equilibrium which beat a 100-fold increase in productivity plus everything else that went right over the last thousand years" - but his version assumes that once all the necessities are available, poverty would be gone. I think that we view clearly impossible past luxuries, like internet connectivity and access to laundry machines as minimal requirements, showing that the hedonic treadmill is stronger than wealth generation!)

Thank you for noticing the raft of reflexive downvotes; it's disappointing how much even Lesswrong seems to react reflexively; even the comments seem not to have read the piece, or at least engaged with the arguments.

On your response - I agree that CEV as a process could arrive at the outcomes you're describing, where ineliminable conflict gets it to throw an error - but think that CEV as approximated and as people assume will work is, as you note, making a prediction that disagreements will dissolve. Not only that, but it asserts that this will have an outcome that preserves what we value. If the tenets of agonism are correct, however, any solution geared towards "efficiently resolving conflict" is destructive of human values - because as we said, "conflict is central to the way society works, not something to overcome." Still, I agree that Eliezer got parts of this right (a decade before almost anyone else even noticed the problem,) and agree that keeping things as multiplayer games with complex novelty, where conflict still matters is critical. The further point, which I think Eliezer's fun theory, as written, kind of elides, is that we also need limits and pain for the conflict to matter. That is, again, it seems possible that part of what makes things meaningful is that we need to ourselves engage in the conflict, instead of having it "solved" via extrapolation of our values.
 

As a separate point, I argued in a different post, we lack the conceptual understanding needed to deal with the question of whether there is some extrapolated version of most agents that is anywhere "close" to their values which is coherent.  But at the very least, "the odds that an arbitrary complex system is pursuing some coherent outcome" approaches zero, and that at least slightly implies almost all agents might not be "close" to a rational agent in the important senses we care about for CEV. 

"This is what's happening and we're not going to change it" isn't helpful - both because it's just saying we're all going to die, and because it fails to specify what we'd like to have happen instead. We're not proposing a specific course for us to influence AI developers, we're first trying to figure out what future we'd want. 

The viability of what approach, exactly? You again seem to be reading something different than what was written.

You said "There is no point in this post where the authors present a sliver of evidence for why it's possible to maintain the 'barriers' and norms that exist in current societies, when the fundamental phase change of the Singularity happens."

Did we make an argument that it was possible, somewhere, which I didn't notice writing? Or can I present a conclusion to the piece that might be useful:

"...the question we should be asking now is where [this] view leads, and how it could be achieved.

That is going to include working towards understanding what it means to align AI after embracing this conservative view, and seeing status and power as a feature, not a bug. But we don’t claim to have 'the' answer to the question, just thoughts in that direction - so we’d very much appreciate contributions, criticisms, and suggestions on what we should be thinking about, or what you think we are getting wrong."