Is this relevant for my variant of "indirect normativity" (i.e. allowing human-designed WBEs, no conceptually confusing tricks in constructing the goal definition)?
(I'm generally skeptical about anything that depends on the concept of "blackmail" distinct from general bargaining, as it seems to be hard to formulate the distinction. It seems to be mostly the affect of being bargained against really unfairly, which might happen with a smart opponent if you are bad at bargaining and are vulnerable to accepting unfair deals, so the solution appears to be to figure out how to bargain well, and avoid bargaining against strong opponents until you do figure that out.)
Yeah, I think it's relevant for your variant as well.
I think I've found a new argument, which I'll call X, against Paul Christiano's "indirect normativity" approach to FAI goals. I just discussed X with Paul, who agreed that it's serious.
This post won't describe X in detail because it's based on basilisks, which are a forbidden topic on LW, and I respect Eliezer's requests despite sometimes disagreeing with them. If you understand Paul's idea and understand basilisks, figuring out X should take you about five minutes (there's only one obvious way to combine the two ideas), so you might as well do it now. If you decide to discuss X here, please try to follow the spirit of LW policy.
In conclusion, I'd like to ask Eliezer to rethink his position on secrecy. If more LWers understood basilisks, somebody might have come up with X earlier.