That no superintelligence could be dumb enough to misinterpret what we mean is the particular belief in question

I didn't come up with that myself, I got it from MugaSofer: 'Well, since the whole genie thing is a metaphor for superintelligence, "this genie is trying to be Friendly but it's too dumb to model you well" doesn't really come up.'

Under reasonable definitions of "superintelligence" it does follow that a superintelligence must know what you mean, but if you pick some other definition and state so outright, I won't argue with it. (It is, however, still subject to "talk to the intelligence to figure out what it's going to do".)

Of course if you are a computer scientist they will say, "But Jiro is not an elite computer scientist!", and if you were an elite computer scientist they would say, "Elite computer scientists don't currently take the issue seriously enough to think about it properly...

I think you're making my case for me.

PS: If you want to reply please post a new reply to the root message since I can't afford the karma hits to respond to you.