You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

TheOtherDave comments on Non-orthogonality implies uncontrollable superintelligence - Less Wrong Discussion

14 Post author: Stuart_Armstrong 30 April 2012 01:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread.

Comment author: TheOtherDave 30 April 2012 02:59:17PM 7 points [-]

Well, yes.

Although I would also consider this conclusion to follow from the broader claim that if A is a superintelligence with respect to B, B cannot control A, regardless of whether there's a true code of morality (a question I'm not weighing in on here).

Well, unless you want to say that if A happens to want what B wants, or want what B would want if B were a superintelligence, or otherwise wants something that B endorses, or that B ought to endorse, or something like that (for example, if A is Friendly with respect to B), then B controls A, or acausally controls A, or something like that.

At which point I suspect we do better to taboo "control", because we're using it in a very strange way.