You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

wedrifid comments on The Kolmogorov complexity of a superintelligence - Less Wrong Discussion

2 Post author: Thomas 26 June 2011 12:11PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (30)

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 26 June 2011 09:27:53PM -2 points [-]

It seems like CEV style friendliness, for example, could at least be reduced to something on the order of complexity of a single "averaged" human brain.

That doesn't sound like it passes the adequately-friendly-to-wedrifid test. I'd thermite such an AI rather than press the on button.

Comment author: Zetetic 26 June 2011 09:37:05PM -1 points [-]

That seems difficult to determine without unpacking what I mean by "averaged"; how did you come to that conclusion? (I'm wondering if you have a clearer concept of what it would mean to "average" over brains, my intuition is nothing more than a sort of vague feeling that makes me think that I should study certain topics and ask certain questions) I don't even know enough to accurately convey my intuition behind my use of "averaged", I was just hoping that it might elicit a response from someone that would give me some useful information that would, in turn, help me in my task of understanding CEV better.

Comment author: wedrifid 26 June 2011 09:45:50PM 1 point [-]

Unfortunately nobody has a good answer to your question. At least none that would satisfy me. The aggregation problem is the hardest part of the problem and hasn't been answered adequately yet. Without such an answer CEV remains basically undefined. (This is something that troubles me.)

Comment author: Zetetic 26 June 2011 10:39:16PM 0 points [-]

I see. Thanks; that is enlightening. I had gotten the impression that this was the case, but wasn't sure.