Aurini comments on The Strangest Thing An AI Could Tell You - Less Wrong

81 Post author: Eliezer_Yudkowsky 15 July 2009 02:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (574)

You are viewing a single comment's thread. Show more comments above.

Comment author: jimrandomh 16 July 2009 04:48:37AM *  20 points [-]

There's an important difference between brain damage and brain mis-development that you're neglecting. The various parts of the brain learn what to expect from each other, and to trust each other, as it develops. Certain parts of the brain get to bypass critical thinking, but that's only because they were completely reliable while the critical thinking parts of the brain were growing. The issue is not that part of the brain is outputting garbage, but rather, that it suddenly starts outputting garbage after a lifetime of being trustworthy. If part of the brain was unreliable or broken from birth, then its wiring would be forced to go through more sanity checks.

Comment author: Aurini 16 July 2009 04:55:09PM 5 points [-]

Oooooh! You're no fun anymore!

In all seriousness though, I agree with you to an extent. Suggestions such as 'all humans have tails' or 'some people who you think are dead are not, you just can't see them' - while surprising and creepy - would be extremely unlikely. I can see direct and obvious disadvantages to a person or species lacking such faculties. In fact, the disadvantages to those two would be so drastic that it would most likely lead to extinction.

And yet... I could still imagine us being blind to certain things. The first sort of blindness would be due to Darwinian irrelevance: for instance, many flowers have beautiful patterns visible in the UV spectrum, but there's no reason for us to see them. That might seem mundane nowadays, but five hundred years ago it would have freaked people out (maybe). I wouldn't be surprised that there are cognitive capabilities we've never suspected to exist.

The second sort of blindness is where it gets weird. True, our brains only allow trustworthy algorythms to bypass the logic circuits... or do they? The brain is not optimal. While I doubt we have invisible tails, that doesn't mean that there isn't some other phenomenon that we're simply incapable of noticing even when it's staring us right in the face.

Comment author: thomblake 17 July 2009 05:16:39PM *  1 point [-]

for instance, many flowers have beautiful patterns visible in the UV spectrum

Just in case anyone is curious about this:

link (via twitter: @izs)