= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on The Strangest Thing An AI Could Tell You - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (574)
I would believe the AI if it told me that human beings all had tails. (That's not even so far from classic anosgnosia - maybe primates just lost the tail-controlling cortex over the course of evolution, instead of the actual tails. Plus some mirror neurons to spread the rationalization to other humans.)
I would believe the AI if it told me that humans were actually "active" during sleep and had developed a whole additional sleeping civilization whose existence our waking selves were programmed to deny and forget.
I would not believe the AI if it told me that 2 + 2 = 3.
I imagine your AI sending its mechanical avatar to a tail making workshop and attempting to persuade the furry fans that what they are doing is wrong, not because it is absurd, not because it is perverted, but because it is redundant.
It isn't redundant. They don't have a tail that helps them emotionally in whatever way it is that furries like to dress up as animals (I don't know that much about furry fandom).
Also, I couldn't follow any of those links.
Since there are people who do have tails that we can perceive just fine, it's almost certain that people who don't have tails really don't.
Unless people perceive others as having one less tail than they see.
Consider the two possible explanations in the first scenario you describe:
Humans really all have tails.
The AI is just a glorified chat bot that takes in English sentences, jumbles them around at random and spits the result out. Admittedly it doesn't have code for self-deception, but it doesn't have any significant intelligence either. All I did to get the supposed 99% success rate was to basically feed in the answers to the test problems along with the questions. Having dedicated X years of my life to working on AI, I have strong motive for deceiving myself about these things.
If I were in the scenario you describe, and inclined to look at the matter objectively, I would have to admit the second explanation is much more likely than the first. Wouldn't you agree?
Presumably the AI was tested with questions whose answers were not known in advance to guard against the problem of self-deception (or more likely, to ensure that you are capable of convincing others that you are not self-deceiving about the AI's accuracy).
Indeed, and I might believe such testing was carried out and was as effective as it was supposed to be. But my point is, it is much more likely that I am wrong in that belief, than that I am wrong in the belief that we don't have tails. This remains true no matter how thorough the testing. It also remains true if you substitute aliens, gods etc. for the AI; the conclusion doesn't depend on the specifics of the information source.
Ah, I see. That's a good argument.
I think you go wrong when you say that it remains true no matter how thorough the testing. Suppose the AI is beating the stock market over the course of months based on massive online information collection; in the meantime, you're reading webcomics and watching the graph of the AI's money fund plot a trajectory ever upwards. According to you, upon being told by the AI that all humanity is hallucinating something utterly wacky, you should believe that it was actually you beating the stock market all the while, even though as far as you can recall, you are sane and you have had no direct input into the process for months.
I think there are some tests for which success and simultaneous self-deception of the human AI programmer is as unlikely as whatever the AI comes up with about humanity in general.
There are intermediates between "the AI isn't intelligent at all and the statement about tails is just a random output that it produces when there's no preprogrammed chatbot response" and "everything the AI says is done with intelligence". The AI could be intelligent but with a flaw which leads it to make unintelligent statements some of the time without everything it says being unintelligent.
What were we talking about again? Replies to comments I made five years ago always catch me off guard...
Anyway, I don't disagree with you. I'm not sure if you mean to be disagreeing with me, but if so, I would note that my comment doesn't imply or rely on a hard dichotomization of the class of AIs into chatbots and infallible AGIs.
The point is that "the AI beats the stock market, so it's intelligent" rules out a completely unintelligent AI, but doesn't rule out the AI having flaws which lead it to produce stupid results when it comes to humans having tails. This is still true if you add the extra step of "maybe I'm deluded about...". Perhaps the AI is actually intelligent when it comes to the stock market, but you're deluded into thinking the AI makes intelligent decisions in more subject areas than it really does.
(This is especially likely if you've examined the AI's code and "proven" that the AI reasons perfectly. You may have missed something that doesn't happen to affect conclusions about the stock market but does affect conclusions about tails.)
I continue to fail to disagree with you...
"Glorified chatbot" is presumed ruled out; you have both seen the AI's code and seen the AI's performance.
But I've also seen people don't have tails. My point is, if we assume that is a hallucination, we should be even more ready to assume the other is a hallucination.