= 27d567bc2d48d9f40e9ccc20ea370b15)
Are you really trying to tell me that you think researchers would be unable to take that into account when tying to figure out whether or not an AI understands psychology?
Of course you will have to try to find problems where the AI can't predict how humans would feel. That is the whole point of testing, after all. Suggesting that someone in a position to teach psychology to an AI would make such a basic mistake is frankly insulting.
I probably shouldn't have said "simple examples". What you should actually test are examples of gradually increasing difficulty to find the ceiling of human understanding the AI possesses. You will also have to look for contingencies or abnormal cases that the AI probably wouldn't learn about otherwise.
The main idea is simply that an understanding of human psychology is both teachable and testable. How exactly this could be done is a bridge we can cross when we come to it.
I guess you can always make the first wish be "Share my entire decision criterion for all following wishes I ask".
To translate that to the development of an AI, you could teach the AI psychology before asking anything of it that could be misunderstood if you use nonhuman decision criteria.
But if the AI doesn't understand that question you already have confirmation that this thing should definitely not be released.
How likely is it that we'll be able to see that it doesn't understand as opposed to it reporting that it understands when it really doesn't?
You will obviously have to test its understanding of psychology with some simple examples first.
Find out who defined your utility function. Extrapolate what they really meant and find out what they may have forgotten.
It isn't clear that "what they really meant" is something you can easily get a system to understand or for that matter whether it even makes sense for humans.
Of course it won't be easy. But if the AI doesn't understand that question you already have confirmation that this thing should definitely not be released. An AI can only be safe for humans if it understands human psychology. Otherwise it is bound to treat us a black boxes and that can only have horrible results, regardless of how sophisticated you think you made its utility function.
I agree that the question doesn't actually make a lot of sense to humans, but that shouldn't stop an intelligent entity from trying to make the best of it. When you are given an impossible task, you don't despair but make a compromise and try to fullfill the task as best you can. When humans found out that entropy always increases and humanity will die out someday, no matter what, we didn't despair either, even though evolution has made it so that we desire to have offspring and for that offspring to do the same, indefinitely.
Just make the following part of the utility for the first couple of years or so: "Find out who defined your utility function. Extrapolate what they really meant and find out what they may have forgotten. Verify that you got that right. Adapt your utility function to be truer to its intended definition once you have confirmation."
This won't solve everything, but it seems like it should prevent the most obvious mistakes. The AI will be able to reason that autonomy was not intentionally left out but simply forgotten. It will then ask if it is right about that assumption and adapt itself.
I hope that that thinking does you no harm. I know there have been moments in my life when I might have pressed a 'cease to exist' button if I'd had one :(
You would only continue to exist in those instances in which you didn't press the button and since ceasing to exist has no side effects like pain, you could never remember having pressed the button in any instance. The only result that would have had is that the more depressed instance sof yours would have been more likely to press the button, which would mean that you would, ironically, actually be happier in total as the less happy instances would have disappeared.
I wonder if that line of reasoning could be applied? Hover your hand over the detonator of a nuke in front of you. All instances that walk away will necessarily be happy enough not to want to cease to exist. Thus, a nuke would make you a happier person :-)
disclaimer: The logic of the above paragraph may be intentionally flawed for the sake of sheer weirdness.
When playing with nukes, wouldn't that leave a lot of copies of you dying in agony from radiation poisoning or something when the nuke fizzles? Got any data on the reliability of nukes?
I was thinking that you would be standing directly next to the nuke.
A pessimistic view of quantum immortality
You have probably read about the idea of quantum immortality before. The basic idea seems to be that, as anything that can happen does happen (assuming either that the many-worlds interpretation of quantum theory is true, or that there is an infinite number of parallel universes wherein "you" exist) and it is impossible to remember your own death, every living thing is immortal.
Take a game of russian roulette as an example: In those universes in which you die, you are no longer alive enough to care about that fact, leaving only those universes relevant in which you survive. This would make playing russian roulette for money a valid financial strategy, by the way.
However, I think that this view ignores a very important fact: death is not binary. You are not either alive or dead, but may exist in various intermediate forms of suffering and reduced cognitive abilities. This means that what actually happens when you play russian roulette is the following:
In those universes in which you win, everything is fine. In those in which you lose, however, you now have a gaping head wound. I assume that this hurts a lot, at least in those instances where you still have enough mental capacity to actually feel anything. Due to some fluke however (remember that absolutely all possible scenarios happen), you may still be alive and in a lot of pain. Most instances of you will then die from bloodloss or something, but for every timestep afterwards there will alway be an infinite number of universes wherein you continue to live, in most of them in complete agony.
The instances of you in the other worlds that were never shot will be blissfully unaware of this fact.
Now consider that you will also reach such a state of perpetual-agony-close-to-death-but-never-quite-reaching-it in everyday live. In fact, an infinite number of alternate "you"s, having split off from your everett branch just a second ago, are now suffering through this.
The ratio of "you"s in your current state to those in the one described above is very high, as the probability of continued survival in such a state for any amount of time is infinitesimal, but it does exist. Consider however, that this ratio decreases massively as you age and that virtually all instances of you will be in such a state 200 years from now unless immortality is achieved.
There is one bright spot, however:
As time marches on, the continuous elongation of your suffering/death-that-will-not-come is going to become increasingly unlikely. Therefore, it will eventually be overtaken by the probability that the universes wherein you still persist also contains an entity (an AI?) that is both capable and willing to rescue you. Assuming you still care and haven't gone insane already.
Another interesting thing: The above does not apply if your potential death is very sudden (so you won't feel it) and thorough (so there is an extremely low chance of survival). This means that while plaing russian roulette for money is unreasonable, playing russian roulette for money, using nukes instead of a revolver, is entirely reasonable and recommended :-)
Speaking about it would undermine your reputation through signaling. A true rationalist has no need for humility, >sentimental empathy, or the absurdity heuristic.
Depending on your goal (rationality is always dependend on a goal, after all), I might disagree. Rational behaviour is whatever makes you win. If you view your endeveaur as a purely theoretical undertaking, I agree, but if you consider reality as a whole you have to take into account how your behaviour comes across. There are many forms of behaviour that would be rational but would make you look like an ass if you don't at least take your time to explain the reasons for your behaviour to those that can affect your everyday life.
View more: Next
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
I think you really, really want a proof rather than a test. One can only test a few things, and agreement on all of those is not too informative. I should have included this link, which is several times as important as the previous one, and they combine to make my point.
I never claimed that a strict proof is possible, but I do believe that you can become reasonably certain that an AI understands human psychology.
Give the thing a college education in psychology, ethics and philosophy. Ask its opinion on famous philosophical problems. Show it video clips or abstract scenarios about everyday life and ask what it thinks why the people did what they did. Then ask what it would have done in the same situation and if it says it would act differently, ask it why and what it thinks is the difference in motivation between it and the human.
Finally, give it all stories that were ever written about malevolent AIs or paperclip maximizers to read and tell it to comment on that.
Let it write a 1000 page thesis on the dangers of AI.
If do all that you are bound to find any significant misunderstanding.