= 27d567bc2d48d9f40e9ccc20ea370b15)
eli_sennesh comments on On Terminal Goals and Virtue Ethics - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (205)
Bzzzt. Wrong. You still haven't explained how to create an agent that will faithfully implement my verbal instruction to bring me a pizza. You have a valid case in the sense of pointing out that there can easily exist a "middle ground" between the Superintelligent Artificial Ethicist (Friendly AI in its fullest sense), the Superintelligent Paper Clipper (a perverse, somewhat unlikely malprogramming of a real superintelligence), and the Reward-Button Addicted Reinforcement Learner (the easiest unfriendly AI to actually build). What you haven't shown is how to actually get around the Addicted Reinforcement Learner <i>and</i> the paper-clipper and actually build an agent that can be sent out for pizza without breaking down at all.
Your current answers seem to be, roughly, "We get around the problem by expecting future AI scientists to solve it for us." However, we are the AI scientists: if we don't figure out how to make AI deliver pizza on command, who will?
You keep misreading me. I am not claiming that to gave a solution. I am claiming that MIRI is overly pessimistic about the problem, and offering an over engineered solution. Inasmuch ad you say there is a middle ground, you kind if agree.
The thing is, MIRI doesn't claim that a superintelligent world-destroying paperclipper is the most likely scenario. It's just illustrative of why we have an actual problem: because you don't need malice to create an Unfriendly AI that completely fucks everything up.
To make reliable predictions, more realistic examples are needed.
So how did you like CATE, over in that other thread? That AI is non-super-human, doesn't go FOOM, doesn't acquire nanotechnology, can't do anything a human upload couldn't do... and still can cause quite a lot of damage simply because it's more dedicated than we are, suffers fewer cognitive flaws than us, has more self-knowledge than us, and has no need for rest or food.
I mean, come on: what if a non-FOOMed but Unfriendly AI becomes as rich as Bill Gates? After all, if Bill Gates did it while human, than surely an AI as smart as Bill Gates but without his humanity can do the same thing, while causing a bunch more damage to human values because it simply does not feel Gates' charitable inclinations.