= 27d567bc2d48d9f40e9ccc20ea370b15)
derekz comments on Changing accepted public opinion and Skynet - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (58)
If dark arts are allowed, it certainly seems like hundreds of millions of dollars spent on AI-horror movies like Terminator are a pretty good start. Barring an actual demostration of progress toward AI, I wonder what could actually be more effective...
Sometime reasonably soon, getting real actual physical robots into the uncanny valley could start to help. Letting imagination run free, I imagine a stage show with some kind of spookily-competent robot... something as simple as competent control of real (not CGI) articulated robots would be rather scary... for example, suppose that this robot does something shocking like physically taking a human confederate and nailing him to a cross, blood and all. Or something less gross, heh.
...
What's "rational persuation", anyway? Is a person supposed to already possess an ability to change their mind according to an agreed-to-be-safe protocol? Teaching rationality and then giving your complex case would be more natural, but isn't necessarily an option.
The problem is that it's possible to persuade that person of many wrong things, that the person isn't safe from falsity. But if whatever action you are performing causes them to get closer to the truth, it's a positive thing to do in their situation, one selected among many negative things that could be done and that happen habitually.
You know, sci-fi that took the realities of mindspace somewhat seriously could be helpful in raising the sanity waterline on AGI; a well-imagined clash between a Friendly AI and a Paperclipper-type optimizer (or just a short story about a Paperclipper taking over) might at least cause readers to rethink the Mind Projection Fallacy.
Won't work, the clash will only happen in their minds (you don't fight a war if you know you'll lose; you can just proceed directly to the final truce agreement). Eliezer's Three Worlds Collide is a good middle ground, with non-anthropomorphic aliens of human-level intelligence allowing to describe familiar kind of action.
IAWYC, but one ingredient of sci-fi is the willingness to sacrifice some true implications if it makes for a better story. It would be highly unlikely for a FAI and a Paperclipper to FOOM at the same moment with comparable optimization powers such that each thinks it gains by battling the other, and downright implausible for a battle between them to occur in a manner and at a pace comprehensible to the human onlookers; but you could make some compelling and enlightening rationalist fiction with those two implausibilities granted.
Of course, other scenarios can come into play. Has anyone even done a good Paperclipper-takeover story? I know there's sci-fi on 'grey goo', but that doesn't serve this purpose: readers have an easy time imagining such a calamity caused by virus-like unintelligent nanotech, but often don't think a superhuman intelligence could be so devoted to something of "no real value".
That's... the opposite of what I was looking for. It's pretty bad writing, and it's got the Mind Projection Fallacy written all over it. (Skynet is unhappy and worrying about the meaning of good and evil?)
Ironically, a line from the original Terminator movie is a pretty good intuition pump for Powerful Optimization Processes:
It can't be bargained with. It can't be 'reasoned' with. It doesn't feel pity or remorse or fear and it absolutely will not stop, ever, until [it achieves its goal].
Shakey the Robot was funded by DARPA; according to my dad, the grant proposals were usually written in such a way as to imply robot soldiers were right around the corner...in 1967. So it only took about 40 years.