wedrifid comments on SotW: Be Specific - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (306)
It doesn't sound like particularly common sense - I'd guess that significantly less than half of humans would arrive at that as a cached 'common sense' solution.
It's utterly trivial application of instrumental rationality. I can come up with it in 2 seconds. If the AI is as smart as I am (and with far less human biases) it can arrive at the solution as simply as I can. Especially after it reads every book on strategy that humans have written. Heck, it can read my comment and then decide whether it is a good strategy.
Artificial intelligences aren't stupid.
Or... not. That's utter nonsense. We have been explicitly describing AIs that have been programmed with terminal goals. The AI would then
CEV is well enough defined that it just wouldn't do that unless you actually do want it - in which case you, well, want it to do that so have no cause to complain. Reading even the incomplete specification from 2004 is sufficient to tell us that a GAI that does that is not implementing something that can reasonably called CEV. I must conclude that you are replying to a straw man (presumably due to not having actually read the materials you criticise.)
CEV is not defined to do what you as-is actually want, but to do what you would of wanted, even in circumstances when you as-is actually want something else, as the 2004 paper cheerfully explains.
In any case, once you assume such intent-understanding interpretative powers of AI, it's hard to demonstrate why instructing the AI in plain English to "Be a good guy. Don't do bad things" would not be a better shot.
Programmed in with great effort, thousands of hours of research and development and even then great chance of failure. That isn't "assumption".
That would seem to be a failure of imagination. That exhortation tells even an FAI-complete AI that is designed to follow commands to do very little.