pjeby comments on What I would like the SIAI to publish - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (218)
Because people make stupid mistakes, especially when programming. And telling your fully-programmed AI what you want it to do still counts as programming.
At this point, I am going to stop my reply, because the remainder of your comment consists of taking things I said out of context and turning them into irrelevancies:
I didn't say an AI would try to convert people to Christianity - I said that humans without sufficient shared background will interpret things literally, and so would AIs.
I didn't say the AI needed to be creative or smart, I said you wouldn't need to be creative or smart to make a list of ways those three simple instructions could be given a disastrous literal interpretation.
There are many paths to superhuman ability, as humans really aren't that smart.
This also means that you can easily be superhuman in ability, and still really dumb -- in terms of comprehending what humans mean... but don't actually say.