bogdanb comments on Fusing AI with Superstition - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (75)
But my real objection is different: For any wrong & unchangeable belief you impose, there’s also the risk of unwanted consequences: suppose you use an, eg, fluorine-turns-to-carbon “watchdog-belief” for a (really correct) FAI. The FAI uploads everyone (willingly; it’s smart enough to convince everyone that it’s really better to do it) inside its computing framework. Then it decides that turning fluorine to carbon would be a very useful action (because “free” transmutation is a potentially infinite energy source, and the fluorine is not useful anymore for DNA). Then everybody dies.
Scenarios like this could be constructed for many kinds of “watchdog beliefs”; I conjecture that the more “false” the belief is the more likely it is that it’ll be used, because it would imply large effects that can’t be obtained by physics (since the belief is false), thus are potentially useful. I’m not sure exactly if this undermines the “seems” in the first sentence.
But there’s another problem: suppose that “find a good watchdog” is just as hard (or even a bit easier, but still very hard) problem as “make the AI friendly”. Then working on the first would take precious resources from solving the second.
A minor point: is English your first language? I’m having a bit of trouble parsing some of your comments (including some below). English is not my first language either, but I don’t have this kind of trouble with most everyone else around here, including Clippy. You might want to try formulating your comments more clearly.