eli_sennesh comments on Debunking Fallacies in the Theory of AI Motivation - LessWrong

8 Post author: Richard_Loosemore 05 May 2015 02:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (343)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 07 May 2015 02:40:17PM 3 points [-]

If the AI starts to get even a hint that there are objections, it has to kick in a serious review of the plan. It will ask everyone (it is an AI, after all: it can do that even if there are 100 billion people on the planet). If it gets feedback from anyone saying that they object to the plan, that is the end of the story: it does not force anyone to go through with it. That means, it is a fundamental feature of the checking code that it will veto a plan under that circumstance. Notice, by the way, that I have generalized "consulting the programmers" to "consulting everyone". That is an obvious extension, since the original programmers were only proxies for the will of the entire species.

This assumes that no human being would ever try to just veto everything to spite everyone else. A process for determining AGI volition that is even more overconstrained and impossible to get anything through than a homeowners' association meeting sounds to me like a bad idea.