TheAncientGeek comments on Debunking Fallacies in the Theory of AI Motivation - LessWrong

8 Post author: Richard_Loosemore 05 May 2015 02:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (343)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheAncientGeek 15 May 2015 06:38:37PM 1 point [-]

I have actually thought about that a lot, too, and my conclusion is that we should not beat ourselves up trying to figure out precisely what the difference might be between these nuanced versions of the idea, because the people who are proposing this idea in the first place have not themselves been clear enough about what is meant

They are clear that they don't mean AIs rigid behaviour is the result of it assessing its own inferrential processes as infallible ... that is what the controversy is all about..

The problem is, we are talking about an AI, and some people talk as if the AI can run its planning engine, then feel compelled to obey the planning engine ... while at the same time being fully cognizant of evidence that the planning engine produced a crappy plan.

That is just what The Genie Knows but doesn't Care is supposed to answer. I think it succeeds in showing that a fairly specific architecture would behave that way, but fails in it's intended goal of showing that this behaviour is universal or likely.