TheAncientGeek comments on Debunking Fallacies in the Theory of AI Motivation - LessWrong

8 Post author: Richard_Loosemore 05 May 2015 02:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (343)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheAncientGeek 14 May 2015 01:08:54PM *  1 point [-]

Which AI? As so often, an architecture dependent issue is being treated as a universal truth.

This statement maps to "programs execute their code." I would be surprised if that were controversial.

Humans can fail to realise the implications of uncontroversial statements. Humans are failing to realise that goal stability is architecture dependent.

This was covered by the comment about "meta-values" earlier, and "Y being a fuzzy object itself," which is probably not as clear as it could be. The goal management system grounds out somewhere, and that root algorithm is what I'm considering the "values" of the AI. 

But you shouldn't be, at least in an un scare quoted sense of values. Goals and values aren't descriptive labels for de facto behaviour. The goal if a paperclipper is to make paperclips; if it crashes, as an inevitable result of executing its code, we don't say, " Aha! It had the goal to crash all along".

Goal stability doesn't mean following code, since unstable systems follow their code too....using the actual meaning of "goal".

Meta: trying to defend a claim by changing the meaning of its terms is doomed to failure.