latanius comments on Wanting to Want - Less Wrong

16 Post author: Alicorn 16 May 2009 03:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (185)

You are viewing a single comment's thread. Show more comments above.

Comment author: Drahflow 17 May 2009 08:30:40PM 0 points [-]

In a perfectly rational agent, no n-th order wants should exist.

Your problems with mountain dew might account for -1 util, you being awake for 2 utils, then you "want" to drink that stuff. Shut up and add.

The only source of multi-level desires I can see is an imperfect caching algorithm, which spews forth "Do not drink mountain dew" although the overall utility would be positive.

Comment author: latanius 18 May 2009 08:20:50AM 2 points [-]

Or the vertical hierarchy in our brains, and our inability to modify it. All four examples show "first order desires" created by lower level subsystems (taste perception, sexuality, motivations, etc.), and the higher level ones are trying to override it (consciousness, planning, models of the future world). But we aren't designed rationally enough to be able to control those lower levels, although we know partially about what they "think". ("We" = the conscious part writing LW comments...)