pjeby comments on The Human's Hidden Utility Function (Maybe) - Less Wrong

44 Post author: lukeprog 23 January 2012 07:39PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (87)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 23 January 2012 10:59:08PM 2 points [-]

Let me try again:

Two-step process = (1) Extract preferences, (2) Extrapolate preferences. This may not work. This is one reason that this discovery about three valuation systems in the brain is so weak and preliminary for the purposes of CEV. I'm not sure it will turn out to be relevant to CEV at all.

Comment author: pjeby 23 January 2012 11:07:15PM 2 points [-]

I think you've also missed the possibility that all three "systems" might just be the observably inconsistent behavior of one system in different edge cases, or at least that the systems are far more entangled and far less independent than they seem.

(I think you may have also ignored the part where, to the extent that the model-based system has values, they are often more satisficing than maximizing.)