LESSWRONG
LW

All of kurt's Comments + Replies

Are wireheads happy?
kurt20

One could also note that 'liking' is something the mammalian and reptile brains are good at, and 'wanting' is often related to deliberation and executive system motivation. Though there are probably different wanting (drive) systems in lower brain layers and in the frontal lobe.

Also, some things we want because they produce pleasure, and some are just interim steps that we carry out 'because we decided'. Evolutionary history determines that we get rewarded when we get actual things far more than when we make one small step. We can sometimes utilize will p... (read more)

Reply
Are wireheads happy?
kurt40

There is a very simple explanation to the seeming discrepancy between wanting and liking, and that is that a person is always experiencing a tension between wanting a bit of pleasure now, versus a lot of pleasure later. Yes, spending time with your family may give you some pleasure now, but staying in NY and putting money aside will give you a lot of security later on. This may not explain the whole difference but perhaps a good chunk of it.

I think wireheading is dismissed far too quickly here and elsewhere. Like it or not (pun intended), the only reason ... (read more)

Reply
2kurt
One could also note that 'liking' is something the mammalian and reptile brains are good at, and 'wanting' is often related to deliberation and executive system motivation. Though there are probably different wanting (drive) systems in lower brain layers and in the frontal lobe. Also, some things we want because they produce pleasure, and some are just interim steps that we carry out 'because we decided'. Evolutionary history determines that we get rewarded when we get actual things far more than when we make one small step. We can sometimes utilize will power, which could be seen as an evolutionarily novel and therefore weak mechanism to provide short term reward for executing steps towards a longer term benefit. Steps that are not rewarded by the opioid system. I think that the more we hack into the brain and the more we will discover that 'wanting' and 'liking' are umbrella terms completely unsuited to capture the subtlety and the sheer messiness of the spaghetti code of human motivation. I am also going to comment on the idea that intelligent agents could have a 'crystalline' (ordered, deterministic) utility evaluation system. We went down that road trying to make AI work, i.e. making brittle systems based on IF/THEN - that stuff doesn't work. So what makes us think that using the same type of approach will work for utility evaluation (which is a hard problem requiring a lot of intelligence)? Humans are adaptable because they get bored, and try new things, and their utility function can change, and different drives can interact in novel ways as the person matures and grows wiser. That can be very dangerous in an AI. But can we really avoid that danger? I am skeptical that we will be able to have a completely bayesian, deterministic utility function. Perhaps we are underestimating how big a chunk of intelligence the evaluation of payoffs really is; and thinking that it won't require the same kind of fine-grained, big-data type of messy, uncertain pattern sm