Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

keex comments on Continuous Improvement - Less Wrong

16 Post author: Eliezer_Yudkowsky 11 January 2009 02:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (24)

Sort By: Old

You are viewing a single comment's thread.

Comment author: keex 06 February 2011 03:26:20PM *  0 points [-]

And on a moral level, it sounds perilously close to tampering with Boredom itself.

I am not sure we need necessarily to shy away from tampering with our reward system. To me it seems, that the whole reward system is already somewhat arbitrary, shaped by necessities, largely from our past.

We may (enjoy) feel rewarded by building our influence/power, to generate and take care of offspring, explore, both spatially and in-depth by understanding things better. All this seems already biased. Also it seems that the younger people seem to be more be oriented towards rewards related to exploration, expansion, whereas the older we get, we seem to shift our focus into retaining, stabilising, attempting to guarantee what we have built during lifetimes.

Although this value set has probably been giving advantage to us for long time, it is likely relatively slow to change, and it is based on assumptions. These same values might not provide direct usefulness by themselves, should these assumptions no longer hold, perhaps even only due to change in our situation as the most intelligent race on the planet.

For example, if mankind's ability of developing abstract sciences would be rendered useless by superintelligence, which would provide us with the results in much more efficient way, then feeling rewarded by re-inventing and trying to prove/disprove theories already available to us might not give us any advantage. Should we work still towards honing our skills to be able to expand what we know ourselves? Perhaps, but not necessarily. Maybe it would give more value for us then, for example, to concentrate on trying to better understand and acquire the information that is already provided to us, concentrate on learning instead of trying to rush ahead, expand.

That kind of shifts, some subtler, some perhaps more radical, they are, in essence, changes in our core "enjoyment" value systems.