You're mixing up different things:
(A)- a program which will produce an optimal existence for me
(B)- the actual optimal existence for me.
You're saying that if (A) is so fully understood that I feel no excitement studying it, then (B) will likewise be unexciting.
This doesn't follow. Tiny fully understood programs produce hugely varied and unanticipated outputs.
If someone fully understands (and is bored by) the laws of quantum mechanics, it doesn't follow that they are bored by art or architecture or economics, even though everything in the universe (including art or architecture or economics) is eventually an application (many, many layers removed) of particle physics.
Another point that doesn't follow is your seeming assumption that "predictable" and "well-understood" is the same as "boring". Not all feelings of beauty and appreciation stem from surprise or ignorance.
You're saying that if (A) is so fully understood that I feel no excitement studying it, then (B) will likewise be unexciting.
Then I wasn't clear enough, because that's not what I tried to say. I tried to say that from the subjective perspective of a program that completely understands a human being and its complex values, the satisfaction of these complex values will be no more interesting than wireheading.
...Tiny fully understood programs produce hugely varied and unanticipated outputs.
If someone fully understands (and is bored by) the laws of quantum m
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.