Whatever is going wrong in the current surge of obesity, it isn't extraordinarily attractive food.

Yeah, that's true.

I agree. But the fundamental question: what is the good of Man? is going unanswered.

In terms of it going unanswered in the Sequences or wider lesswrong, I somewhat disagree. The sequences specifically argue that good is complex and fragile -- complex meaning it would take a long time to write down all the details and they can't just be summarized with a pattern that gives rise to them; fragile meaning that we need to get all the details right. This means, specifically, that Eliezer did not expect anyone to be able to write everything humans value down and get it right in one shot, even given considerable effort. Instead, some aspects were addressed which were particularly important to illustrate one point or another.

As for me, I also was not expecting to be able to fully articulate what it is that I, or humans, value. I'm trying to articulate my intuitions about this particular issue.

I'm missing a description of what those terminally-valuable goals might be, though.

I think the reason that you're asking is because you think I'm pushing everything off the table, in trying to make a distinction between pleasurable things and actually-valuable things. At times in this conversation, under varying degrees of devil's-advocacy, I've pushed things ranging from fiction to taking a walk in the park off the table. I can see why you're concerned.

My intuition tends to say that nothing is very valuable in isolation. Things gain meaning by their connection to each other (beyond just instrumental value of being able to physically cause more value down the line). This is because value comes from patterns of things, and systems of interconnected structure. A thing like an ice cream cone is not totally devoid of this kind of beauty; it's a matter of degree.

I'd better climb out of the devil's advocate position before I dig myself too deep a hole.

gjm's reply is perfect in terms of describing the position being outlined.

I really do want to make a distinction between pleasurable things and terminally-valuable things, though. At least I think I do.

The way you're reacting makes me think that you don't -- that you find it puzzling that I want to differentiate between superstimulus and actually good things at all, regardless of questions about fiction and such.

I think it would be unfortunate if future civilizations decided maximum wireheading was the greatest ethical good.

I think it would also be unfortunate (but less so) if future civilizations decided that finely crafted full sensory experiences, akin to movies, were the ultimate good.

I furthermore think it would be unfortunate (but significantly less so) if future civilizations decided that finely crafted interactive experiences, akin to 1-player games with only non-sentient NPCs, were the ultimate good.

(With significantly more uncertainty, I think it would be much worse if all of the movies or interactive experiences were identical. The image of billions or more identical clones (human-optimal in whatever sense) watching identical recordings of a single extremely well-crafted thousand-year movie does not appeal very much to me. I'm not sure it's more preferable than a single human experiencing this best-of-all-possible-movies. Similarly, but less so, for interactive experiences.)

The ideal case seems much more like a massively multiplayer one, despite the fact that players will tend to clash with one another and it's much harder to optimize properly (will have to be worse in other respects as a result).

Applying the intuitions from these rather distant scenarios to more everyday matters, the enjoyment from ice cream does fall rather far toward the beginning of the spectrum I've just outlined. It seems rather like a small dose of wireheading (except when enjoyed socially).

(I find it quite amusing that I'm getting push-back on the ice cream thing.)

Probably the most relevant post:

http://www.overcomingbias.com/2007/01/we_cant_foresee.html

Another game proposed to shed light on this:

http://www.overcomingbias.com/2007/01/the_coin_guessi.html

I think the relationship to Aumann's theorem is direct and strong. It's the same old question of how Aumann-like reasoning plays out in practice, for only partially rational agents, that was much discussed back in the Overcoming Bias days.

What concerns me is that I think we should be striving for something better, not justifying the status quo.

That's begging the question: what's better and why?

It's not begging that particular question... the implicit assumption is that the current state of affairs is far from optimal, not that my particular definition of optimal is correct. In fact part of the point is to explore what values we might hold (and still hold after reflection on those values) that would value fiction. I feel this is a valuable exercise largely because when I do reflect on it, arguments to the effect that fiction is something I reflectively value are rather difficult to make. If I were to discover that I don't in fact value fiction on reflection, that would be good news: easy life improvement by no longer acting as if I value fiction.

Many good things are best consumed in moderation. [...] The very name binge-watching labels it as an injurious behavior akin to binge drinking.

True. The point I was trying to make is that when I talk to people about this, they tend to give rather high-minded justifications of the value of fiction (usually as a means to other ends, not as an end in itself). While these high-minded justifications may in fact be correct, they seem very different from the motivation which actually causes people to consume fiction. The result of this difference is that the kind of fiction which is readily available on the market is more often "potato chip fiction" as opposed to "baked potato fiction": still food, but awfully greasy.

This point may not be that relevant to the overall set of questions.

That seems like a fully general argument against any solitary activity

I feel like this remark ignores the part before the text you quote ("there's an actual person") which is very much not a fully general argument, but rather an argument against solitary activities which are misleading superstimulus telling system 1 it's achieving things it's not.

There's also a big difference between creative activities (spending solitary time writing a book, say) and consumptive activities. It's certainly possible to spend alone time without the activity being "isolated" in the sense that I mean.

It's also possible for someone to be entirely creative and not engage in fiction at all while still being "isolated" in the sense I mean. When I imagine a version of me or someone else toiling away at something they love with plans to totally burn it without showing anyone before they die, I feel like something is lost in this sense; not interconnected with the web of life.

Perhaps that's just a bad intuition I have about my values, and/or perhaps it's not a thing many other people value. I don't know.

Beyond that, I think the difference I'm pointing to is that fiction feels disproportionately like a good thing (because it's designed to). If there is any distinction to be made between what feels immediately valuable and what I'd find valuable on reflection, fiction will tend to optimize for the first. (This is also true of other types of information I might consume, but fiction has particularly large freedom to optimize these differences.)

I think people (not just me) generally develop a kind of "memetic immune system" defense against the types of superstimulus which are present in abundance in their culture. We have a much higher bar for humor on television than in real life, because funny things happen with much greater frequency on television. No matter how much more attractive the people on television are, we adjust our expectations (somewhat, at least) and are able to find the people we meet in person relatively attractive. And so on. This generally gives us the ability to not waste too much time on the superstimulus. (It also means that they may not have similar defenses against superstimulus available in other places; an example is people over-doing recreational drugs when they visit places where more things are legal. The native population of those places is not prone to the same excess.)

There's a big difference between saying wireheading and superstimulus are bad and saying enjoyment is bad. The way I'm framing it, that's roughly like the difference between saying that counterfeit money is bad and saying money is bad.

Regardless of how well it follows the random walk, it already violates the assumption of rational agents.

Yes, good point.

Yes, my actual position on this is much closer to "fiction is bad if it's not a social activity" rather than "fiction is bad".

This does not work as an argument against the extremist position, however. Continuing the devil's-advocate line of thought, I say: if fiction is just good as a social activity because I have friends who like fiction, isn't that just me being the elephant tied with a chain to a non-optimal social situation?

I am not saying you're wrong -- in fact I think you are right. What concerns me is that I think we should be striving for something better, not justifying the status quo. That's why I think this is a useful exercise. I'm very skeptical that our current behavior here would just happen to be anywhere near the best we can do. In fact I think fiction is very often more like the ice cream. Our motives for binge-watching an entire series or such are more often self-defeating than good, in any plausible interpretation of the word "good".

As for the final question -- what makes fiction feel more like wireheading than a relationship -- my answer is that there's a real person as opposed to the projected image of a non-real person. The difference is somewhat analogous to the difference between visiting your bank's website and seeing a large sum of money in the account, and visiting a fake banking website whose sole purpose is to simulate the experience of seeing a large sum of money in your account. The actual relationship with an actual person is good in that it not only creates a sequence of feelings and impressions in the brains of both people, but furthermore creates a richly interconnected dance between the two people which is lacking in fiction.