Note that I am not the person making the argument, just clarifying what is meant by "utility", which in its use around here specifically means that which is constructed by the VNM theorem. I am not a particular fan of applying the concept to universal decision-making.

You still end up with zero at the end of things.

Are you arguing that all things end, therefore there is no value in anything?

Well, there is precedent:

All is vanity. What does man gain by all the toil at which he toils under the sun?

I said in my heart, “Come now, I will test you with pleasure; enjoy yourself.” But behold, this also was vanity. I said of laughter, “It is mad,” and of pleasure, “What use is it?”

Then I considered all that my hands had done and the toil I had expended in doing it, and behold, all was vanity and a striving after wind, and there was nothing to be gained under the sun.

The wise person has his eyes in his head, but the fool walks in darkness. And yet I perceived that the same event happens to all of them. Then I said in my heart, “What happens to the fool will happen to me also. Why then have I been so very wise?” And I said in my heart that this also is vanity. For of the wise as of the fool there is no enduring remembrance, seeing that in the days to come all will have been long forgotten. How the wise dies just like the fool! So I hated life, because what is done under the sun was grievous to me, for all is vanity and a striving after wind.

"If you come back an tell me that "these scenarios assume an unlimited availability of time" or something like that, I'll ask to see if the dragon in your garage is permeable to flour."

Not being realistic is not a valid criticism of a theoretical situation if the theoretical situation is not meant to represent reality. I've made no claims of how it carries over to the real world

"Insufficient context" - the context is perfectly well defined. How tired do I get considering large numbers? You don't get tired at all! What is the opportunity cost of considering large number? There is no opportunity cost at all. And so on. It's all very well defined.

"Responded that the solution is to not play the game, but for the actor to grab as much utility as it could get within a certain finite time limit according to its stopping function and go about its business." - except that's not a single solution, but multiple solutions, depending on which number you stop at.

"If it does not, then what is the point?" - This is only part 1. I plan to write more on this subject eventually. As an analogy, a reader of a book series can't go to an author and demand that they release volume 2 right now so that they can understand part 1 in its full context. My objective here is only to convince people of this abstract theoretical point, because I suspect that I'll need it later (but I don't know for certain).

Heat death is a problem that the builders of the game have to deal with. Every time I type out BB(BB(BB(...))) the builder of the game has to figure out how I can get a noncomputable increase to the degree of the function by which the multiple of my preference for the world increases. If there is some conceivable world with no heat death which I prefer any computable amount more than any world with a heat death (and infinity is not a utility!), then by playing this game I enter such a world.

Note that I am not the person making the argument, just clarifying what is meant by "utility", which in its use around here specifically means that which is constructed by the VNM theorem. I am not a particular fan of applying the concept to universal decision-making.

You still end up with zero at the end of things.

Are you arguing that all things end, therefore there is no value in anything?

Well, there is precedent:

All is vanity. What does man gain by all the toil at which he toils under the sun?

I said in my heart, “Come now, I will test you with pleasure; enjoy yourself.” But behold, this also was vanity. I said of laughter, “It is mad,” and of pleasure, “What use is it?”

Then I considered all that my hands had done and the toil I had expended in doing it, and behold, all was vanity and a striving after wind, and there was nothing to be gained under the sun.

The wise person has his eyes in his head, but the fool walks in darkness. And yet I perceived that the same event happens to all of them. Then I said in my heart, “What happens to the fool will happen to me also. Why then have I been so very wise?” And I said in my heart that this also is vanity. For of the wise as of the fool there is no enduring remembrance, seeing that in the days to come all will have been long forgotten. How the wise dies just like the fool! So I hated life, because what is done under the sun was grievous to me, for all is vanity and a striving after wind.

True, everything does exist in context. And the context being considered here, is not the real world, but the behaviour in a purely theoretically constructed world. I have made no claims that it corresponds to the real world as of yet, so claiming that it doesn't correspond to the real world is not a valid criticism.

The point of utilons is to scale linearly, unlike, say dollars. Maybe there's a maximum utility that can be obtained, but they never scale non-linearly. The task where you can name any number below 100, but not 100 itself, avoids these issues though.

I don't understand your objection to the Unlimited Swap scenario, but isn't it plausible that a perfectly rational agent might not exist?

My gut response to the unbounded questions is that a perfectly rational agent would already know (or have a good guess as to) the maximum utility that it could conceivably expect to use within the limit of the expected lifespan of the universe.

There is also an economic objection; at some point it seems right to expect the value of every utilon to decrease in response to the addition of more utilons into the system.

In both objections I'm approaching the same thing from different angles: the upper limit on the "unbounded" utility in this case depends on how much the universe can be improved. The question of how to achieve maximum utility in those scenarios is malformed similarly to that of asking the end state of affairs after completing certain supertasks. More context is needed. I suspect the same is also true for the Unlimited Swap scenario.

I think placebomancy is the best word I've seen in years!

What is success for a religion?

I'd say it seems pretty fundamental that Religion is a kind of meme. One measure of success for memes is their ability to spread (virulence). If that is your only measure, you're likely to have some ethically terrible things going on. It seems (to me) like an obvious constraint that the meme must spread without causing any obvious harm in its wake (except to related memes that may be in competition).

What useful purposes does religion serve?

I consider [create an in-group] to be a pretty central (usually unstated) optimization point (not exactly a purpose) in most religions. This has some well-studied psychological effects on said group that can benefit all its members socially, psychologically, and (further down the causal line) medically. Although it tends to lead to some unfortunate side-effects for the out-group.

How would you design a "rational religion", if such an entity is possible?

Problem is that I've seen more than one source define "religion" as something like "systematic belief in the supernatural". I'm not convinced that such belief can be "rational" (optimal for making more effective decisions). Perhaps as a stepping stone -- use a religious-type belief system as an infection vector, then strip the supernatural elements away bit by bit; but that seems awfully deceitful to me.