Very interesting post, but your conclusion seems too strong. Presumably, if instead of messing around with artificial experiencers, we just fill the universe with humans being wireheaded, we should be able to get large quantities of real pleasure with fairly little actually worthwhile experiences; we might even be able to get away with just disembodied human brains. Given this, it seems highly implausible that if we try to transfer this process to a computer, we are forced to create agent so rich and sophisticated that their lives are actually worth living.
Correspondence (matching theories to observations) is a subset of coherence (matching everything with everything)
Correspondence is not just matching theories to observation. It is matching theories to reality. Since we don't have pure transcendent access to reality, this involves a lot of matching theories to observation and to each other, and rejecting the occasional observation as erroneous; however, the ultimate goal is different from that of coherence, since perfectly coherent sets of statements can still be wrong.
If your point is that "reality" is not a meaningful concept and we should write off the philosophizing of correspondence theorists and just focus on what they actually do, then what they actually do is identical to what coherentists actually do, not a subset.
Those are entirely valid points, but they only show that human desires are harder to satiate than you might think, not that satiating them would be insufficient to eliminate scarcity. And in fact, that could not possibly be the case even granting economy's definition of scarcity, because if you have no unmet desires, you do not need to make choices about what uses to put things to. If once you digitize your books, you want nothing in life except to read all the books you can now store, you don't need to put the shelf space to another use; you can just leave it empty.
You should add a link to the previous post at the top, so people who come across this don't get confused by the sand metaphor.
This will hopefully be addressed in later posts, but on its own, this reads like an attempt to legislate a definition of the word 'scarcity' without a sufficient justification for why we should use the word in that way. (It could also be an explanation of how 'scarcity' is used as a technical term in economics, but it is not obvious to me that the alternate uses/unsatiated desires distinction is relevant to what most economists spend their time on. If this is your intention, could you elaborate and give evidence that economists do in fact use the term in this way?) Naively, it seems that if we have enough Star Trek replicators or whatever to satiate all human desires, then for all practical purposes scarcity has been eliminated, and insisting that it hasn't really because things have alternative uses because some things still have alternative uses seems like playing unproductive word games. Can you explain why thinking of scarcity in your terms is advantageous?
Overall, this series has so far been quite fun to read. I look forward to more.
From Yudkowsky's Epistle to the New York Less Wrongians:
Knowing about scope insensitivity and diminishing marginal returns doesn't just mean that you donate charitable dollars to "existential risks that few other people are working on", instead of "The Society For Curing Rare Diseases In Cute Puppies". It means you know that eating half a chocolate brownie appears as essentially the same pleasurable memory in retrospect as eating a whole brownie, so long as the other half isn't in front of you and you don't have the unpleasant memory of exerting willpower not to eat it.
If you mainly value brownie-eating memories, this is perfectly reasonable advice. If you instead you eat brownies for the experience, it is unhelpful, since eating half a brownie means the experience is either half as long or less intense.
Is this the sort of thing you're looking for?
OrphanWilde appears to be talking about morality, not decision theory. The moral Utility Function of utilitarianism is not necessarily the decision-theoretic utility function of any agent, unless you happen to have a morally perfect agent lying around, so your procedure would not work.
The most obvious explanation for this is that utility is not a linear function of response time: the algorithm taking 20 s is very, very bad, and losing 25 ms on average is worthwhile to ensure that this never happens. Consider that if the algorithm is just doing something immediately profitable with no interactions with anything else (e.g. producing some crytptocurrency), the first algorithm is clearly better (assuming you are just trying to maximize expected profit), since on the rare occasions when it takes 20 s, you just have to wait almost 200 times as long for your unit of profit. This suggests that the only reason the second algorithm is typically preferred is that most programs do have to interact with other things, and an extremely long response time will break everything. I don't think any more convoluted decision theoretic reasoning is necessary to justify this.
Part of the issue is that you are not subject to the principle of explosion. You can assert contradictory things without also asserting that 2+2=3, so you can be confident that you will never tell anyone that 2+2=3 without being confident that you will never contradict yourself. Formal systems using classical logic can't do this: if they prove any contradiction at all, they also prove that 2+2=3, so proving that they don't prove 2+2=3 is exactly the same thing as proving that they are perfectly consistent, which they can't consistently do.
You cannot possibly gain new knowledge about physics by doing moral philosophy. At best, you have shown that any version of utilitarianism which adheres to your assumptions must specify a privileged reference frame in order to be coherent, but this does not imply that this reference frame is the true one in any physical sense.
So if I spouted 100 billion true statements at you, then said, "It would be good for you to give me $100,000," you'd pay up?