Or you can just join a Quaker meeting.

I realise it is over a year later but can I ask how it went, or whether anyone has advice for someone in a similar position? I felt similar existential terror when reading The Selfish Gene and realising on one level I'm just a machine that will someday break down, leaving nothing behind. How do you respond to something like that? I get that you need to strike a balance between being sufficiently aware of your fragility and mortality to drive yourself to do things that matter (ideally supporting measures that will reduce said human fragility) but not so much you obsess over it and become depressed, but it can seem a pretty tricky balance to strike, especially if you are temperamentally inclined towards obsessiveness, negativity and akrasia.

Now I'm thoroughly confused about your position. Here are some claims to which you appear to have committed yourself:

(1) You can only talk about probability distribution over the microstates of a system if you treat that system as a sub-system of some larger system that includes an observer.

(2) Entropy is just a measure of subjective uncertainty, which means it is (presumably) a property of a probability distribution.

(3) You can talk about the entropy of a system without including the observer but this is just an idealization and it does not involve a probability distribution over the microstates of the system.

To me, this third claim is just flat-out in contradiction with the first and second claims. How can you talk about the entropy of something from a stat. mech. point of view without it being a property of a probability distribution? Is there really some completely different concept of entropy that comes into play when you exclude the observer from your analysis?

I will also note that the approach I talked about in my original comment does not deny that probability is in the mind. Probability can be "in the mind" without just being subjective uncertainty. Furthermore, accepting that probability is in the mind does not mean that one cannot attribute probability distributions to systems without explicitly representing the system as a subsystem of a supersystem containing an observer.

As it happens, if you want to draw the boundary to exclude me, then the two-gas-system-without-pump also happens to be entropy increasing...

This is what I'm disputing you can get if you treat entropy as subjective uncertainty, while also assuming that the only way to update subjective uncertainty is Bayesian conditionalization. Perhaps you can explain how the two-gas system turns out to be entropy increasing on that viewpoint if you draw the boundary to exclude the observer. How does the entropy of the probability distribution describing the system increase?

If this is your standard for entropy increase, then surely it shouldn't matter what you are observing. If you are observing a refrigerator, your brain is pumping just as much heat into the environment as when you are observing spontaneous equilibriation. Yet in the case of the refrigerator we usually say, in contrast to the two-gas system I described, that it is entropy decreasing. But if the basis for calling the two-gas system entropy increasing is the heat output by the observer's brain, why doesn't the refrigerator qualify as entropy increasing as well?

Or are you disagreeing that we would (or should) call a two-gas system exhibiting spontaneous thermal equilibriation an entropy-increasing system? I think I'm not getting your view exactly right.

There is no re-definition of "arrow of time" going on here. Shalizi is using the phrase in its standard thermodynamic sense, describing the fact that a number of macroscopic processes are thermodynamically irreversible.

Consider a specific example: two boxes of gas initially at different temperatures are brought into contact through a diathermal barrier. I check the temperature of these gases using a thermometer periodically. I observe that over time temperature difference vanishes. The gases spontaneously equilibriate.

What would you say about what's going on here? The standard story is that the thermal equilibriation takes place due to the Second Law of Thermodynamics. Heat transfer from the hotter gas to the colder one leads to entropy increase. From a (Boltzmannian) statistical mechanical perspective, the region of phase space corresponding to both gases having the same temperature is larger than the region of phase space corresponding to them having different temperatures. So a distribution that is uniform over the former region (and vanishes elsewhere) will have a higher entropy than a distribution that is uniform over the latter region. Note that none of this requires any appeal to the entropy associated with the observer. The entropy increase in this case has nothing to do with the observer's memory. It has to do with heat flowing from one box of gas to the other.

Now it seems like the guy you link to objects that we can't say that the Second Law applies to this two-gas system because the system is not completely isolated. But this ignores two things. First, the Second Law has productively been used a huge number of times in the past to describe the behavior of systems exactly like this. Second, by this standard the Second Law does not apply to any system. There is no actual system that is completely isolated, except the universe as a whole.

The thing is, the two-gas system is "isolated enough". There is no significant mechanical work being performed on or by it (discounting the negligible amount of work required to raise the mercury in the thermometer I use), as there is in the case of a refrigerator. Observing a system's state does involve some exchange of energy with it, but it need not involve doing work on the system.

Now, Shalizi's point is that if we are strict Bayesians about the state of the system, then the entropy of the distribution we associate with it will not increase, so we would say that the entropy of the system is decreasing. But this is wrong! The entropy of the system is increasing. Not the entropy of the system+observer combo, the entropy of the system itself. If your approach to statistical mechanics tells you it is not, then you are the one flying in the face of orthodox thermodynamics, not Shalizi.

Two things to say here:

(1) The view articulated in that answer, that the Second Law only applies to systems that are genuinely closed, would render the Law empirically useless. There are no systems of this sort, except for the entire universe. But we appeal to the Second Law all the time to account for the time-directedness of systems that aren't completely closed (such as ice melting in a glass of water, or gas spreading through a room). We're really working with an approximate sense of closure, one that allows us to describe reasonably insulated systems as closed (with the denotation of "reasonably" depending on context), even though technically they are exchanging some amount of energy with their environments. If we go by the standards in that post, then yes, no system we observe would be governed by the Second Law. But by the same token, the "system plus observer" supersystem wouldn't be governed by the Second Law either, since this supersystem isn't closed. So then I don't see the point of defending the Second Law by including the observer in the system.

(2) The "begging the question" charge I raised in my post is not merely hypothetical. Shalizi is genuinely skeptical of Landauer's principle, the claim that information erasure must have an entropic cost. So invoking Landauer's principle won't fly against him. I think the right response to the sort of problems he raises with the principle (best captured in the John Norton paper linked in his post) is a view of the sort I recommend above. I'd probably need to say a lot more to make this obvious, but I won't unless you're specifically interested.

ETA: Also worth noting: All competent defenses of Landauer's principle that I have read assume that the observer is governed by the Second Law. The usual argument involves pointing out that erasure involves a reduction of the information theoretic entropy of the data stored by the observer. Since the Second Law holds, this reduction of entropy must be compensated by an increase of entropy in the non-information-bearing degrees of freedom, which usually amounts to the observer releasing heat into the environment. But if we go by the reasoning in the answer to which you link, we have no warrant for assuming the observer is governed by the Second Law unless the observer counts as a genuinely closed system. Of course, no actual observer would qualify. So the poster's own reasoning undermines his appeal to Landauer's principle.

I've just skimmed Shalizi's paper, so I might be wrong, but it seems to me his argument can be summarized as follows:

If we suppose that entropy is a measure of subjective uncertainty, then it would only increase if the subject lost information about the state of the system as it evolves. If the dynamical laws governing the microscopic evolution of the system are information-preserving, then this loss of information can only come from the way in which the subject updates his/her beliefs about the system's state. But if the subject updates by simply conditionalizing on the system's new macroscopic state, then this cannot happen. Bayesian conditionalization can only add information; it cannot subtract information. So, generically, updating one's beliefs about the system by conditionalization will lead to decrease in uncertainty about the system and therefore a decrease in the system's entropy.

I don't think points (1) and (3) in Eliezer's comment are an adequate response to this argument. Point (1) says that when the observer measures the system in order to conditionalize, the entropy of the observer's memory registers increases, which I guess is supposed to compensate for the decrease in system entropy induced by measurement. But this is a non-response. When we do statistical mechanics, we are not usually interested in the entropy of the system plus the observer; we are just interested in the entropy of the system, and it is this entropy that is observed to increase. Also, the response seems to beg the question. On what grounds does Eliezer claim that measurement increases the entropy of the observer's memory? Couldn't Shalizi's argument just be re-applied at this level?

Eliezer's point 3 (as far as I can make sens of it) is that in a quantum universe, from a within-a-branch perspective, the system evolution will not be unitary (and therefore not information-preserving) because the system will have decohered. This is the same point jimrandomh makes here. This is fair enough, but I don't think the Bayesian should be happy attributing entropy increase solely to quantum world-splitting. Statistical mechanics originated with the assumption that the underlying laws are classical, and in the majority of applications this assumption is retained for computational convenience. If the Bayesian position amounts to a rejection of a majority of the work done in statistical mechanics, it seems a pretty big bullet to bite.

Eliezer's point 2 is ultimately where I think the action's at. We don't update statistical distributions simply by conditionalization. Every statistical mechanics text points out that there is a coarse-graining step. When we update our distribution, we coarse-grain over the fine details of the distribution, "smoothing" it out. It is this step that accounts for entropy increase. Now Shalizi's response is that if you are a Bayesian then adding this non-Bayesian step is epistemically incoherent. One way to respond to this is as Eliezer does: Yup, none of us are perfect Bayesians. We are not even close to logically omniscient, so we are doomed to incoherence.

I think there's another response, which is that the best way to think about the probability distributions in statistical mechanics is not as accurate representations of our degrees of belief. The distributions are constructed to remove distinctions between microscopic states that are irrelevant to our macroscopic interactions with the system. Suppose I pour a blob of milk into a cup of coffee on the right side of the cup and then stir. Eventually the milk will be completely mixed with the coffee. If I had poured the blob on the left side of the cup, the milk would also eventually have ended up in a mixed state. Now, technically, my state of knowledge about the microstate of the mixed cup is different in these two cases. In the first case I know that the microstate must be one that evolves from the milk being poured on the right. In the second case I know it must be one that evolves from the milk being poured on the left. If the dynamics of the cup are information-preserving, then these are disjoint subsets of phase space. If I was updating as a Bayesian, the distributions would be totally different from one another.

But the thing is, the original position of the blob of milk makes no difference to my practical ability to interact with the milk and coffee system now that the milk is mixed. I might remember this original position, but I cannot now use that information to extract work from the system. My causal capacities are not sufficiently fine-grained to allow me to do that. So the information is irrelevant to how I now treat the system, from a thermodynamic point of view. To conserve computational resources, I might as well pick a distribution that ignores this information. That distribution will not be the distribution that best represents my knowledge of the system, but it will be the distribution that most effectively allows me to plan interactions with the system.

So I guess ultimately I agree with Shalizi. Thinking of thermodynamic entropy as the same thing as subjective uncertainty is wrong. This doesn't mean it doesn't have a lot to do with subjective uncertainty, though, since our uncertainty about systems is a very important constraint on our ability to interact with them.

I agree that the signal being sent is coarse-grained.

I agree that finegraining it is a lovely thing for people to do if they can do it in a way that's low-cost to everyone else.

I disagree with your implicit separation of signalling community (dis)approval on the one hand, and reputation costs on the other. The reputation in this case is precisely a function of community (dis)approval; I don't see how you can sensibly separating them. If I endorse explicitly signalling community (dis)approval at all (which I do), I can't help but endorse explicitly raising/lowering reputation.

My only concern with the FAQ approach is the question of what an individual voter whose reasons for (dis)approval don't align with the FAQ ought to do. If I'm understanding you, your idea is that the FAQ trumps the actual preferences of people in the community -- that is, I'm expected to vote in accordance with the FAQ rather than my own preferences. That makes the existence and contents of the FAQ an implicit power structure, and such things are best approached with caution.

That said, I don't object to it if so approached.