I think you're ignoring the difference between the Boltzmann and Gibbs entropy, both here and in your original comment. This is going to be long, so I apologize in advance.

Gibbs entropy is a property of ensembles, so it doesn't change when there is a spontaneous fluctuation towards order of the type you describe. As long as the gross constraints on the system remain the same, the ensemble remains the same, so the Gibbs entropy doesn't change. And it is the Gibbs entropy that is most straightforwardly associated with the Shannon entropy. If you interpret the ensemble as a probability distribution over phase space, then the Gibbs entropy of the ensemble is just the Shannon entropy of the distribution (ignoring some irrelevant and anachronistic constant factors). Everything you've said in your comments is perfectly correct, if we're talking about Gibbs entropy.

Boltzmann entropy, on the other hand, is a property of regions of phase space, not of ensembles or distributions. The famous Boltzmann formula equates entropy with the logarithm of the volume of a region in phase space. Now, it's true that corresponding to every phase space region there is an ensemble/distribution whose Shannon entropy is identical to the Boltzmann entropy, namely the distribution that is uniform in that region and zero elsewhere. But the converse isn't true. If you're given a generic ensemble or distribution over phase space and also some partition of phase space into regions, it need not be the case that the Shannon entropy of the distribution is identical to the Boltzmann entropy of any of the regions.

So I don't think it's accurate to say that Boltzmann and Shannon entropy are the same concept. Gibbs and Shannon entropy are the same, yes, but Boltzmann entropy is a less general concept. Even if you interpret Boltzmann entropy as a property of distributions, it is only identical to the Shannon entropy for a subset of possible distributions, those that are uniform in some region and zero elsewhere.

As for the question of whether Boltzmann entropy can decrease spontaneously in a closed system -- it really depends on how you partition phase space into Boltzmann macro-states (which are just regions of phase space, as opposed to Gibbs macro-states, which are ensembles). If you define the regions in terms of the gross experimental constraints on the system (e.g. the volume of the container, the external pressure, the external energy function, etc.), then it will indeed be true that the Boltzmann entropy can't change without some change in the experimental constraints. Trivially true, in fact. As long as the constraints remain constant, the system remains within the same Boltzmann macro-state, and so the Boltzmann entropy must remain the same.

However, this wasn't how Boltzmann himself envisioned the partitioning of phase space. In his original "counting argument" he partitioned phase space into regions based on the collective properties of the particles themselves, not the external constraints. So from his point of view, the particles all being scrunched up in one corner of the container is not the same macro-state as the particles being uniformly spread throughout the container. It is a macro-state (region) of smaller volume, and therefore of lower Boltzmann entropy. So if you partition phase space in this manner, the entropy of a closed system can decrease spontaneously. It's just enormously unlikely. It's worth noting that subsequent work in the Boltzmannian tradition, ranging from the Ehrenfests to Penrose, has more or less adopted Boltzmann's method of delineating macrostates in terms of the collective properties of the particles, rather than the external constraints on the system.

Boltzmann's manner of talking about entropy and macro-states seems necessary if you want to talk about the entropy of the universe as a whole increasing, which is something Carroll definitely wants to talk about. The increase in the entropy of the universe is a consequence of spontaneous changes in the configuration of its constituent particles, not a consequence of changing external constraints (unless you count the expansion of the universe, but that is not enough to fully account for the change in entropy on Carroll's view).

Yeah, I think the initial exclusivity of Facebook really helped. I went to a school near Harvard at the time Facebook launched, and we were all vaguely aware of the site when it first launched as a Harvard-only site. It then expanded to include our school and a few others -- maybe ten or so, all quite prestigious -- and there was widespread adoption almost instantaneously on our campus. I think the sense of being invited to join an exclusive club had a lot to do with that. I don't know if Zuckerberg intended it, but playing on the elitism of college students was a very effective strategy to achieve rapid adoption at the early stage, and of course once that was achieved, there was enough momentum to ensure success once the site steadily opened up to larger and larger populations.

From a decision-theory perspective, I should essentially just ignore the possibility that I'm in the first 100 rooms - right?

Well, what do you mean by "essentially ignore"? If you're asking if I should assign substantial credence to the possibility, then yeah, I'd agree. If you're asking whether I should assign literally zero credence to the possibility, so that there are no possible odds -- no matter how ridiculously skewed -- I would accept to bet that I am in one of those rooms... well, now I'm no longer sure. I don't exactly know how to go about setting my credences in the world you describe, but I'm pretty sure assigning 0 probability to every single room isn't it.

Consider this: Let's say you're born in this universe. A short while after you're born, you discover a note in your room saying, "This is room number 37". Do you believe you should update your belief set to favor the hypothesis that you're in room 37 over any other number? If you do, it implies that your prior for the belief that you're in one of the first 100 rooms could not have been 0.

(But. on the other hand, if you think you should update in favor of being in room x when you encounter a note saying "You are in room x", no matter what the value of x, then you aren't probabilistically coherent. So ultimately, I don't think intuition-mongering is very helpful in these exotic scenarios. Consider my room 37 example as an attempt to deconstruct your initial intuition, rather than as an attempt to replace it with some other intuition.)

Theoretically there are as many multiples of 10 as not (both being equinumerous to the integers), but if we define rationality as the "art of winning", then shouldn't I guess "not in a multiple of 10"?

Perhaps, but reproducing this result doesn't require that we consider every room equally likely. For instance, a distribution that attaches a probability of 2^(-n) to being in room n will also tell you to guess that you're not in a multiple of 10. And it has the added advantage of being a possible distribution. It has the apparent disadvantage of arbitrarily privileging smaller numbered rooms, but in the kind of situation you describe, some such arbitrary privileging is unavoidable if you want your beliefs to respect the Kolmogorov axioms.

When I say the probability distribution doesn't exist, I'm not talking about the possibility of the world you described. I'm talking about the coherence of the belief state you described. When you say "The probability of you being in the first 100 rooms is 0", it's a claim about a belief state, not about the mind-independent world. The world just has a bunch of rooms with people in them. A probability distribution isn't an additional piece of ontological furniture.

If you buy the Cox/Jaynes argument that your beliefs must adhere to the probability calculus to be rationally coherent, then assigning probability 0 to being in any particular room is not a coherent set of beliefs. I wouldn't say this is a case of probability theory not being "expressive enough". Maybe you want to argue that the particular belief state you described ("Being in any room is equally likely") is clearly rational, in which case you would be rejecting the idea that adherence to the Kolmogorov axioms is a criterion for rationality. But do you think it is clearly rational? On what grounds?

(Incidentally, I actually do think there are issues with the LW orthodoxy that probability theory limns rationality, but that's a discussion for another day.)

There is no such thing as a uniform probability distribution over a countably infinite event space (see Toggle's comment). The distribution you're assuming in your example doesn't exist.

Maybe a better example for your purposes would be picking a random real number between 0 and 1 (this does correspond to a possible distribution, assuming the axiom of choice is true). The probability of the number being rational is 0, the probability of it being greater than 2 is also 0, yet the latter seems "more impossible" than the former.

Of course, this assumes that "probability 0" entails "impossible". I don't think it does. The probability of picking a rational number may be 0, but it doesn't seem impossible. And then there's the issue of whether the experiment itself is possible. You certainly couldn't construct an algorithm to perform it.

Do you think that this is what utilitarianism is, or ought to be?

Utilitarianism does offer the possibility of a precise, algorithmic approach to morality, but we don't have anything close to that as of now. People disagree about what "utility" is, how it should be measured, and how it should be aggregated. And of course, even if they did agree, actually performing the calculation in most realistic cases would require powers of prediction and computation well beyond our abilities.

The reason I used the phrase "artificially created", though, is that I think any attempt at systematization, utilitarianism included, will end up doing considerable violence to our moral intuitions. Our moral sensibilities are the product of a pretty hodge-podge process of evolution and cultural assimilation, so I don't think there's any reason to expect them to be neatly systematizable. One response is that the benefits of having a system (such as bias mitigation) are strong enough to justify biting the bullet, but I'm not sure that's the right way to think about morality, especially if you're a moral realist. In science, it might often be worthwhile using a simplified model even though you know there is a cost in terms of accuracy. In moral reasoning, though, it seems weird to say "I know this model doesn't always correctly distinguish right from wrong, but its simplicity and precision outweigh that cost".

So, do you think that, absent a formal algorithm, when presented with a "save" formulation, a (properly trained) philosopher should immediately detect the framing effect, recast the problem in the "die" formulation (or some alternative framing-free formulation), all before even attempting to solve the problem, to avoid anchoring and other biases?

Something like this might be useful, but I'm not at all confident it would work. Sounds like another research project for the Harvard Moral Psychology Research Lab. I'm not aware of any moral philosopher proposing something along these lines, but I'm not extremely familiar with that literature. I do philosophy of science, not moral philosophy.

Realize what's occurring here, though. It's not that individual philosophers are being asked the question both ways and are answering differently in each case. That would be an egregious error that one would hope philosophical training would allay. What's actually happening is that when philosophers are presented with the "save" formulation (but not the "die" formulation) they react differently than when they are presented with the "die" formulation (but not the "save" formulation). This is an error, but also an extremely insidious error, and one that is hard to correct for. I mean, I'm perfectly aware of the error, I know I wouldn't give conflicting responses if presented with both options, but I am also reasonably confident that I would in fact make the error if presented with just one option. My responses in that case would quite probably be different than in the counterfactual where I was only provided with the other option. In each case, if you subsequently presented me with the second framing, I would immediately recognize that I ought to give the same answer as I gave for the first framing, but what that answer is would, I anticipate, be impacted by the initial framing.

I have no idea how to avoid that sort of error, beyond basing my answers on some artificially created algorithm rather than my moral judgment. I mean, I could, when presented with the "save" formulation, think to myself "What would I say in the 'die' formulation?" before coming up with a response, but that procedure is still susceptible to framing effects. The answer I come up with might not be the same as what I would have said if presented with the "die" formulation in the first place.

This assumption comes from expecting an expert to know the basics of their field.

I wouldn't characterize the failure in this case as reflecting a lack of knowledge. What you have here is evidence that philosophers are just as prone to bias as non-philosophers at a similar educational level, even when the tests for bias involve examples they're familiar with. In what sense is this a failure to "know the basics of their field"?

A trained physicist's intuition is rather different from "human intuition" on physics problems, so that's unlikely.

A relevantly similar test might involve checking whether physicists are just as prone as non-physicists to, say, the anchoring effect, when asked to estimate (without explicit calculation) some physical quantity. I'm not so sure that a trained physicist would be any less susceptible to the effect, although they might be better in general at estimating the quantity.

Take, for instance, evidence showing that medical doctors are just as susceptible to framing effects in medical treatment contexts as non-specialists. Does that indicate that doctors lack knowledge about the basics of their fields?

I think what this study suggests is that philosophical training is no more effective at de-biasing humans (at least for these particular biases) than a non-philosophical education. People have made claims to the contrary, and this is a useful corrective to that. The study doesn't show that philosophers are unaware of the basics of their field, or that philosophical training has nothing to offer in terms of expertise or problem-solving.

I thought the defining feature of being a p-zombie was acting as if they had consciousness while not "actually" having it

It's more than just a matter of behavior. P-zombies are supposed to be physically indistinguishable from human beings in every respect while still lacking consciousness.

I teach about 8000 miles away from upstate New York, I'm afraid.