= 27d567bc2d48d9f40e9ccc20ea370b15)
passive_fist comments on Open thread, Jan. 19 - Jan. 25, 2015 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (302)
I think you're getting several things wrong here.
The assumption of zero correlation is valid for ideal gases. It will not break if there is a density difference. We're talking about statistical correlation here.
"Entropy is in the mind" doesn't mean that you need consciousness for entropy to exist. All you need is a model of the world. Part of Jaynes' argument is that even though probabilities are subjective, entropy emerges as an objective value for a system (provided the model is given), since any rational Bayesian intelligence will arrive at the same value, given the same physical model and same information about the system.
Statistical independence means the chance that a molecule is at a particular spot depends not at all on where the other molecules are. Certainly if the molecules never hit each other, they only bounce off the walls of the volume, then this would be true as the molecules don't interract with each other so their probability of being one place or another is not changed by putting the other molecules anywhere, as long as they don't interract.
But molecules in a gas do interact they bounce off each other. Even an ideal gas. There is an average distance they travel before bouncing off another molecule called a mean free path. A situation where the mean free path is << size of volume is typical at STP.
Does this interaction break non-correlation? My intuition is that it does. But the thing I know for sure is that the only derivation I have ever seen for calculating the probability that all the gas is in 1/2 the volume was done with the assumptions of zero correlations, which we only know is the case for zero interaction, which is NOT an assumption required in the ideal gas models. And is certainly not true of any real gases.
This is as true for Entropy as it is for Energy. By this standard, Entropy and Energy are both in the mind, neither one is "realer" than the other.
Entropy is in the mind in exactly the same sense that probability is in the mind. See the relevant Sequence post if you don't know what that means.
The usual ideal gas model is that collisions are perfectly elastic, so even if you do factor in collisions they don't actually change anything. Interactions such as van der Waals have been factored in. The ideal gas approximation should be quite close to the actual value for gases like Helium.
Without a link! So I went to the sequences page in the wiki and the word entropy doesn't even appear on the page! Good job referring me there without a link.
Okay... Is that the same sense in which Energy is in the mind? Considering that this seems to be my claim that you are responding to, AND there is no reasonable way to get to a sequence page that corresponds to your not-quite-on-topic-but-not-quite-orthogonal response, that would be awfully nice to know.
Are you agreeing with me and amplifying, or disagreeing with me and explaining?
Probability is in the Mind.
THank you.
The thing that leaps out at me is that the rhetorical equation in that article between the sexiness of a woman being in the mind and the probability of two male children being in the mind is bogus.
I look at a woman and think she is sexy. If I assume the sexiness is in the woman, and that an alien creature would think she is sexy, or my wife would think she is sexy, because they would see the sexiness in her, then the article claims I have been guilty of the mind projection fallacy because the woman's sexiness is in my mind, not in the woman.
The article then proceeds to enumerate a few situations in which I am given incomplete information about reality and each different scenario corresponds to a different estimate that a person has two boy children.
BUT... it seems to me, and I would love to know if Eliezer himself would agree, even an alien given the same partial information would, if it were rational and intelligent, reach the same conclusions about the probabilities involved! So... probability, even Bayesian probability based on uncertainty is no more or less in my head than is 1+1=2. 1+1=2 whether I am an Alien mind or a Human mind, unlike that woman is sexy which may only be true in heterosexual male, homosexual female, and bisexual human minds, but not Alien minds.
But be that as it may, your comment still ignores the entire discussion, which is is Entropy and more or less "real" than Energy? The fact is that Aliens who had steam engines, internal combustion engines, gas turbines, and air conditioners would almost certainly have thermodynamics, and understand entropy, and agree with Humans on the laws of thermodynamics and the trajectories of entropy in the various machines.
If Bayesian probability is in the mind, and Entropy is in the mind, then they are like 1+1=2 being in the mind, things which would be in the mind of anything which we considered rational or intelligent. They would NOT be like "sexiness."
Probability depends on state of knowledge, which is a fact about your mind. Another agent with the same state of knowledge will assign the same probabilities. Another agent fully aware of your state of knowledge will be able to say what probabilities you should be assigning.
Sexiness depends on sexual preferences, which are a fact about your mind. Another agent with the same sexual preferences will assess sexiness the same way. Another agent fully aware of your sexual preferences will be able to say how sexy you will find someone.
I don't see that there's a big difference here. Except maybe for the fact that "states of knowledge", unlike "sexual preferences", can (in principle) be ranked: it's just plain better for your state of knowledge to be more accurate.
Well yes. Of course everything you can say about probability and sexiness you can say about Energy, Entropy, and Apple. That is, the estimate of the energy or entropy relationships in a particular machine or experimental scenario depend on the equations for energy and entropy, the measurements you make on the system to find the values of the elements that go into those equations. Any mind with the same information will reach the same conclusions about the Energy and Entropy that you would, assuming you are all doing it "right." Any intelligence desiring to transform heat producing processes into mechanical or electrical energy will even discover the same relationships to calculate energy and entropy as any other intelligence and will build similar machines, machines that would not be too hard for technologists from the other civilization to understand.
Even determining if something is an apple. Any set of intelligences that know the definitions of apples common among humans on earth will be able to look at various earth objects and determine which of them are apples, which are not, and which are borderline. (I'm imagining there must be some "crabapples" that are marginally edible that people would argue over whether to call apples or not, as well as a hybrid between an apple and a pear that some would call an apple and some wouldn't).
So "Apple" "Sexy" "Entropy" "Energy" and "Probability" are all EQUALLY in the mind of the intelligence dealing with them.
If you check, you will see this discussion started by suggesting that Energy was "realer" than Entropy. That Entropy was more like Probability and Sexiness, and thus, not as real, while Energy was somehow actually "out there" and therefore realer.
My contention is that all these terms are equally as much in the mind as in reality, that as you say any intelligence who knows the definitions will come up with the same conclusions about any given real situation, and that there is no distinction in "realness" between Energy and Entropy, no distinction between these and Apple, and indeed no distinction between any of these and "Bayesian Probability." That pointing out that features of the map are not features of the territory does NOT allow you to privilege some descriptive terms as being "really" part of the territory after all, even though they are words that can and should obviously be written down on the map.
If you are going to explicate further, please state whether you agree or disagree that some of these terms are realer than others, as this is how the thread started and open-ended explication is ambiguous.
Anything at all is "in the mind" in the sense that different people might for whatever reason choose to define the words differently. Because this applies to everything, it's not terribly interesting and usually we don't bother to state it. "Apple" and "energy" are "in the mind" in this sense.
But (in principle) someone could give you a definition of "energy" that makes no reference to your opinions or feelings or health or anything else about you, and be confident that you or anyone else could use that definition to evaluate the "energy" of a wide variety of systems and all converge on the same answer as your knowledge and skill grows.
"Entropy" (in the "log of number of possibilities" sense) and "probability" are "in the mind" in another, stronger sense. A good, universally applicable definition of "probability" needs to take into account what the person whose probability it is already knows. Of course one can define "probability, given everything there is to know about mwengler's background information on such-and-such an occasion" and everyone will (in principle) agree about that, but it's an interesting figure primarily for mwengler on that occasion and not really for anyone else. (Unlike the situation for "energy".) And presumably it's true that for all (reasonable) agents, as their knowledge and skill grow, they will converge on the same probability-relative-to-that-knowledge for any given proposition -- but frequently that won't in any useful sense be "the probability that it's true", it'll be either 0 or 1 depending on whether the proposition turns out to be true or false. For propositions about the future (assuming that we fix when the probability is evaluated) is might end up being something neither 0 nor 1 for quantum-mechanical reasons, but that's a special case.
Similarly, entropy in the "log of number of possibilities" sense is meaningful only for an agent with given knowledge. (There is probably a reasonably respectable way of saying "relative to what one could find out by macroscopic observation, not examining the system too closely", and I think that's often what "entropy" is taken to mean, and that's fine. But that isn't quite the meaning that's being advocated for in this post.)
Sexiness is "in the mind" in an even stronger sense, I suppose. But I think it's reasonable to say that on the scale from "energy" to "sexiness", probability is a fair fraction of the way towards "sexiness".
Aha! So it would seem the original sense that "Energy" is "realer" (more like Apple) than Entropy is because Entropy is associated with Probability, and Bayesian Probability, the local favorite, is more in the mind than other things because its accurate estimation requires information about the state of knowledge of the person estimating it.
So it is proposed there is a spectrum "in the mind" (or dependent on other things in the mind as well as things in the real world) to "real" (or in the mind only to the extent that it depends on definitions all minds would tend to share).
We have Sexiness is in the mind, and thinking it is in reality is a projection fallacy. At the other end of the spectrum, we have things like Energy and Apple which are barely in the mind, which depend in straightforward ways on straightforward observations of reality, and would be agreed upon by all minds that agreed on the definitions.
And then we have probability. Frequentist definitions of probability are intended to be like Energy and Apple, relatively straightforward to calculate from easy to define observations.
But then we have Bayesian probability, which is a statement which links our current knowledge of various details with our estimate of probability. So considering that different minds can have different bits of other knowledge in them than other minds, different minds can "correctly" estimate different probabilities for the same occurrences, just as different minds can estimate different amounts of sexiness for the same creatures, depending on the species and genders of the different minds.
And then we have Entropy. And somebody defines Entropy as the "log of number of possibilities" and possibilities are like probabilities, and we prefer Bayesian "in the mind" probability to Frequentist "in reality" definitions of probability. And so some people think Entropy might be in the mind like Bayesian probability and sexiness, rather than in reality like Energy and Apple.
Good summary? I know! It is!
So here is the thing. Entropy in physics is defined as
That is, the entropy is very deterministically added to a system by heating the system with an unambiguously determined amount of energy dQrev, and dividing that amount of energy by an unambiguously determined temperature of the system. That sure doesn't look like it has any probabilities in it. So THIS definition of Entropy is as real as Energy and Apple. And this is where I have been coming from. You me and an alien from Alpha Centauri can all learn the thermodynamics required to build steam engines, internal combustion engines, and refrigerators, and we will all find the same definitions for Energy and Entropy (however we might name them), and we will all determine the same trajectories in time and space for Energies and Entropies for any given thermodynamic system we analyze. Entropy defined this way is as real as Energy and Apples.
But what about that "log of number of possibilities" thing? Well a more pedantic answer would be, that the number of possibilities has nothing to do with probabilities. I have a multiparticle state with known physics of interactions. Its state when first specified, the possibility it initially occupies, has a certain amount of energy associated with it. The energy (we consider only closed systems for now) will stay constant, and EVERY possible point in parameter space which has the SAME energy as our initial state shows up on our list of possibilities for the system, and every point in parameter space with a DIFFERENT energy than our initial state is NOT a possible state of this system.
So counting the possibilities does NOT seem to involve any Bayesian probabilities at all. You, me, and an alien from Alpha Centauri who all look at the same system all come up with the same Entropy curves, just as we all come up with the same energy curves.
But perhaps I can do better than this. Tie this in to the intuition that entropy has something to do with probabilities. And I can.
The probabilities that entropy has to do with are FREQUENTIST probabilities. Enumerations of the physically possible states of the system. We could estimate them mathematically by hypothesizing a map of the system called parameter space, or we could take 10^30 snapshots of the physical system spread out over many millenia and just observe all the states the system gets into. Of course this second is impractical, but when has impractical ever stopped a lesswrong discussion?
So the real reason Entropy, Energy and Apple are "real" even though Bayesian Probability like Sexiness is "in the mind" is because Entropy is unambiguously defined for physical systems in terms of other unambiguous physical quantities "Energy" and "Temperature." (BTW, Temperature is Average Kinetic Energy of the particles, not some ooky "in the mind" mind thing. Or for simplicity, define temperature as what the thermometer tells you.)
And to the extent you love Bayesian probability so much that you want somehow to interpret a list of states in parameter space that all have the same energy as somehow "in the mind," you just need to realize that a frequentist interpretation of probability is more appropriate for any discussion of entropy than is a bayesian one: we use entropy to calculate what systems we know "enough" about will do, not to estimate how different people in different states of ignorance will bet on what they will do. If we enumerate the states wrong we get the wrong entropy and our engine doesn't work the way we said it would, we don't get to be right, in the subjective sense that our estimate was as good as it could be given what we knew.
I hope this is clear enough to be meaningful to anybody following this topic. It sure explains to me what has been going on.
They don't change ANYTHING? Suppose I start with a gas of molecules all moving at the same speed but in different directions, and they have elastic collisions off the walls of the volume. If they do not collide with each other, they never "thermalize," their speeds stay the same forever as they bounce off the walls but not off each other. But if they do bounce off each other, the velocity distribution does become thermalized by their collisions, even when these collisions are elastic. So collisions don't chage ANYTHING? They change the distribution of velocities to a thermal one, which seems to me to be something.
So even if an ideal gas maintained perfect decorrelation between molecule positions in an ideal gas with collisions, which I do not think you can demonstrate (and appealing to an unlinked sequence does not count as a demonstration), you would still have to face the fact that an actual gas like Helium would be "quite close" to uncorrelated, which is another way of saying... correlated.