= 27d567bc2d48d9f40e9ccc20ea370b15)
passive_fist comments on Open thread, Jan. 19 - Jan. 25, 2015 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (302)
I've been thinking about (and writing out my thoughts on) the real meaning of entropy in physics and how it relates to physical models. It should be obvious that entropy(physical system) isn't well-defined; only entropy(physical model, physical system) is defined. Here, 'physical model' might refer to something like the kinetic theory of gases, and 'physical system' would refer to, say, some volume of gas or a cup of tea. It's interesting to think about entropy from this perspective because it becomes related to the subjectivist interpretation of probability. I want to know if anyone knows of any links to similar ideas and thoughts.
There are approximations in figuring entropy and thermal statistics that may be wrong in very nearly immeasurable ways. The one that used to stick in my head was the calculation of the probability of all the gas in a volume showing up briefly in one-half the volume. Without doing math I figured it is actually much less than the classic calculated result, because the classic result assumes zero correlation between where any two molecules are, and once any kind of significant density difference exists between the two sides of the volume this will break.
But entropy is still real in the sense that it is "out there." An entire civilization is powered (and cooled) by thermodynamic engines, engines which quite predictable provide useful functionalities in ways predictable in detail from calculations of entropy.
A glass of hot water burns your skin even if you know the water and the skin's precise characterization in parameter space before they come in contact. Fast moving (relative to the skin) molecules of water break the bonds of some bits of skin they come in contact with. On the micro scale it may look like a scene from the matrix with a lot of slow moving machine gun bullets. The details of the destruction may be quite beautiful and "feel" cold, but essentially thanks to the central limit theorem, a whole lot of what happens will be predictable in a quite useful, and quite unavoidable way without having to appeal to the detail.
I think the only sense in which you can extract energy from water with a specially built machine that is custom designed for the current parameter space of the water, it is the machine which is at 0 or at least low temperature. And so the fact that useful energy can be extracted from the interaction of finite temperature water and a cold machine is totally consistent with entropy being real, thermal differences can power machines. And they do, witness the cars, trucks, airplanes and electric grid that are essential for our economy. The good news is you can get all the energy you need without knowing the detailed parameter space of the hot water, which is helpful because you then don't have to redesign your cold machine every few microseconds as you bring in new hot water to it from which to extract the next bit of energy.
Entropy is as real as energy whether it feels that way or not, and that is why machines work even when left unattended by consciousnesses to perceive their entropy and its flows.
I think you're getting several things wrong here.
The assumption of zero correlation is valid for ideal gases. It will not break if there is a density difference. We're talking about statistical correlation here.
"Entropy is in the mind" doesn't mean that you need consciousness for entropy to exist. All you need is a model of the world. Part of Jaynes' argument is that even though probabilities are subjective, entropy emerges as an objective value for a system (provided the model is given), since any rational Bayesian intelligence will arrive at the same value, given the same physical model and same information about the system.
Statistical independence means the chance that a molecule is at a particular spot depends not at all on where the other molecules are. Certainly if the molecules never hit each other, they only bounce off the walls of the volume, then this would be true as the molecules don't interract with each other so their probability of being one place or another is not changed by putting the other molecules anywhere, as long as they don't interract.
But molecules in a gas do interact they bounce off each other. Even an ideal gas. There is an average distance they travel before bouncing off another molecule called a mean free path. A situation where the mean free path is << size of volume is typical at STP.
Does this interaction break non-correlation? My intuition is that it does. But the thing I know for sure is that the only derivation I have ever seen for calculating the probability that all the gas is in 1/2 the volume was done with the assumptions of zero correlations, which we only know is the case for zero interaction, which is NOT an assumption required in the ideal gas models. And is certainly not true of any real gases.
This is as true for Entropy as it is for Energy. By this standard, Entropy and Energy are both in the mind, neither one is "realer" than the other.
Entropy is in the mind in exactly the same sense that probability is in the mind. See the relevant Sequence post if you don't know what that means.
The usual ideal gas model is that collisions are perfectly elastic, so even if you do factor in collisions they don't actually change anything. Interactions such as van der Waals have been factored in. The ideal gas approximation should be quite close to the actual value for gases like Helium.
Without a link! So I went to the sequences page in the wiki and the word entropy doesn't even appear on the page! Good job referring me there without a link.
Okay... Is that the same sense in which Energy is in the mind? Considering that this seems to be my claim that you are responding to, AND there is no reasonable way to get to a sequence page that corresponds to your not-quite-on-topic-but-not-quite-orthogonal response, that would be awfully nice to know.
Are you agreeing with me and amplifying, or disagreeing with me and explaining?
Probability is in the Mind.
THank you.
The thing that leaps out at me is that the rhetorical equation in that article between the sexiness of a woman being in the mind and the probability of two male children being in the mind is bogus.
I look at a woman and think she is sexy. If I assume the sexiness is in the woman, and that an alien creature would think she is sexy, or my wife would think she is sexy, because they would see the sexiness in her, then the article claims I have been guilty of the mind projection fallacy because the woman's sexiness is in my mind, not in the woman.
The article then proceeds to enumerate a few situations in which I am given incomplete information about reality and each different scenario corresponds to a different estimate that a person has two boy children.
BUT... it seems to me, and I would love to know if Eliezer himself would agree, even an alien given the same partial information would, if it were rational and intelligent, reach the same conclusions about the probabilities involved! So... probability, even Bayesian probability based on uncertainty is no more or less in my head than is 1+1=2. 1+1=2 whether I am an Alien mind or a Human mind, unlike that woman is sexy which may only be true in heterosexual male, homosexual female, and bisexual human minds, but not Alien minds.
But be that as it may, your comment still ignores the entire discussion, which is is Entropy and more or less "real" than Energy? The fact is that Aliens who had steam engines, internal combustion engines, gas turbines, and air conditioners would almost certainly have thermodynamics, and understand entropy, and agree with Humans on the laws of thermodynamics and the trajectories of entropy in the various machines.
If Bayesian probability is in the mind, and Entropy is in the mind, then they are like 1+1=2 being in the mind, things which would be in the mind of anything which we considered rational or intelligent. They would NOT be like "sexiness."
Probability depends on state of knowledge, which is a fact about your mind. Another agent with the same state of knowledge will assign the same probabilities. Another agent fully aware of your state of knowledge will be able to say what probabilities you should be assigning.
Sexiness depends on sexual preferences, which are a fact about your mind. Another agent with the same sexual preferences will assess sexiness the same way. Another agent fully aware of your sexual preferences will be able to say how sexy you will find someone.
I don't see that there's a big difference here. Except maybe for the fact that "states of knowledge", unlike "sexual preferences", can (in principle) be ranked: it's just plain better for your state of knowledge to be more accurate.
Well yes. Of course everything you can say about probability and sexiness you can say about Energy, Entropy, and Apple. That is, the estimate of the energy or entropy relationships in a particular machine or experimental scenario depend on the equations for energy and entropy, the measurements you make on the system to find the values of the elements that go into those equations. Any mind with the same information will reach the same conclusions about the Energy and Entropy that you would, assuming you are all doing it "right." Any intelligence desiring to transform heat producing processes into mechanical or electrical energy will even discover the same relationships to calculate energy and entropy as any other intelligence and will build similar machines, machines that would not be too hard for technologists from the other civilization to understand.
Even determining if something is an apple. Any set of intelligences that know the definitions of apples common among humans on earth will be able to look at various earth objects and determine which of them are apples, which are not, and which are borderline. (I'm imagining there must be some "crabapples" that are marginally edible that people would argue over whether to call apples or not, as well as a hybrid between an apple and a pear that some would call an apple and some wouldn't).
So "Apple" "Sexy" "Entropy" "Energy" and "Probability" are all EQUALLY in the mind of the intelligence dealing with them.
If you check, you will see this discussion started by suggesting that Energy was "realer" than Entropy. That Entropy was more like Probability and Sexiness, and thus, not as real, while Energy was somehow actually "out there" and therefore realer.
My contention is that all these terms are equally as much in the mind as in reality, that as you say any intelligence who knows the definitions will come up with the same conclusions about any given real situation, and that there is no distinction in "realness" between Energy and Entropy, no distinction between these and Apple, and indeed no distinction between any of these and "Bayesian Probability." That pointing out that features of the map are not features of the territory does NOT allow you to privilege some descriptive terms as being "really" part of the territory after all, even though they are words that can and should obviously be written down on the map.
If you are going to explicate further, please state whether you agree or disagree that some of these terms are realer than others, as this is how the thread started and open-ended explication is ambiguous.
They don't change ANYTHING? Suppose I start with a gas of molecules all moving at the same speed but in different directions, and they have elastic collisions off the walls of the volume. If they do not collide with each other, they never "thermalize," their speeds stay the same forever as they bounce off the walls but not off each other. But if they do bounce off each other, the velocity distribution does become thermalized by their collisions, even when these collisions are elastic. So collisions don't chage ANYTHING? They change the distribution of velocities to a thermal one, which seems to me to be something.
So even if an ideal gas maintained perfect decorrelation between molecule positions in an ideal gas with collisions, which I do not think you can demonstrate (and appealing to an unlinked sequence does not count as a demonstration), you would still have to face the fact that an actual gas like Helium would be "quite close" to uncorrelated, which is another way of saying... correlated.
Both the "entropy is in the mind" and "entropy is real" explanations seem plausible to me (well, I am not a physicist, so anything may seem plausible), so now that I think about it... maybe the problem is that even if we would be able to know a lot of stuff, we might still be limited in ways we can use this knowledge. And the knowledge you can't realistically use, it's as if you wouldn't even have it.
So, in theory, there could be a microscopical demon able to travel between molecules of boiling water without hitting any of them -- so from the demon's point of view, there is nothing hot about that water -- the problem is that we cannot do this with real stuff; not even with nanomachines probably. Calculating the path for the nanomachine would be computationally too expensive, and it is probably too big to fit between the molecules. So the fact is that a few molecules are going to hit that nanomachine, or any greater object, anyway.
Or perhaps we could avoid the whole paradox by saying: "Actually no, you cannot have the knowledge about all molecules of the boiling water. How specifically would you get it, and how specifically would you keep it up to date?"
This is pretty much it, and it's a really subtle detail that causes a lot of confusion. This is why the real problem with Maxwell's demon isn't how you obtain the information, it's how you store the information, as Landauer showed. To extract useful work you have to erase bits ('forget' knowledge) at some point. And this raises the entropy.
I made a post about this a month or so ago. Yay!
That's pretty much exactly what I had in mind. Thanks.
In this way entropy is not much different from energy. The latter also depends on the model as much as on the physical system itself.
I'm going to disagree with you here. Not that energy doesn't depend on our models. It just depends on them in a very different way. The entropy of a physical system is the Shannon entropy of its distribution of 'microstates'. But there is no distribution of microstates 'out there'. It's a construction that purely exists in our models. Whereas energy does exist 'out there'. It's true that no absolute value can be given for energy and that it's relative, but in a way energy is far more 'real' than entropy.
Potential energy depends on what you set the zero level to, but I agree that this is very different than entropy. In particular, the difference in energy between two systems is well-defined.
"Out there" are fields, particles, interacting, moving, bumping into each other, turning into each other. Energy is a convenient description of some part of this process in many models. Just like with Jaynes' entropy, knowing more about the system changes its energy. For example, just like knowing about isotopes affects the calculated entropy of a mixed system, knowing about nuclear forces changes the calculated potential energy of the system.
I agree with passive_fist, and my argument hasn't changed since last time.
If we learn that energy changes in some process, then we are wrong about the laws that the system is obeying. If we learn that entropy goes down, then we can still be right about the physical laws, as Jaynes shows.
Another way: if we know the laws, then energy is a function of the individual microstate and nothing else, while entropy is a function of our probability distribution over the microstates and nothing else.
I agree that it feels different. It certainly does to me. Energy feels real, while entropy feels like an abstraction. A rock falling on one's head is a clear manifestation of its potential (turned kinetic) energy, while getting burned by a hot beverage does not feel like a manifestation of the entropy increase. it feels like the beverage's temperature is to blame. On the other hand, if we knew precisely the state of every water molecule in the cup, would we still get burned? The answer is not at all obvious to me. Passive_fist claims that the cup would appear to be a absolute zero then:
I do not know enough stat mech to assess this claim, but it seems wrong to me, unless the claim is that we cannot know the state of the system unless it's already at absolute zero to begin with. I suppose a toy model with only a few particles present might shed some light on the issue. Or a link to where the issue is discussed.
An easy toy system is a collection of perfect billiard balls on a perfect pool table, that is, one without rolling friction and where all collisions conserve energy. For a few billiard balls it would be quite easy to extract all of their energy as work if you know their initial positions and velocities. There are plenty of ways to do it, and it's fun to think of them. This means they are at 0 temperature.
If you don't know the microstate, but you do know the sum of the square of their velocities, which is a constant in all collisions, you can still tell some things about the process. For instance, you can predict the average number of collisions with one wall and the corresponding energy, related to the pressure. If you stick your hand on the table for five seconds, what is the chance you get hit by a ball moving faster than some value that will cause pain? All these things are probabilistic.
In the limit of tiny billiard balls compared to pool table size, this is the ideal gas.
If you know precisely the state of every water molecule in the system, there's no need for your finger to get burned. Just touch your finger to the cup whenever a slow-moving molecule is approaching, and remove it whenever a fast-moving molecule is approaching (Maxwell's demon).
Right, supposing you can have a macroscopic Maxwell's demon. So the claim is not that it is necessarily at absolute zero, but that it does not have a well-defined temperature, because you can choose it to behave (with respect to your finger) as if it were at any temperature you like. Is this what you are saying?
Well, no.
Temperature is the thermodynamic quantity that is shared by systems in equilibrium. "Cup of tea + information about all the molecules in the cup of tea" is in thermodynamic equilibrium with "Ice cube + kinetic energy (e.g. electricity)", in that you can arrange a system where the two are in contact but do not exchange any net energy.
Note that it is NOT in thermodynamic equilibrium with anything hotter than an ice cube, as Eliezer described in spxtr's linked article: http://lesswrong.com/lw/o5/the_second_law_of_thermodynamics_and_engines_of/
Basically, if you, say, try to use the information about the water and a Demon to put the system in thermal equilibrium with some warm water and electricity, you'll either be prevented by conservation of energy or you'll wind up not using all the information at your disposal. And if you don't use the information it's as if you didn't have it.
The salient point is that the system is not in thermal equilibrium with anything 'warmer' than "Ice cube + free energy."
If you know everything about the cup of tea, it really is at absolute zero, in the realest sense you could imagine.
Hm. I have to think more about this.
Expanding on the billiard ball example: lets say one part of the wall of the pool table adds some noise to the trajectory of the balls that bounce off of that spot, but doesn't sap energy from them on average. After a while we won't know the exact positions of the balls at an arbitrary time given only their initial positions and momenta. That is, entropy has entered our system through that part of the wall. I know this language makes it sound like entropy is in the system, flowing about, but if we knew the exact shape of the wall at that spot then it wouldn't happen.
Even with this entropy entering our system, the energy remains constant. This is why total energy is a wonderful macrovariable for this system. Systems where this works are usually easily solved as a microcanonical ensemble. If, instead, that wall spot was at a fixed temperature, we would use the canonical ensemble.
Again, this is very different from the situation with entropy. I think you're confusing two meanings of the word 'model'. It's one thing to have an incomplete description of the physics of the system (for instance, lacking nuclear forces, as you describe). It's another to lack knowledge about the internal microstates of the system, even if all relevant physics are known. (In the statistics view, these two meanings are analogous to the 'model' and the 'parameters', respectively). Entropy measures the uncertainty in the distribution of the parameters. It measures something about our information about the system. The most vivid demonstration of this is that entropy changes the more you know about the parameters (microstates) of the system. In the limit of perfect microstate knowledge, the system has zero entropy and is at absolute zero. But energy (relative to ground state) doesn't change no matter how much information you gain about a system's internal microstates.
I understand what you are saying, but I am not convinced that there is a big difference.
How would you change this uncertainty without disturbing the system?
How would you gain this information without disturbing the system (and hence changing its energy)?
EDIT: see also my reply to spxtr.
You have to define what 'disturbing the system' means. This is just the classical Maxwell's demon question, and you can most definitely change this uncertainty without changing the thermodynamics of the system. Look at http://en.wikipedia.org/wiki/Maxwell%27s_demon#Criticism_and_development
Especially, the paragraph about Landauer's work is relevant (and the cited Scientific American article is also interesting).
If you haven't already read Jaynes derivation of maxent, and the further derivation of much of statistical mechanics from those principles, that would be a good place to start.