Here's an interesting but very old paper - two theories of Heat Control.
It discusses mental models of home heating systems (thermostats) non-experts use.
These models tend to be extremely wrong from theoretical perspective, but surprisingly useful in practice.
The findings are applicable to a much wider range of subjects than just thermostats, and have certain epistemological significance, especially with regard to compartmentalization.
This was a fascinating article!
The part I found especially poignant was the house-dwellers who incorrectly tried to correct their spouses, e.g. "no it actually wastes energy to turn down the heat at night".
The general phenomenon seems to exactly parallel the article Medieval Ballistics and Experiement here on Less Wrong, where lucidfox asked how anyone could ever have believed in the Aristotelian impulse theory when it predicts trajectories that look nothing like a parabola -- and then RichardKennaway pointed out that in the presence of air resistance, cannonball trajectories match the Aristotelian model quite closely and look nothing like parabolas.
There really seems to be two modes of thinking here: one which notices that the house gets colder when the outside temperature drops, and concludes that you'd better turn up the thermostat, and one which notes that the thermostat controls a set point, and concludes that the house could not possibly have gotten colder.
It's clear which way of thinking is more rational, but I certainly find myself working in the theory-driven mode a lot, and I think this may be a problem particularly with the kind of people who like Less Wrong (since we are comfortable thinking about elaborate intellectual theories). It's definitely related to Noticing Confusion.
The part I found especially poignant was the house-dwellers who incorrectly tried to correct their spouses, e.g. "no it actually wastes energy to turn down the heat at night".
I wonder how much of that is motivated by people who see their spouse's complaints of being too cold at night as whining or frivolous, but will cooperate if there is a "technical" justification.
I don't think the blame belongs on using "theory-driven mode." The problem is not thinking like reality—for example, most people who think "the house could not possibly have gotten colder" are probably at least somewhat aware that their house's walls don't comprise a perfect Dewar flask, but if "walls" don't happen to be in the same mental filing bin as "thermostats" they are automatically assumed to be irrelevant.
Another area where in recent years I've gained some respect for folk theories is economics. Now of course, folk economics is even more full of falsities and fallacies than folk physics, and a basic education in economics will rectify a lot of these. However, at the same time, "scientific" economics has its own share of awful fallacies that have become a firmly established part of its conventional wisdom, and are happily parroted by legions of economists eager to scoff at anyone who questions them as an ignoramus or crackpot. On some issues where I would myself scoff at folk theories a few years ago, I have come to realize that they may actually be closer to reality than the overconfident assertions of economists.
In particular, if your folk economics intuition tells you that you're getting shafted but learned economists nevertheless keep assuring you that it's all in your own best interest, there's a pretty good chance that your intuition is pointing in the right direction while their assurances are just clever ideological propaganda.
I've actually had this idea for a while, but I can rarely bring myself to find enough time for a proper article rather than just writing short comments. Thanks for expressing your interest - it has certainly given me additional motivation.
In the meantime, if you haven't seen them already, you can find discussions where I question some established wisdom in economics here and here.
just saw this point repeated 10 minutes ago.
on Tabbarok[http://marginalrevolution.com/marginalrevolution/2011/03/privatize-the-spectrum.html] saying
"But even if consumer prices did rise the merger is probably still a good idea. It’s long been known that even small cost savings can outweigh losses to consumers from a price increase"
to which Tomasz Wegrzanowski replied: Williamson trade-off model is exactly why people hate economists.
Not once in history of the universe has there been anything benefiting the rich and powerful, for which some economists didn’t make a model showing it’s really in everybody’s best interest.
Cost savings are bullshit, price increases are real.
Funny, just a few weeks ago I made a very similar point in a comment here, although I've never seen that paper until now.
Something rather curious about the universe is that there are well-defined, exploitable regularities that supervene upon the most basic elements of the physical universe. We might expect the most fundamental level of reality to be defined by relatively simple, predictable laws because of the probability penalties associated with complexity. But why should we expect that physics to result in such helpful regularities on the macro level?
Are there only certain possible physical universes that produce useful macro regularities? The anthropic principle might explain some of the regularities we see- there have to be some for macro-sized organisms to exist- perhaps this is why the social sciences are so hard, our universe hasn't been anthropically selected for regularities in the behavior of societies like it has for physics and chemistry.
Conversely, perhaps the relationship is just mathematic and macro-regularities exist for most fundamentally simple systems. Maybe The Game of Life is relevant here? Are all macro-regularities just statistical?
"Macro regularities" often have mathematical explanations
I think this is a better justification for Occam's razor then the one based on Solomonoff induction.
I am not convinced. Kempton asserts that the technically incorrect valve theory is exactly as usefull as the correct feedback theory, however he does not satisfactorily back up the assertion. For example,
A second issue is whether a higher setting provides faster warmup. The feedback theory denies faster warmup because the furnace runs at a constant rate. However, due to human comfort factors and characteristics of interacting systems, a person entering a cold house from outside will not feel warm when the air first reaches the correct temperature.’ Thus, greater comfort would be realized if the thermostat were set high, to raise the air temperature above normal initially, then returned to the normal setting. Again, the correct action would logically follow from the valve model, but not from the feedback model.
is a fallacious argument. It assumes that a person believing the feedback theory will be unwilling to raise the temperature when feeling cold after coming from outside. Why should it be?
Also the wrong conclusion that energy is not saved by turning the termostat down temporarily is neither implied by the feedback theory nor prevented by the valve theory ("but you will need to set the burner faster in the morning to heat everything back to 65!").
I take it that by "extremely wrong from a theoretical perspective" you mean that they contradict the more general theories physicists use to explain the universe? Or something like that?
Yes, thermostats are simple on/off devices, with on/off switch based on temperature setting and sensed current temperature ('threshold theory').
Folk theory held by about a third of the public (called "valve theory' in paper) states that they regulate heating in a way linearly proportional to current setting, without any temperature sensing.
Valve theory is trivially demonstrable to be wrong, and yet it has about the same number of successful operational predictions as threshold theory.
It turns out that threshold theory needs a lot of extra complexity until it definitely beats valve theory.
Valve theory is trivially demonstrable to be wrong
While this is true of thermostats as such, not all heating controls are thermostats. My parents' house had wall-mounted ovens which worked like a kitchen stove: They put out heat proportional to the setting. As an aside, central heating seems to be rare in Norway; instead each room has its own heating. I'm not sure why this should be so.
Michael Vassar once mentioned the example of some tribe of people who believed in faeries, because some of them could see hints of them from the corner of their eyes. Turns out that they had some sort of a degenerative eye disease that caused them to see things.
IIRC, he commented that while the tribe's theory about faeries was wrong, they were still onto something. They didn't just imagine that they were seeing things. They were seeing things, and constructed the best explanation they could given the information available to them. Even such a blatantly incorrect explanation was still correlated with the truth on some level.
All models are false, but some are useful. (Even if the faerie explanation probably wasn't very useful.)
Theories in general can be useful even when they're entirely wrong. There are entire fields dedicated to this (with or without the practitioners' knowledge of this fact), in fact most "high level" subjects have to be theoretically false because to be theoretically correct is just analytically and/or computationally impossible. I would put forward economics and social science in general as examples of this.
A broken clock is right twice per day. If value theory is incidentally correct, it doesn't make folk theories valuable on the margins - unless of course, if people who hold folk theories do consistently better than rationalists, but then I'd question the rationalist label.
All people hold folk theories. The question is whether, given the set of folk theories you unknowingly hold, correcting an interacting folk theory to a scientifically sound one will improve or degrade your overall performance.
The title mislead me: a useful theory is bound to make accurate predictions. Therefore it is not entirely wrong.
"Macro regularities" often have mathematical explanations
Tao, Draft article on universality
I think this is a better justification for Occam's razor then the one based on Solomonoff induction.