= 27d567bc2d48d9f40e9ccc20ea370b15)
Perplexed comments on The Irrationality Game - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (910)
It is risky to deprecate something as "meaningless" - a ritual, a practice, a word, an idiom. Risky because the actual meaning may be something very different than you imagine. That seems to be the case here with attaching numbers to subjective probabilities.
The meaning of attaching a number to something lies in how that number may be used to generate a second number that can then be attached to something else. There is no point in providing a number to associate with the variable 'm' (i.e. that number is meaningless) unless you simultaneously provide a number to associate with the variable 'f' and then plug both into "f=ma" to generate a third number to associate with the variable 'a', an number which you can test empirically.
Similarly, a single isolated subjective probability estimate may seem somewhat meaningless in isolation, but if you place it into a context with enough related subjective probability estimates and empirically measured frequencies, then all those probabilities and frequencies can be combined and compared using the standard formulas of Bayesian probability:
So, if you want to deprecate as "meaningless" my estimate that the Democrats have a 40% chance to maintain their House majority in the next election, go ahead. But you cannot then also deprecate my estimate that the Republicans have a 70% of reaching a House majority. Because the conjunction of those two probability estimates is not meaningless. It is quite respectably false.
I think you're not drawing a clear enough distinction between two different things, namely the mathematical relationships between numbers, and the correspondence between numbers and reality.
If you ask an astronomer what is the mass of some asteroid, he will presumably give you a number with a few significant digits and and uncertainty interval. If you ask him to justify this number, he will be able to point to some observations that are incompatible with the assumption that the mass is outside this interval, which follows from a mathematical argument based on our best knowledge of physics. If you ask for more significant digits, he will say that we don't know (and that beyond a certain accuracy, the question doesn't even make sense, since it's constantly losing and gathering small bits of mass). That's what it means for a number to be rigorously justified.
But now imagine that I make an uneducated guess of how heavy this asteroid might be, based on no actual astronomical observation. I do of course know that it must be heavier than a few tons or otherwise it wouldn't be noticeable from Earth as an identifiable object, and that it must be lighter than 10^20 or so tons since that's roughly the range where smaller planets are, but it's clearly nonsensical for me to express that guess with even one digit of precision. Yet I could insist on a precise guess, and claim that it's "meaningful" in a way analogous to your above justification of subjective probability estimates, by deriving various mathematical and physical implications of this fact. If you deprecate my claim that its mass is 4.5237 x 10^15kg, then you cannot also deprecate my claim that it is a sphere of radius 1km and average density 1000kg/m^3, since the conjunction of these claims is by the sheer force of mathematics false.
Therefore, I don't see how you can argue that a number is meaningful by merely noting its relationships with other numbers that follow from pure mathematics. Or am I missing something with this analogy?
The only thing you are missing is the first paragraph of my reply. Just because something doesn't have the kind of meaning you think it ought to have (by virtue of being a number, for example) that doesn't justify your claim that it is meaningless.
Subjective probabilities of isolated propositions don't have the kind of meaning you want numbers to have. But they have exactly the kind of meaning I want them to have - specifically they can be used in computations that produce consistent results.
Do you think that the digits of pi beyond the first half dozen are also meaningless?
Perplexed:
Fair enough, but I still don't see how this solves the problem of the correspondence between numbers and reality. Any number can be used in computations that produce consistent results if you just start plugging it into formulas derived from some consistent mathematical theory. It is when the numbers are used as basis for claims about the real, physical world that I insist on an explanation of how exactly they are derived and how their claimed correspondence with reality is justified.
The digits of pi are an artifact of pure mathematics, so I don't think it's a good analogy for what we're talking about. Once you've built up enough mathematics to define lengths of curves in Euclidean geometry, the ratio between the circumference and diameter of a circle follows by pure logic. Any suitable analogy for what we're talking about must encompass empirical knowledge, and claims which can be falsified by empirical observations.
It doesn't have to. That is a problem you made up. Other people don't have to buy in to your view on the proper relationship between numbers and physical reality.
My viewpoint on numbers is somewhere between platonism and formalism. I think that the meaning of a number is a particular structure in my mind. If I have an axiom system that is categorical (and, of course, usually I don't) then that picture in my mind can be made inter-subjective in that someone who also accepts those axioms can build an isomorphic structure in their own mind. The real world has absolutely nothing to do with Tarski's semantics - which is where I look to find out what the "meaning" of a number is.
Your complaint that subjective probabilities have no meaning is very much like the complaint of a new convert to atheism who laments that without God, life has no meaning. My advice: stop telling other people what the word "meaning" should mean.
However, if you really need some kind of affirmation, then I will provide some. I agree with you that the numbers used in subjective probabilities are less, ... what is the right word, ... less empirical than are the numbers you usually find in science classes. Does that make you feel better?
Perplexed:
You probably wouldn't buy that same argument if it came from a numerologist, though. I don't think I hold any unusual and exotic views on this relationship, and in fact, I don't think I have made any philosophical assumptions in this discussion beyond the basic common-sense observation that if you want to use numbers to talk about the real world, they should have a clear connection with something that can be measured or counted to make any sense. I don't see any relevance of these (otherwise highly interesting) deep questions of the philosophy of math for any of my arguments.
There is nothing philosophically wrong with your position except your choice of the word "meaningless" as an epithet for the use of numbers which cannot be empirically justified. Your choice of that word is pretty much the only reason I am disagreeing with you.
Given your position on the meaninglessness of assigning a numerical probability value to a vague feeling of how likely something is, how would you decide whether you were being offered good odds if offered a bet? If you're not in the habit of accepting bets, how do you think someone who does this for a living (a bookie for example) should go about deciding on what odds to assign to a given bet?
mattnewport:
In reality, it is rational to bet only with people over whom you have superior relevant knowledge, or with someone who is suffering from an evident failure of common sense. Otherwise, betting is just gambling (which of course can be worthwhile for fun or signaling value). Look at the stock market: it's pure gambling, unless you have insider knowledge or vastly higher expertise than the average investor.
This is the basic reason why I consider the emphasis on subjective Bayesian probabilities that is so popular here misguided. In technical problems where probability calculations can be helpful, the experts in the field already know how to use them. On the other hand, for the great majority of the relevant beliefs and conclusions you'll form in life, they offer nothing useful beyond what your vague common sense is already telling you. If you start taking them too seriously, it's easy to start fooling yourself that your thinking is more accurate and precise than it really is, and if you start actually betting on them, you'll be just gambling.
I'm not familiar with the details of this business, but from what I understand, bookmakers work in such a way that they're guaranteed to make a profit no matter what happens. Effectively, they exploit the inconsistencies between different people's estimates of what the favorable odds are. (If there are bookmakers who stake their profit on some particular outcome, then I'm sure that they have insider knowledge if they can stay profitable.) Now of course, the trick is to come up with a book that is both profitable and offers odds that will sell well, but here we get into the fuzzy art of exploiting people's biases for profit.
You're dodging the question. What if the odds arose from a natural process, so that there isn't a person on the other side of the bet to compare your state of knowledge against?
I think this is right. The idea that you would be betting against another person is inessential, an unfortunate distraction arising from the choice of thought experiment. Admittedly it's a natural way to understand the thought experiment, but it's inessential. The experiment could be revised to exlude it. In fact every moment we make decisions whose outcomes depend on things we don't know, and in making those decisions we are therefore in effect gambling. We are surrounded by risks, and our decisions reveal our assessment of those risks.
jimrandomh:
Maybe it's my failure of English comprehension (I'm not a native speaker, as you might guess from my frequent grammatical errors), but when I read the phrase "being offered good odds if offered a bet," I understood it as asking about a bet with opponents who stand to lose if my guess is right. So, honestly, I wasn't dodging the question.
But to answer your question, it depends on the concrete case. Some natural processes can be approximated with models that yield useful probability estimates, and faced with some such process, I would of course try to use the best scientific knowledge available to calculate the odds if the stakes are high enough to justify the effort. When this is not possible, however, the only honest answer is that my decision would be guided by whatever intuitive feeling my brain happens to produce after some common-sense consideration, and unless this intuitive feeling told me that losing the bet is extremely unlikely, I would refuse to bet. And I honestly cannot think of a situation where translating this intuitive feeling of certainty into numbers would increase the clarity and accuracy of my thinking, or provide for any useful practical guidelines.
For example, if I come across a ditch and decide to jump over to save the effort of walking around to cross over a bridge, I'm effectively betting that it's narrow enough to jump over safely. In reality, I'll feel intuitively either that it's safe to jump or not, and I'll act on that feeling, produced by some opaque module for physics calculations in my brain. Of course, my conclusion might be wrong, and as a kid I would occasionally injure myself by judging wrongly in such situations, but how can I possibly quantify this feeling of certainty numerically in a meaningful way? It simply makes no sense. The overwhelming majority of real-life cases where I have to produce some judgment, and perhaps even bet on it, are of this sort.
It would be cool to have a brain that produces confidence estimates for its conclusions with greater precision, but mine simply isn't like that, and it's useless to pretend that it is.
You still have to be able to translate your superior relevant knowledge into odds in order to set the terms of the bet however. Do you not believe that this is an ability that people have varying degrees of aptitude for?
Vastly higher expertise than the average investor would appear to include something like the ability in question - translating your beliefs about the future into a probability such that you can judge whether investments have positive expected value. If you accept that true alpha exists (and the evidence suggests that though rare a small percentage of the best investors do appear to have positive alpha) then what process do you believe those who possess it use to decide which investments are good and which bad?
What's your opinion on prediction markets? They seem to produce fairly good probability estimates so presumably the participants must be using some better-than-random process for arriving at numerical probability estimates for their predictions.
They certainly aim for a balanced book but they wouldn't be very profitable if they were not reasonably competent at setting initial odds (and updating them in the light of new information). If the initial odds are wildly out of line with their customers' then they won't be able to make a balanced book.
mattnewport:
They sure do, but in all the examples I can think of, people either just follow their intuition directly when faced with a concrete situation, or employ rigorous science to attack the problem. (It doesn't have to be the official accredited science, of course; the Venn diagram of official science and valid science features only a partial overlap.) I just don't see any practical examples of people successfully betting by doing calculations with probability numbers derived from their intuitive feelings of confidence that would go beyond what a mere verbal expression of these feelings would convey. Can you think of any?
Well, if I knew, I would be doing it myself -- and I sure wouldn't be talking about it publicly!
The problem with discussing investment strategies is that any non-trivial public information about this topic necessarily has to be bullshit, or at least drowned in bullshit to the point of being irrecoverable, since exclusive possession of correct information is a sure path to getting rich, but its effectiveness critically depends on exclusivity. Still, I would be surprised to find out that the success of some alpha-achieving investors is based on taking numerical expressions of common-sense confidence seriously.
In a sense, a similar problem faces anyone who aspires to be more "rational" than the average folk in any meaningful sense. Either your "rationality" manifests itself only in irrelevant matters, or you have to ask yourself what is so special and exclusive about you that you're reaping practical success that eludes so many other people, and in such a way that they can't just copy your approach.
I agree with this assessment, but the accuracy of information aggregated by a prediction market implies nothing about your own individual certainty. Prediction markets work by cancelling out random errors and enabling specialists who wield esoteric expertise to take advantage of amateurs' systematic biases. Where your own individual judgment falls within this picture, you cannot know, unless you're one of these people with esoteric expertise.