Comment author: David_Chapman 23 November 2013 10:57:56PM 2 points [-]

Hi, I have a site tech question. (Sorry if this is the wrong place to post that!—I couldn't find any other.)

I can't find a way to get email notifications of comment replies (i.e. when my inbox icon goes red). If there is one, how do I turn it on?

If there isn't one, is that a deliberate design feature, or a limitation of the software, or...?

Thanks (and thanks especially to whoever does the system maintenance here—it must be a big job.)

Probability and radical uncertainty

11 David_Chapman 23 November 2013 10:34PM

In the previous article in this sequence, I conducted a thought experiment in which simple probability was not sufficient to choose how to act. Rationality required reasoning about meta-probabilities, the probabilities of probabilities.

Relatedly, lukeprog has a brief post that explains how this matters; a long article by HoldenKarnofsky makes meta-probability  central to utilitarian estimates of the effectiveness of charitable giving; and Jonathan_Lee, in a reply to that, has used the same framework I presented.

In my previous article, I ran thought experiments that presented you with various colored boxes you could put coins in, gambling with uncertain odds.

The last box I showed you was blue. I explained that it had a fixed but unknown probability of a twofold payout, uniformly distributed between 0 and 0.9. The overall probability of a payout was 0.45, so the expectation value for gambling was 0.9—a bad bet. Yet your optimal strategy was to gamble a bit to figure out whether the odds were good or bad.

Let’s continue the experiment. I hand you a black box, shaped rather differently from the others. Its sealed faceplate is carved with runic inscriptions and eldritch figures. “I find this one particularly interesting,” I say.

continue reading »
Comment author: bokov 04 October 2013 03:32:21AM 0 points [-]

We could call this meta-probability, although that’s not a standard term.

Then why use it instead of learning the standard terms and using those? This might sound like pedantic, but it matters because this kind of thing leads to proliferation of unnecessary jargon and sometimes reinventing the wheel.

Are we talking about conditional probability? Joint probability?

Also, a minor nitpick about your next-to-last figure: given what's said about the boxes, it's not two bell curves centered at 0 and 0.9. It should be a point mass (vertical line) at 0 and a bell curve centered at 0.9.

Comment author: David_Chapman 23 November 2013 10:24:20PM 1 point [-]

Then why use it instead of learning the standard terms and using those?

The standard term is A_p, which seemed unnecessarily obscure.

Re the figure, see the discussion here.

(Sorry to be slow to reply to this; I got busy and didn't check my LW inbox for more than a month.)

Comment author: Gunnar_Zarncke 21 September 2013 09:00:32AM 2 points [-]

If you enjoy this sort of thing, you might like to work out what the exact optimal algorithm is.

I guess this is a joke. From wikipedia: "Originally considered by Allied scientists in World War II, it proved so intractable that, according to Peter Whittle, it was proposed the problem be dropped over Germany so that German scientists could also waste their time on it.[10]" (note that your wikipedia-link is broken)

Comment author: David_Chapman 21 September 2013 04:34:56PM 0 points [-]

Thank you very much—link fixed!

That's a really funny quote!

Multi-armed bandit problems were intractable during WWII probably mainly because computers weren't available yet. In many cases, the best approach is brute force simulation. That's the way I would approach the "blue box" problem (because I'm lazy).

But exact approaches have also been found: "Burnetas AN and Katehakis MN (1996) also provided an explicit solution for the important case in which the distributions of outcomes follow arbitrary (i.e., nonparametric) discrete, univariate distributions." The blue box problem is within that class.

Comment author: TheOtherDave 20 September 2013 07:44:39PM 2 points [-]

FWIW, I understood David to be requesting some specific examples of how members of the set "everyone else ever" handle this problem, which on your account is the same as how Jaynes handles it, in order to more clearly see the similarity you reference.

Comment author: David_Chapman 21 September 2013 04:35:31AM 1 point [-]

Thanks, yes! I.e. who is this "everyone else," and where do they treat it the same way Jaynes does? I'm not aware of any examples, but I have only a basic knowledge of probability theory.

It's certainly possible that this approach is common, but Jaynes wasn't ignorant, and he seemed to think it was a new and unusual and maybe controversial idea, so I kind of doubt it.

Also, I should say that I have no dog in this fight at all; I'm not advocating "Jaynes is the greatest thing since sliced bread", for example. (Although that does seem to be the opinion of some LW writers.)

Comment author: Douglas_Knight 20 September 2013 03:51:47PM 1 point [-]

The “meta-probability” approach I’ve taken here is the Ap distribution of E. T. Jaynes. I find it highly intuitive, but it seems to have had almost no influence or application in practice. We’ll see later that it has some problems, which might explain this.

I don't see how this differs from how anyone else ever handles this problem. I hope you explain the difference in this example, before going on to other examples.

Comment author: David_Chapman 20 September 2013 05:41:03PM 0 points [-]

Can you point me at some other similar treatments of the same problem? Thanks!

Comment author: roystgnr 19 September 2013 06:32:08PM *  4 points [-]

I'm looking forward to the rest of your sequence, thanks!

I was recently reading through a month-old blog post where one lousy comment was arguing against a strawman of Bayesian reasoning wherein you deal with probabilities by "mushing them all into a single number". I immediately recollected that the latest thing I saw on LessWrong was a fantastic summary of how you can treat mixed uncertainty as a probability-distribution-of-probability-distributions. I considered posting a belated link in reply, until I discovered that the lousy comment was written by David Chapman and the fantastic summary was written by David_Chapman.

I'm not sure if later you're going to go off the rails or change my mind or what, but so far this looks like one of the greatest attempts at "steelmanning" that I've ever seen on the internet.

Comment author: David_Chapman 20 September 2013 05:37:43PM 1 point [-]

Thanks, that's really funny! "On the other hand" is my general approach to life, so I'm happy to argue with myself.

And yes, I'm steelmanning. I think this approach is an excellent one in some cases; it will break down in others. I'll present a first one in the next article. It's another box you can put coins in that (I'll claim) can't usefully be modeled in this way.

Here's the quote from Jaynes, by the way:

What are we doing here? It seems almost as if we are talking about the ‘probability of a probability’. Pending a better understanding of what that means, let us adopt a cautious notation that will avoid giving possibly wrong impressions. We are not claiming that P(Ap|E) is a ‘real probability’ in the sense that we have been using that term; it is only a number which is to obey the mathematical rules of probability theory.

Comment author: roystgnr 18 September 2013 02:22:11AM 4 points [-]

The statement "probability estimates are not, by themselves, adequate to make rational decisions" could apparently have been replaced with the statement "my definition of the phrase 'probability estimates' is less inclusive than yours" - what you call a "meta-probability" I would have just called a "probability". In a world where both epistemic and aleatory uncertainty exist, your expectation of events in that world is going to look like a probability distribution over a space of probability distributions over outputs; this is still a probability distribution, just a much more expensive one to do approximate calculations with.

Comment author: David_Chapman 18 September 2013 06:47:05PM 0 points [-]

Yes, meta-probabilities are probabilities, although somewhat odd ones; they obey the normal rules of probability. Jaynes discusses this in his Chapter 18; his discussion there is worth a read.

The statement "probability estimates are not, by themselves, adequate to make rational decisions" was meant to describe the entire sequence, not this article.

I've revised the first paragraph of the article, since it seems to have misled many readers. I hope the point is clearer now!

Probability, knowledge, and meta-probability

38 David_Chapman 17 September 2013 12:02AM

This article is the first in a sequence that will consider situations where probability estimates are not, by themselves, adequate to make rational decisions. This one introduces a "meta-probability" approach, borrowed from E. T. Jaynes, and uses it to analyze a gambling problem. This situation is one in which reasonably straightforward decision-theoretic methods suffice. Later articles introduce increasingly problematic cases.

continue reading »
Comment author: Eliezer_Yudkowsky 16 September 2013 09:53:52PM 1 point [-]

I'm not sure I follow this. There is no prior distribution for the per-coin payout probabilities that can accurately reflect all our knowledge.

Are we talking about the Laplace vs. fair coins? Are you claiming there's no prior distribution over sequences which reflects our knowledge? If so I think you are wrong as a matter of math.

Comment author: David_Chapman 16 September 2013 10:40:54PM 1 point [-]

Are you claiming there's no prior distribution over sequences which reflects our knowledge?

No. Well, not so long as we're allowed to take our own actions into account!

I want to emphasize—since many commenters seem to have mistaken me on this—that there's an obvious, correct solution to this problem (which I made explicit in the OP). I deliberately made the problem as simple as possible in order to present the A_p framework clearly.

Are we talking about the Laplace vs. fair coins?

Not sure what you are asking here, sorry...

View more: Prev | Next