prase comments on Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased) - Less Wrong

75 Post author: HoldenKarnofsky 18 August 2011 11:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (249)

Comments (249)

You are viewing a single comment's thread. Show more comments above.

Comment author: multifoliaterose 19 August 2011 05:15:05PM 0 points [-]

But, are there people who advocate such an approach?

See my response to Michael Vassar.

Now it is possible that I am responsible for the misinterpretations...

I don't have a clear understanding of the difference between the two italicized statements. Your first paragraphsing doesn't address three crucial (and in my opinion valid) points of the post that your second paraphrasing does:

  1. The degree to which one should trust one's prior over the new information depends on the measurement error attached to the new information.

  2. Assuming a normal/log-normal distribution for effectiveness of actions, the appropriate Bayesian adjustment is huge for actions with prima facie effectiveness many standard above the mean but which have substantial error bars.

  3. Humans do not have conscious/explicit access to most of what feeds into their implicit Bayesian prior and so in situations where a naive Fermi calculation yields high expected value and where there's high uncertainty, evolutionarily & experientially engrained heuristics can be a better guide to assessing the value of the action than attempting to reason explicitly as in the section entitled "Applying Bayesian adjustments to cost-effectiveness estimates for donations, actions, etc."

Comment author: prase 20 August 2011 01:14:38AM *  2 points [-]
  1. The first paraphrasing certainly addresses your first point. Assuming the prior has normal distribution with variance σ0^2 and mean μ0 and you measure μ, assuming that errors are normally distributed with variance σ^2, the updated mean is (μ σ0^2 + μ0 σ0^2)/(σ0^2 + σ0^2). The higher σ, the less the updating moves the estimate. This is standard Bayes.
  2. I can't parse your second point.
  3. It is indeed possible that evolutionarily engrained heuristics are a better guide than a more formal approach. But is is also possible that they aren't, especially applied to situations which are rare and so didn't exert selection pressure during human evolution. Many times I have seen common sense declared superior over formal analysis, but almost never had such declaration been supported by good arguments.

I agree that the Pascalesque arguments that appear on LW now and then, based on Fermi calculations and astronomical figures, such as

Even then the expected lives saved by SIAI is ~10^28.

are suspicious. But the problem lies already in the outlandish figure - the author should have taken the Fermi calculation, considered its error bars and combine it with his prior; presumably the result would be much lower then. If the HoldenKarnofsky's post suggests that, I agree. Only I am not sure whether the post doesn't rather suggest to accept that the expected value is unknowable and we should therefore ignore it, or even that it really may be 10^28, but we should ignore it nevertheless.

Comment author: multifoliaterose 20 August 2011 01:50:27AM *  0 points [-]

Re: #1, okay, it was unclear to me that all of that was packaged into your first paraphrasing.

Re: #2, not sure how else to explain; I had in mind the Pascal Mugging portion of the post.

Re: #3, as I mentioned to benelliott I would advocate a mixed strategy (heuristics together with expected value calculations). Surely our evolutionarily engrained heuristics a better guide than formal reasoning for some matters bearing on the efficacy of a philanthropic action.

But the problem lies already in the outlandish figure - the author should have taken the Fermi calculation, considered its error bars and combine it with his prior; presumably the result would be much lower then. If the HoldenKarnofsky's post suggests that, I agree.

All but the final section of the post are arguing precisely along these lines.

Only I am not sure whether the post doesn't rather suggest to accept that the expected value is unknowable and we should therefore ignore it, or even that it really may be 10^28, but we should ignore it nevertheless.

Not sure what Holden would say about this; maybe he'll respond clarifying.

Comment author: prase 20 August 2011 12:10:59PM *  0 points [-]

All but the final section of the post are arguing precisely along these lines.

That seems probable, and therefore I haven't said that I disagree with the post, but that I am confused about what it suggests. But I have some problems with the non-final sections, too, mainly concerning the terminology. For example, the phrases "estimated value" and "expected value", e.g. in

The crucial characteristic of the EEV approach is that it does not incorporate a systematic preference for better-grounded estimates over rougher estimates. It ranks charities/actions based simply on their estimated value, ignoring differences in the reliability and robustness of the estimates.

are used as if it simply meant "result of the Fermi calculation" instead of "mean value of probability distribution updated by the Fermi calculation". It seems to me that the post nowhere explicitly says that such estimates are incorrect and that it is advocating standard Bayesian reasoning, only done properly. After first reading I rather assumed that it proposes an extension to Bayes, where the agent after proper updating classifies the obtained estimates based on their reliabilities.

Also, I was not sure whether the post discusses a useful everyday technique when formal updating is unfeasible, or whether it proposes an extension to probability theory valid on the fundamental level. See also cousin_it's comments.

As for #2, i.e.

Assuming a normal/log-normal distribution for effectiveness of actions, the appropriate Bayesian adjustment is huge for actions with prima facie effectiveness many standard above the mean but which have substantial error bars.

mostly I am not sure what you refer to by appropriate Bayesian adjustment. On the first reading I interpreted is as "the correct approach, in contrast to EEV", but then it contradicts your apparent position expressed in the rest of the comment, where you argue that substantial error bars shoul prevent huge updating. The second interpretation may be "the usual Bayes updating", but then it is not true, as I argued in #1 (and in fact, I only repeat Holden's calculations).

Comment author: multifoliaterose 20 August 2011 07:32:01PM 0 points [-]

For example, the phrases "estimated value" and "expected value" [...] are used as if it simply meant "result of the Fermi calculation" instead of "mean value of probability distribution updated by the Fermi calculation". It seems to me that the post nowhere explicitly says that such estimates are incorrect and that it is advocating standard Bayesian reasoning, only done properly.

I'm very sure that in the section to which you refer "estimated value" means "result of a Fermi calcuation" (or something similar) as opposed "mean value of probability distribution updated by the Fermi calculation." (I personally find this to be clear from the text but may have been influenced by prior correspondence with Holden on this topic.)

The reference to "differences in the reliability and robustness of the estimates" refers to the size of the error bars (whether explicit or implicit) about the initial estimate.

Also, I was not sure whether the post discusses a useful everyday technique when formal updating is unfeasible, or whether it proposes an extension to probability theory valid on the fundamental level. See also cousin_it's comments.

Here too I'm very sure that the post is discussing a useful everyday technique when formal updating is unfeasible rather than an extension to probability theory valid on a fundamental level.

On the first reading I interpreted is as "the correct approach, in contrast to EEV", but then it contradicts your apparent position expressed in the rest of the comment, where you argue that substantial error bars shoul prevent huge updating. The second interpretation may be "the usual Bayes updating", but then it is not true, as I argued in #1 (and in fact, I only repeat Holden's calculations).

Here we had a simple misunderstanding; I meant "updating from the initial (Fermi calculation-based) estimate to a revised estimate after taking into account one's Bayesian prior" rather than "updating one's Bayesian prior to a revised Bayesian prior based on the initial (Fermi calculation-based) estimate."

I was saying "when there are large error bars about the initial estimate, the initial estimate should be revised heavily", not "when there are large error bars about the initial estimate, one's Bayesian prior should be revised heavily." On the contrary, larger the error bars about the initial estimate, the less one's Bayesian prior should change based on the estimate.

I imagine that we're in agreement here. I think that the article is probably pitched at someone with less technical expertise than you have; what seems obvious and standard to you might be genuinely new to many people and this may lead to you to assume that it's saying more than it is.

Comment author: prase 20 August 2011 09:42:05PM 0 points [-]

Then I suppose that we don't have disagreement, too.

Powered by Reddit Powered by Reddit