Hi Holden,

I just read this thread today. I made a clarification upthread about the description of my comment above, under Louie's. Also, I'd like to register that I thought your characterization of that interview as such was fine, even without the clarifications you make here.

They both argue that standard Bayesian inference indicates against the literal use of non-robust expected value estimates, particularly in "Pascal's Mugging" type scenarios.

As a technical point, I don't think these posts address "Pascal's Mugging" scenarios in any meaningful way.

Bayesian adjustment is a standard part of Pascal's Mugging. The problem is that Solomonoff complexity priors have fat tails, because describing fundamental laws of physics that allow large payoffs is not radically more complex than laws that only allow small payoffs. It doesn't take an extra 10^1000 bits to describe a world where an action generates 2^(10^1000) times as many, e.g. happy puppies. So we can't rule out black swans a priori in that framework (without something like an anthropic assumption that amounts to the Doomsday Argument).

The only thing in your posts that could help with Pascal's Mugging is the assumption of infinite certainty in a distribution without relevantly fat tails or black swans, like a normal or log-normal distribution. But that would be an extreme move, taking coherent worlds of equal simplicity and massively penalizing the ones with high payoffs, so that no evidence that could fit in a human brain could convince us we were in the high-payoff worlds. Without some justification, that seems to amount to assuming the problem away, not addressing it.

Disclaimer 1: This is about expected value measured in the currency of "goods" like happy puppies, rather than expected utility, since agents can have bounded utility, e.g. simply not caring much more about saving a billion billion puppies rather than a billion. This seems fairly true of most people, at least emotionally.

Disclaimer 2: Occam's razor priors give high value to Pascal's Mugging cases, but they also give higher expectations to all other actions. For instance, the chance that space colonization will let huge populations be created increases the expected value of reducing existential risk by many orders of magnitude to total utilitarians. But it also greatly increases the expected payoffs of anything else that reduces existential risk by even a little. So if vaccinating African kids is expected to improve the odds of human survival going forward (not obvious but plausible) then its expected value will be driven to within sight of focused existential risk reductions, e.g. vaccination might be a billionth the cost-effectiveness of focused risk-reduction efforts but probably not smaller by a factor of 10^20. By the same token, different focused existential risk interventions will compete against one another, so one will not want to support the relatively ineffective ones.

Thanks for the helpful comments! I was uninformed about all those details above.

These posts are not about GiveWell's process.

One of the posts has the sub-heading "The GiveWell approach" and all of the analysis in both posts use examples of charities you're comparing. I agree you weren't just talking about the GiveWell process... you were talking about a larger philosophy of science you have that informs things like the GiveWell process.

I recognize that you're making sophisticated arguments for your points. Especially the assumptions that you claim simply must be true to satisfy your intuition that charities should be rewarded for transparency and punished otherwise. Those seem wise from a "getting things done" point of view for an org like GiveWell -- even when there is no mathematical reason those assumptions should be true -- but only a human-level tit-for-tat shame/enforcement mechanism you hope eventually makes this circularly "true" through repeated application. Seems fair enough.

But adding regression adjustments to cancel out the effectiveness of any charity which looks too effective to be believed (based on the common sense of the evaluator) seems like a pretty big finger on the scale. Why do so much analysis in the beginning if the last step of the algorithm is just "re-adjust effectiveness and expected value to equal what feels right"? Your adjustment factor amounts to a kind of Egalitarian Effectiveness Assumption: We are all created equal at turning money into goodness. Or perhaps it's more of a negative statement, like, "None of us is any better than the best of us at turning money into goodness" -- where the upper limit on the best is something like 1000x or whatever the evaluator has encountered in the past. Any claims made above the best limit gets adjusted back down -- those guys were trying to Pascal's Mug us! That's the way in which there's a blinding effect. You disbelieve the claims of any groups who claims to be more effective per capita than you think is possible.