= 27d567bc2d48d9f40e9ccc20ea370b15)
Ok, but if that's your reference class, "isn't a donkey sanctuary" counts as evidence you can update on. It seems there's large classes of charities we can be confident will not be extraordinarily effective, and these don't include FHI, MIRI etc.
Yes. There's a choice as to what to put into the prior and what to put into the likelihood. This makes it more difficult to make claims like "this number is a reasonable prior and this one is not". Instead, one has to specify the population the prior is about, and this in turn affects what likelihood ratios are reasonable.
In response to
Bayesian Adjustment Does Not Defeat Existential Risk Charity
Good post. Asking "okay, how sensitive is Karnofsky's counterargument to the size of the priors?" and actually answering that question was very worthwhile IMO.
Your post was funded by MIRI. Can you tell us what they asked? Was it "evaluate Karnofsky's argument", "rebut this post", "check the sensitivity of the argument to the priors' size and expand on it", "see how much BA affects our estimates", or what?
The project was initially described as synthesizing some of the comments on Karnofsky's post into a response mentioning counterintuitive implications of the approach, or into whichever synthesis of responses I thought was accurate.
Interesting. A parliamentary model applied to moral uncertainty definitely fails axiom 1 if any of the moral theories you're aggregating isn't VNM-rational. It probably still fails axiom 1 even if all of the individual moral theories are VNM-rational because the entire parliament is probably not VNM-rational. That's okay from Bostom's point of view because VNM-rationality could be one of the things you're uncertain about.
What if it is not, in fact, one of the things you're uncertain about?
Can you be more specific about what you mean by a parliamentary model? (If I had to guess, though, axiom 1.)
I'd be curious to see someone reply to this on behalf of parliamentary models, whether applied to preference aggregation or to moral uncertainty between different consequentialist theories. Do the choices of a parliament reduce to maximizing a weighted sum of utilities? If not, which axiom out of 1-3 do parliamentary models violate, and why are they viable despite violating that axiom?
Axiom 1: Every person, and the FAI, are VNM-rational agents.
[...]
So why should you accept my axioms?
Axiom 1: The VNM utility axioms are widely agreed to be necessary for any rational agent.
Though of course, humans are not VNM-rational.
Presumably there would be first be an extrapolation phase resulting in rational preferences.
All of that also applies to the year calibration questions in previous surveys and yet people did much better in those.
Because they weren't about events that occurred surprisingly early.
In response to
2012 Survey Results
The calibration question is an n=1 sample on one of the two important axes (those axes being who's answering, and what question they're answering). Give a question that's harder than it looks, and people will come out overconfident on average; give a question that's easier than it looks, and they'll come out underconfident on average. Getting rid of this effect requires a pool of questions, so that it'll average out.
I would agree that this explains the apparent atrocious calibration. It's worth an edit to the main post. No reason to beat ourselves up needlessly.
People were answering different questions in the sense that they each had an interval of their own choosing to assign a probability to, but obviously different people's performance here was going to be strongly correlated. Bayes just happens to be the kind of guy who was born surprisingly early. If everyone had literally been asked to assign a probability to the exact same proposition, like "Bayes was born before 1750" or "this coin will come up heads", that would have been a more extreme case. We'd have found that events that people predicted with probability x% actually happened either 0% or 100% of the time, and it wouldn't mean people were infinitely badly calibrated.
I think the concept is that content is included from trusting volunteers who were told to expect Crocker's Rules in the audience, and if you're not willing to abide by that trust, you shouldn't read.
So it sounds like the content can't be posted under Crocker's rules, because it's unreasonable to unilaterally exempt oneself from all ordinary social norms of politeness, even when people (sort of) have the option not to read; and the content can't be posted not under Crocker's rules, because the authors were promised that if it were posted, it would be under Crocker's rules. Maybe that means that if we're serious about upholding norms, it means daenerys has torpedoed her own project by making a promise she couldn't keep.
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
The much bigger issue is that for some anthropogenic risk (such as AI), the risk is caused by people, and can be increased by funding some groups of people. The expected utility thus has both positive and negative terms, and if you generate a biased list (e.g. by listening to what organization says about itself), and sum it, the resulting sum tells you nothing about the sign of expected utility.
I agree: the argument given here doesn't address whether existential risk charities are likely to be helpful or actively harmful. The fourth paragraph of the conclusion and various caveats like "basically competent" were meant to limit the scope of the discussion to only those whose effects were mostly positive rather than negative. Carl Shulman suggested in a feedback comment that one could set up an explicit model where one multiplies (1) a normal variable centered on zero, or with substantial mass below zero, intended to describe uncertainty about whether the charity has mostly positive or mostly negative effects, with (2) a thicker-tailed and always positive variable describing uncertainty about the scale the charity is operating on.