Okay. I'm sure you've seen this question before, but I'm going to ask it anyway.
Given a choice between
Are you really going to choose the former? What's your reasoning?
The second option is a world with seven billion -1 really happy people and one person who is a tiny bit less than mildly happy?
My reason to choose the former would be that all of those lives are experienced by only one person and everyone experiences only one life. In the former case, no subjective experience is worse than mildly happy. In the latter case, a subjective experience is worse than that. It doesn't matter how much happiness or pain a number of people will cumulatively experience because no one actually experiences the cumulative experience. All that matters is improving the worst life at any given moment.
I won't be surprised if my reasoning is bullshit, but I'm not seeing it.
I am trying to decide how to allocate my charitable donations between GiveWell's top charities and MIRI, and I need a probability estimate to make an informed decision. Could you help me?
Background on my moral system: I place a greater value on reducing high doses of suffering of conscious entities than merely preventing death. An unexpected, instant, painless death is unfortunate, but I would prefer it to a painful and chronic condition.
Given my beliefs, it follows logically that I would pay a relatively large amount to save a conscious entity from prolonged torture.
The possibility of an AI torturing many conscious entities has been mentioned1 on this site, and I assume that funding MIRI will help reduce its probability. But what is its current probability?
Obviously a difficult question, but it seems to me that I need an estimate and there is no way around it. I don't even know where to start...suggestions?
1 http://lesswrong.com/lw/1pz/the_ai_in_a_box_boxes_you/