As everyone here knows, it would be a stupid idea to switch from airplanes to cars out of safety/terrorism concerns: Cars are a much more risky means of transportation than airplanes. But what other major risks are there that many people systematically undervalue or are not even consciously aware of? The same can be asked for chances.
TL;DR by lavalamp: Treating "computers running minds" as discrete objects might cause a paradox in probability calculations that involve self-location. "The probability of being a certain mind" is probably an extensive physical quantity, i.e. rises proportionally to the size of the physical system doing the associated computations. There are two...
Many people see themselves in various groups (member of the population of their home country, or their social network), and feel justified in caring more about the well-being of people in this group than about that of others. They will argue with reciprocity: "Those people pay taxes in our country,...
The idea is to compare not the results of actions, but the results of decision algorithms. The question that the agent should ask itself is thus: "Suppose everyone1 who runs the same thinking procedure like me uses decision algorithm X. What utility would I get at the 50th percentile (not:...
As everyone here knows, it would be a stupid idea to switch from airplanes to cars out of safety/terrorism concerns: Cars are a much more risky means of transportation than airplanes. But what other major risks are there that many people systematically undervalue or are not even consciously aware of?...
Have you ever convinced a religious person to become atheistic? How did you do this? How long did it take? Were the people in some sort of life crisis, or were they just living along? This is probably a quite difficult task of persuasion. So stories how people were successful...
Changed it, that sounds better.
I still think that the scenario you describe is not obviously and according to all philosophical intuitions the same as one where both minds exist in parallel.
Also, the expected number of paperclips (what you describe) is not equal to the expected experienced number of paperclips (what would be the relevant weighting for my post). After all, if A involves killing the maximizer before generating any paperclip, the paperclip-maximizer would choose A, while the experienced-paperclip-maximizer would choose B. The probability of experiencing paperclips would be obviously different from the probability of paperclips existing, when choosing A.
If I understand you correctly, your scenario is different from the one I had in mind in that I'd have both computers instantiated at the same time (I've clarified that in the post), and then considering the relative probability of experiencing what the 1 kg computer experiences vs experiencing what the 2 kg computer experiences. It seems like one could adapt your scenario by creating a 1 kg and a 2 kg computer at the same time, offering both of them a choice between A and B, and then generating 5 paperclips if the 1 kg computer chooses A and (additionally) 4 paperclips if the 2 kg computer chooses B. Then, the... (read more)
TL;DR by lavalamp: Treating "computers running minds" as discrete objects might cause a paradox in probability calculations that involve self-location. "The probability of being a certain mind" is probably an extensive physical quantity, i.e. rises proportionally to the size of the physical system doing the associated computations.
There are two computers simulating two minds. At some time, one of the minds is being shown a red light, and the other one is shown a green one (call this "Situation 1"). Conditioned on you being one of the minds, what is the probability you should assign to seeing red?
Naively, the answer seems to be 1/2, which comes from assigning being each of the minds... (read 572 more words →)
Many people see themselves in various groups (member of the population of their home country, or their social network), and feel justified in caring more about the well-being of people in this group than about that of others. They will argue with reciprocity: "Those people pay taxes in our country, they are entitled to more support from 'us' than others!" My question is: Is this inconsistent with some rationality axioms that seem obvious? What often-adopted or reasonable axioms are there that make this inconsistent?
Depending on the rest of your utility distribution, that is probably true. Note, however, that an additional 10^6 utility in the right half of the utility function will change the median outcome of your "life": If 10^6 is larger than all the other utility you could ever receive, and you add a 49 % chance of receiving it, the 50th percentile utility after that should look like the 98th percentile utility before.
I want that it is possible to have a very bad outcome: If I can play a lottery that has 1 utilium cost, 10^7 payoff and a winning chance of 10^-6, and if I can play this lottery enough times, I want to play it.
The idea is to compare not the results of actions, but the results of decision algorithms. The question that the agent should ask itself is thus:
"Suppose everyone1 who runs the same thinking procedure like me uses decision algorithm X. What utility would I get at the 50th percentile (not: what expected utility should I get), after my life is finished?"
Then, he should of course look for the X that maximizes this value.
Now, if you formulate a turing-complete "decision algorithm", this heads into an infinite loop. But suppose that "decision algorithm" is defined as a huge table for lots of different possible situations, and the appropriate outputs.
Let's see what results such a thing... (read 163 more words →)
As everyone here knows, it would be a stupid idea to switch from airplanes to cars out of safety/terrorism concerns: Cars are a much more risky means of transportation than airplanes. But what other major risks are there that many people systematically undervalue or are not even consciously aware of?
The same can be asked for chances.
Me too!
Have you ever convinced a religious person to become atheistic? How did you do this? How long did it take? Were the people in some sort of life crisis, or were they just living along?
This is probably a quite difficult task of persuasion. So stories how people were successful at it could be very interesting to improve ones' persuasion abilities.
Relatedly, it might be interesting to know what religious groups have gathered on techniques to convert people to their religion - are there some manuals/techniques floating around?
The problem seems to vanish if you don't ask "What is the expectation value of utility for this decision, if I do X", but rather "If I changed my mental algorithms so that they do X in situations like this all the time, what utility would I plausibly accumulate over the course of my entire life?" ("How much utility do I get at the 50th percentile of the utility probability distribution?") This would have the following results:
For the limit case of decisions where all possible outcomes happen infinitely often during your lifetime, you would decide exactly as if you wanted to maximize expectation value in an individual case.
You would not decide to
Seconding, though I originally meant the second question. I am also not sure whether you are referring to "conceptual analysis" (then the second question would be clear to me) or "nailing down a proper (more proper) definition before arguing about something" (then it would not).
Now, for concepts like "democracy", the unnatural approach does prove to be worse.
Why?
The rock on top of the computer wouldn't count into the "amount doing the computation". Apart from that, I agree that weight shouldn't be the right quantity. A better way to formulate what I am getting at would maybe be that "probability of being a mind is an extensive physical quantity". I have updated the post accordingly.
Regarding your second paragraph: No, the TV screens aren't part of the matter that does the computation.