You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Irrationality Game III

11 Post author: CellBioGuy 12 March 2014 01:51PM

The 'Irrationality Game' posts in discussion came before my time here, but I had a very good time reading the bits written in the comments section.  I also had a number of thoughts I would've liked to post and get feedback on, but I knew that being buried in such old threads not much would come of it.  So I asked around and feedback from people has suggested that they would be open to a reboot!

I hereby again quote the original rules:

Please read the post before voting on the comments, as this is a game where voting works differently.

Warning: the comments section of this post will look odd. The most reasonable comments will have lots of negative karma. Do not be alarmed, it's all part of the plan. In order to participate in this game you should disable any viewing threshold for negatively voted comments.

Here's an irrationalist game meant to quickly collect a pool of controversial ideas for people to debate and assess. It kinda relies on people being honest and not being nitpickers, but it might be fun.

Write a comment reply to this post describing a belief you think has a reasonable chance of being true relative to the the beliefs of other Less Wrong folk. Jot down a proposition and a rough probability estimate or qualitative description, like 'fairly confident'.

Example (not my true belief): "The U.S. government was directly responsible for financing the September 11th terrorist attacks. Very confident. (~95%)."

If you post a belief, you have to vote on the beliefs of all other comments. Voting works like this: if you basically agree with the comment, vote the comment down. If you basically disagree with the comment, vote the comment up. What 'basically' means here is intuitive; instead of using a precise mathy scoring system, just make a guess. In my view, if their stated probability is 99.9% and your degree of belief is 90%, that merits an upvote: it's a pretty big difference of opinion. If they're at 99.9% and you're at 99.5%, it could go either way. If you're genuinely unsure whether or not you basically agree with them, you can pass on voting (but try not to). Vote up if you think they are either overconfident or underconfident in their belief: any disagreement is valid disagreement.

That's the spirit of the game, but some more qualifications and rules follow.

If the proposition in a comment isn't incredibly precise, use your best interpretation. If you really have to pick nits for whatever reason, say so in a comment reply.

The more upvotes you get, the more irrational Less Wrong perceives your belief to be. Which means that if you have a large amount of Less Wrong karma and can still get lots of upvotes on your crazy beliefs then you will get lots of smart people to take your weird ideas a little more seriously.

Some poor soul is going to come along and post "I believe in God". Don't pick nits and say "Well in a a Tegmark multiverse there is definitely a universe exactly like ours where some sort of god rules over us..." and downvote it. That's cheating. You better upvote the guy. For just this post, get over your desire to upvote rationality. For this game, we reward perceived irrationality.

Try to be precise in your propositions. Saying "I believe in God. 99% sure." isn't informative because we don't quite know which God you're talking about. A deist god? The Christian God? Jewish?

Y'all know this already, but just a reminder: preferences ain't beliefs. Downvote preferences disguised as beliefs. Beliefs that include the word "should" are are almost always imprecise: avoid them.

That means our local theists are probably gonna get a lot of upvotes. Can you beat them with your confident but perceived-by-LW-as-irrational beliefs? It's a challenge!

Additional rules:

  • Generally, no repeating an altered version of a proposition already in the comments unless it's different in an interesting and important way. Use your judgement.
  • If you have comments about the game, please reply to my comment below about meta discussion, not to the post itself. Only propositions to be judged for the game should be direct comments to this post. 
  • Don't post propositions as comment replies to other comments. That'll make it disorganized.
  • You have to actually think your degree of belief is rational.  You should already have taken the fact that most people would disagree with you into account and updated on that information. That means that  any proposition you make is a proposition that you think you are personally more rational about than the Less Wrong average.  This could be good or bad. Lots of upvotes means lots of people disagree with you. That's generally bad. Lots of downvotes means you're probably right. That's good, but this is a game where perceived irrationality wins you karma. The game is only fun if you're trying to be completely honest in your stated beliefs. Don't post something crazy and expect to get karma. Don't exaggerate your beliefs. Play fair.
  • Debate and discussion is great, but keep it civil.  Linking to the Sequences is barely civil -- summarize arguments from specific LW posts and maybe link, but don't tell someone to go read something. If someone says they believe in God with 100% probability and you don't want to take the time to give a brief but substantive counterargument, don't comment at all. We're inviting people to share beliefs we think are irrational; don't be mean about their responses.
  • No propositions that people are unlikely to have an opinion about, like "Yesterday I wore black socks. ~80%" or "Antipope Christopher would have been a good leader in his latter days had he not been dethroned by Pope Sergius III. ~30%." The goal is to be controversial and interesting.
  • Multiple propositions are fine, so long as they're moderately interesting.
  • You are encouraged to reply to comments with your own probability estimates, but  comment voting works normally for comment replies to other comments.  That is, upvote for good discussion, not agreement or disagreement.
  • In general, just keep within the spirit of the game: we're celebrating LW-contrarian beliefs for a change!

I would suggest placing *related* propositions in the same comment, but wildly different ones might deserve separate comments for keeping threads separate.

Make sure you put "Irrationality Game" as the first two words of a post containing a proposition to be voted upon in the game's format.

Here we go!

EDIT:  It was pointed out in the meta-thread below that this could be done with polls rather than karma so as to discourage playing-to-win and getting around the hiding of downvoted comments.  If anyone resurrects this game in the future, please do so under that system  If you wish to test a poll format in this thread feel free to do so, but continue voting as normal for those that are not in poll format. 

Comments (204)

Comment author: D_Malik 16 March 2014 06:02:42AM 0 points [-]

Irrationality game: The straightforward view of the nature of the universe is fundamentally flawed. 90%

By "fundamentally flawed", I mean things like:

  • I am currently dreaming.
  • The singularity has already happened, and this world maximizes my CEV.
  • I am a Boltzmann brain.
  • I am in a simulation.
  • Or some similar thing is true, but I haven't thought of it.
Comment author: [deleted] 16 March 2014 03:10:38AM -2 points [-]

Irrationality Game: We need a way to give feedback on irrationality game entries that the troll toll won't mess with. (98%)

Submitting...

Comment author: Alexei 14 March 2014 06:33:03PM *  6 points [-]

Irrationality Game: One can reliable and predictably make $1M / year, and it's not that difficult. (Confidence: 75%)

Comment author: John_Maxwell_IV 16 March 2014 07:37:41PM 2 points [-]

Please elaborate!

Comment author: [deleted] 16 March 2014 03:21:30AM 5 points [-]

What do you mean by “one”? Literally anyone at all? Anyone at least as smart as the average LWer? Something else?

Comment author: Alexei 16 March 2014 09:30:57PM 1 point [-]

Let's say someone with an engineering bachelor's degree.

Comment author: polymathwannabe 14 March 2014 06:48:42PM 2 points [-]

Care to describe how?

Comment author: Alexei 16 March 2014 09:31:12PM -1 points [-]

Figuring that out right now.

Comment author: blacktrance 14 March 2014 04:49:09PM 0 points [-]

Irrationality game: most opposition to wireheading comes from seeing it as weird and/or counterintuitive in the same way that most non-LWers see cryonics/immortalism as weird. Claiming to have multiple terminal values is an attempt to justify this aversion. 75%

Comment author: Douglas_Knight 14 March 2014 04:14:33PM -1 points [-]

The Hellenistic astronomers (300BC-0) were generally heliocentric. 90%

Comment author: lmm 14 March 2014 12:48:31PM 2 points [-]

Irrationality game: people are happier when living in traditional social structures, and value being part of their traditions[1]. The public existence of "weird" relationships (homosexuality, polyamory, BDSM, ...) is actively harmful to most people; the open practice of them is a net negative for world utility. Morally good actions include condemnation and censorship of such things.

[1] Or rather what they believe are their traditions; these beliefs may not be particularly well-correlated with reality.

Comment author: John_Maxwell_IV 16 March 2014 07:39:09PM 0 points [-]

How did you come to believe this?

Comment author: [deleted] 16 March 2014 03:22:12AM 1 point [-]

Probability?

Comment author: lmm 17 March 2014 09:03:13AM 1 point [-]

Ah, sorry. 0.8

Comment author: Lumifer 14 March 2014 02:42:04PM 2 points [-]

That looks to be a pretty standard neo-reactionary position.

Comment author: lmm 15 March 2014 10:24:29PM 1 point [-]

Sure, but I expect the proportion of LW that actually agrees with neoreaction to be small (and I disagree with most neoreactionary positions).

Comment author: Fhyve 14 March 2014 04:21:08AM *  -1 points [-]

Irrationality Game:

Politics (in particular, large governments such as the US, China, and Russia) are a major threat to the development of friendly AI. Conditional on FAI progress having stopped, I give a 60% chance that it was because of government interference, rather than existential risk or some other problem.

Comment author: CellBioGuy 14 March 2014 01:37:06AM *  1 point [-]

Irrationality Game:

Everyone alive in developed nations today will die a fairly standard biological death by age:

150: 75%

250: 95%

(This latter figure accounts for the possibility that the stories of the odd Chinese monk living to age 200+ after only eating wild herbs from age 10 on up is actually true and not an exaggeration, or someone sticking to unreasonably-effective calorie restriction regimes religiously combined with some interesting metabolic rejiggering in the coming decade or two).

The majority (90+%) of people born in developed nations today will die a fairly standard biological death by age:

120: 85%

150: 99%

Comment author: Eugine_Nier 14 March 2014 01:52:28AM *  3 points [-]

I think the probability of a nuclear war or bio-engineered plague is higher than 5%.

Comment author: [deleted] 16 March 2014 03:19:49AM 1 point [-]

I think there's a good chance that a nuclear war would kill less than 90% of the population, though.

Comment author: [deleted] 16 March 2014 02:24:53PM 0 points [-]

(But now that I think about it, even the ones it doesn't kill straight away will be much less likely to live to 120 than they otherwise would.)

Comment author: Brillyant 13 March 2014 02:39:21PM 37 points [-]

Irrationality Game: Less Wrong is simply my Tyler Durden—a disassociated digital personality concocted by my unconcious mind to be everything I need it to be to cope with Camusian absurdist reality. 95%.

Comment author: Dallas 16 March 2014 12:59:33PM 0 points [-]

I am very curious as to what your evidence for backing up this proposition is or would be.

Comment author: [deleted] 16 March 2014 03:12:55AM 1 point [-]

If you're right, your unconscious mind is awesome.

Comment author: ThisSpaceAvailable 16 March 2014 01:02:51AM 2 points [-]

So, I'm supposed to upvote this unless I believe that I am a figment of your imagination? This seems a bit cheaty. Maybe I should post "No one other than ThisSpaceAvailable believes that this statement is true. 99.99%"

Comment author: kokotajlod 14 March 2014 12:21:08PM 1 point [-]

Are we supposed to judge the rationality of this statement based on how rational we think it is for you to believe it, or based on how rational we think it is for us to believe it?

Comment author: gothgirl420666 13 March 2014 10:21:44PM *  1 point [-]

Wow. I think this one might win.

Comment author: Lalartu 13 March 2014 10:19:05AM 32 points [-]

Irrationality game:

Most posthuman societies will have violent death rate much higher than humans ever had. Most poshumans who will ever live will die at wars. 95%

Comment author: aausch 26 July 2015 06:30:44PM 0 points [-]

i think the concept of death is extremely poorly defined under most variations of posthuman societies; death as we interpret it today depends on a number of concepts that are very likely to break down or be irrelevant in a post-human-verse


take, for example, the interpretation of death as the permanent end to a continuous distinct identity:

if i create several thousand partially conscious partial clones of myself to complete a task (say, build a rocketship), and then reabsorb and compress their experiences, have those partial clones died? if i lose 99.5% of my physical incarnations and 50% of my processing power to an accident, did any of the individual incarnations die? have i died? what if some other consciousness absorbs them (with or without my, or the clones', permission or awareness)? what if i become infected with a meme which permanently alters my behavior? my identity?

Comment author: Nornagest 14 March 2014 05:40:53PM *  4 points [-]

Seems fairly reasonable on its face, actually; once you've gotten rid of disease and age, what's left is accidents, violence, and suicide if you're counting that separately from violence.

Upvoted, though, because I think you're undercounting accidents (more Americans already die in automotive accidents than die violently, by a large margin; I'd expect the same is true for the rest of the First World but haven't seen statistics) and making too strong a statement about the structure of posthuman society.

Comment author: NancyLebovitz 13 March 2014 12:43:28PM 5 points [-]

Are you imagining that outcome because you expect resource shortages? Peaceful lives are just too boring? Posthumans are generally too alien to each other for stable cooperation?

Now that I think about it, I find the last alternative pretty plausible.

Comment author: drnickbone 13 March 2014 12:10:59PM 8 points [-]

Not a bad hypothesis, but your confidence level is too high... hence my upvote.

Comment author: [deleted] 16 March 2014 03:13:47AM 0 points [-]

Same here. I'd give it around 20%.

Comment author: khafra 13 March 2014 10:55:56AM 8 points [-]

Interesting. So, you have Robin Hanson's belief that we won't get a strong singleton; but you lack his belief that emulated minds will be able to evaluate each other's abilities with enough confidence that trade (taking into account the expected value of fighting) will be superior to fighting? That's quite the idiosyncratic position, especially for 95% confidence.

Comment author: ChristianKl 13 March 2014 09:53:34AM *  0 points [-]

Irrationality game:

Most progress in medicine in the next 50 years won't be due to advances in molecular biology and the production of drugs that are designed to target specific biochemical pathways but through other paradigms.

Probability: 75%

Comment author: polymathwannabe 13 March 2014 03:46:20PM 0 points [-]

Which other paradigms do you predict will become more relevant?

Comment author: ChristianKl 13 March 2014 10:29:57PM 0 points [-]

I think that you can learn a lot via empirical measurement without needing to understand underlying biochemistry. Apart from direct measurement it also about developing better metrics for things like lung function, That partly why I invested significant effort into Quantified Self community building.

Exploring the phenomenological aspect of illnesses provides an area with a lot of untapped potential for knowledge gathering.

I think there are large returns found in studying human movement in detail. As it stands a method like Feldenkrais isn't has some studies to support it but no strong scientifically investigated base. It should be possible to use cameras and computers to get accurate models of human movement and investigate an approach like Feldenkrais in deeper scientific way.

Relaxation is generally poorly understood. I think most patients could benefit from spending hours in floating tanks after having a major operation, yet few hospitals have floating tanks or maximize the relaxation in other ways. In general hospital beds and hospital food doesn't seem to be optimized for health outcomes.

Having a well developed theory that can predict placebo effects accurately would be good. On the one hand it will make it easier to gather knowledge, on the other hand it will help doctors to frame their interactions in a way that helps patients.

Having good empathy training for doctors has potential to improve healthcare.

Psychological interventions like hypnosis.

Biofeedback.

That's just a list of possibilities I can think of. There are probably things that are unknown unknowns for me.

Comment author: polymathwannabe 13 March 2014 10:47:37PM -1 points [-]

Those approaches risk turning therapy into stabs in the dark by neglecting the details of what is actually going on inside the black box.

Comment author: ChristianKl 13 March 2014 11:14:01PM 0 points [-]

Those approaches risk turning therapy into stabs in the dark by neglecting the details of what is actually going on inside the black box.

Most people who pretend that they know what goes on inside the black box are wrong anyway.

Drug companies would like to predict which components work based on understanding biochemistry but they still have to run expensive trials in which over 90% of the components fail. Furthermore it becomes exponentially more expensive to discover additional drugs in that way.

That said, the idea of preaching blindness that currently haunts medical research is exactly about the virtue of stabing into the dark. If you would see what you are doing you wouldn't be objective anymore.

Comment author: drnickbone 13 March 2014 08:17:55AM 15 points [-]

The universe is finite, and not much bigger than the region we observe. There is no multiverse (in particular Many Worlds Interpretation is incorrect and SIA is incorrect). There have been a few (< million) intelligent civilisations before human beings but none of them managed to expand into space, which explains Fermi's paradox. This also implies a mild form of the "Doomsday" argument (we are fairly unlikely to expand ourselves) but not a strong future filter (if instead millions or billions of civilisations had existed, but none of them expanded, there would be a massive future filter). Probability: 90%.

Comment author: polymathwannabe 13 March 2014 02:06:59PM 0 points [-]

I don't know how to vote on this. I have very strong suspicions that MWI is incorrect (its Copernican allure is its only favorable point), but I disagree that the universe is finite. I feel inclined toward SIA, but I generally reject anthropic reasoning (that's perhaps a statement about myself rather than about your arguments).

(Also, I require more detailed arguments to dissolve Fermi's paradox because I don't believe paradoxes exist in reality.)

Comment author: shminux 16 March 2014 02:17:55AM -1 points [-]

I have very strong suspicions that MWI is incorrect

How would you evaluate correctness of something untestable?

Comment author: polymathwannabe 16 March 2014 03:55:58AM *  -1 points [-]

I don't know whether this counts as a correctness assessment, but my expectations do not vary with the trueness of MWI, so it's a needless hypothesis.

Comment author: drnickbone 13 March 2014 03:06:54PM *  0 points [-]

I'd suggest that since you agree with some parts but disagree with others, you assign probability a lot less than 90% to the whole hypothesis. So you should think I'm irrationally overconfident in the whole lot, and upvote please!

If you want some detail, I start from the "Great Filter" argument (see http://hanson.gmu.edu/greatfilter.html). I find it very hard to believe that there is a super-strong future filter ahead of us (such that we have < 1 in a million or < 1 in a billion chance of passing it and then expanding into space). But a relatively weak filter implies that rather few civilizations can have got to our stage of development - there can't have been millions or billions of them, or some would have got past the filter and expanded, and we would not expect to see the world as we do in fact see it. The argument to the universe being finite (and not too big) then follows from there being a limited number of civilizations. SIA and MWI must also be wrong, because they each imply a very large or infinite number of civilizations.

Comment author: Nisan 13 March 2014 04:14:57PM 1 point [-]

Your conclusion doesn't follow from your premises. The lack of a strong filter implies that a not insignificant proportion of civilizations colonize space. This is consistent with there being a large universe containing many intergalactic civilizations we will never observe because of the expansion of the universe.

Comment author: drnickbone 13 March 2014 05:34:00PM *  0 points [-]

No, in that large universe model we'd expect to be part of one of the expanded, intergalactic civilisations, and not part of a small, still-at-home civilisation. So, as I stated "we would not expect to see the world as we do in fact see it". Clearly we could still be part of a small civilisation (nothing logically impossible about being in a tiny minority), or we could be in some sort of zoo or ancestor simulation within a big civilisation. But that's not what we'd expect to see. You might want to see Ken Olum's paper for more on this: http://arxiv.org/abs/gr-qc/0303070

Incidentally, Olum considers several different ways out of the conflict between expectation and observation: the finite universe is option F (page 5) and that option seems to me to be a lot more plausible than any of the alternatives he sketches. But if you disagree, please tell me which option you think more likely.

Comment author: ThisSpaceAvailable 16 March 2014 02:12:17AM 0 points [-]

I find that sort of anthropic argument to Prove Too Much. For instance, our universe is about 14 billion years old, but many models have the universe existing trillions of years into the future. If the universe were to survive 280 billion years, then that would put us within the first 5% of the universe's lifespan. So, if we take an alpha of 5%, we can reject the hypothesis that the universe will last more than 280 billion years. We can also reject the hypothesis that more than 4 trillion humans lives will take place, that any given 1-year-old will reach the age of 20, that humans will have machines capable of flight for more than 2000 years, etc.

Olum appears to be making a post hoc argument. The probability that the right sperm would fertilize the right egg and I would be conceived is much less than 1 in a billion, but that doesn't mean I think I need a new model. The probability of being born prior to a galactic-wide expansion may be very low, but someone has to be born before the expansion. What's so special about me, that I should reject the possibility that I such a person?

Comment author: drnickbone 17 March 2014 01:07:12PM *  1 point [-]

If the universe were to survive 280 billion years, then that would put us within the first 5% of the universe's lifespan. So, if we take an alpha of 5%, we can reject the hypothesis that the universe will last more than 280 billion years.

That sounds like "Copernican" reasoning (assume you are at a random point in time) rather than "anthropic" reasoning (assume you are a random observer from a class of observers). I'm not surprised the Copernican approach gives daft results, because the spatial version (assume you are at a random point in space) also gives daft results: see here in this thread point 2.

Incidentally, there is a valid anthropic version of your argument: the prediction is that the universe will be uninhabitable 280 billion years from now, or at least contain many fewer observers than it does now. However, in that case, it looks like a successful prediction. The recent discovery that the stars are beginning to go out and that 95% of stars that will ever form have formed already is just the sort of thing that would be expected under anthropic reasoning. But it is totally surprising otherwise.

We can also reject the hypothesis that more than 4 trillion humans lives will take place

The correct application of anthropic reasoning only rejects this as a hypothesis about the average number of observers in a civilisation, not about human beings specifically. If we knew somehow (on other grounds) that most civilisations make it to 10 trillion observers, we wouldn't predict any less for human beings.

that any given 1-year-old will reach the age of 20,

That's an instance of the same error: anthropic reasoning does NOT reject the particular hypothesis. We already know that an average human lifespan is greater than 20, so we have no reason to predict less than 20 for a particular child. (The reason is that observing one particular child at age 1 as a random observation from the set of all human observations is no less probable if she lives to 100 than if she lives to 2).

The probability that the right sperm would fertilize the right egg and I would be conceived is much less than 1 in a billion, but that doesn't mean I think I need a new model

Anthropic reasoning is like any Bayesian reasoning: observations only count as evidence between hypotheses if they are more likely on one hypothesis than another. Also, hypotheses must be fairly likely a priori to be worth considering against the evidence. Suppose you somehow got a precise observation of sperm meeting egg to make you, with a genome analysis of the two: that exact DNA readout would be extremely unlikely under the hypothesis of the usual laws of physics, chemistry and biology. But that shouldn't make you suspect an alternative hypothesis (e.g. that you are some weird biological experiment, or a special child of god) because that exact DNA readout is extremely unlikely on those hypotheses as well. So it doesn't count as evidence for these alternatives.

The probability of being born prior to a galactic-wide expansion may be very low, but someone has to be born before the expansion. What's so special about me, that I should reject the possibility that I such a person?

If all hypotheses gave extremely low probability of being born before the expansion, then you are correct. But the issue is that some hypotheses give high probability that an observer finds himself before expansion (the hypotheses where no civilisations expand, and all stay small). So your observations do count as evidence to decide between the hypotheses.

Comment author: drnickbone 14 March 2014 05:18:33PM *  0 points [-]

I got a bit distracted by the "anthropic reasoning is wrong" discussion below, and missed adding something important. The conclusion that "we would not expect to see the world as we in fact see it" holds in a big universe regardless of the approach taken to anthropic reasoning. It's worth spelling that out in some detail.

  1. Suppose I don't want to engage in any form of anthropic reasoning or observation sampling hypothesis. Then the large universe model leaves me unable to predict anything much at all about my observations. I might perhaps be in a small civilisation, but then I might be in a simulation, or a Boltzmann Brain, or mad, or a galactic emperor, or a worm, or a rock, or a hydrogen molecule. I have no basis for assigning significant probability to any of these - my predictions are all over the place. So I certainly can't expect to observe that I'm an intelligent observer in a small civilisation confined to its home planet.

  2. Suppose I adopt a "Copernican" hypothesis - I'm just at a random point in space. Well now, the usual big and small universe hypotheses predict that I'm most likely going to be somewhere in intergalactic or interstellar space, so that's not a great predictive success. The universe model which most predicts my observations looks frankly weird... instead of a lot of empty space, it is a dense mass of "computronium" running lots of simulations of different observers, and I'm one of them. Even then I can't expect to be in a simulation of a small civilisation, since the sim could be of just about anything. Again, not a great predictive success.

  3. Suppose I adopt SIA reasoning. Then I should just ignore the finite universes, since they contribute zero prior probability. Or if I've decided for some reason to keep all my universe hypotheses finite, then I should ignore all but the largest ones (ones with 3^^^3 or more galaxies). Among the infinite-or-enormous universes, they nearly all have expanded civilisations, and so under SIA, nearly all predict that I'm going to be in a big civilisation. The only ones which predict otherwise include a "universal doom" - the probability that a small civilisation ever expands off its home world is zero, or negligibly bigger than zero. That's a massive future filter. So SIA and big universes can - just about - predict my observations, but only if there is this super-strong filter. Again, that has low prior probability, and is not what I should expect to see.

  4. Suppose I adopt SSA reasoning. I need to specify the reference class, and it is a bit hard to know which one to use. In a big universe, different reference classes will lead to very different predictions: picking out small civilisations, large civilisations, AIs, SIMs, emperors and so on (plus worms, rocks and hydrogen for the whackier reference classes). As I don't know which to use, my predictions get smeared out across the classes, and are consequently vague. Again, I can't expect to be in a small civilisation on its home planet.

By contrast, look at the small universe models with only a few civilisations. A fair chunk of these models have modest future filters so none of the civilisations expand. For those models, SSA looks in quite good shape, as there is quite a wide choice of reference classes that all lead to the same prediction. Provided the reference class predicts I am an intelligent observer at all then it must predict I am in a small civilisation confined to its home planet (because all civilisations are like that). Of course there are the weird classes which predict I'm a worm and so on - nothing we can do about those - but among the sensible classes we get a hit.

So this is where I'm coming from. The only model which leads me to expect to see what I actually see is a small universe model, with a modest future filter. Within that model, I will need to adopt some sort of SSA-reasoning to get a prediction, but I don't have to know in advance which reference class to use: any reference class which selects an intelligent observer predicts roughly what I see. None of the other models or styles of reasoning lead to that prediction.

Comment author: Squark 14 March 2014 12:18:44PM 0 points [-]

This sort of anthropic reasoning is wrong. Consider the following experiment.

A fair coin is tossed. If the result is H, you are cloned into 10^10 copies, and all of those copies except one are placed in the Andromeda galaxy. Another copy remains in the Milky Way. If the result is T, no cloning occurs and you remain in the Milky Way. Either way, the "you" in Milky Way has no immediate direct way to know about the result of the coin toss.

Someone, call her "anthropic mugger", comes to you an offers a bet. She can perform an experiment which will reveal the result of the coin toss (but she hasn't done it yet). If you accept the bet and the coin toss turns out to be H, she pays you 1$. If you accept the bet and the coin toss turns out to be T, you pay her 1000$. Do you accept the bet?

Reasoning along the same lines as you did to conclude there are no large civilizations, you should accept the bet. But this means your expected gain before the coin toss is -499.5$. So, before the coin toss it is profitable for you to change your way of reasoning so you won't be tempted to accept the bet.

There's no reason to accept the bet unless in the cloning scenario you care much less about the copy of you in Milky Way than in the no-cloning scenario. So, there's no reason to assume there are no large civilizations if the existence of large civilizations wouldn't make us care much less about our own.

Comment author: drnickbone 14 March 2014 02:40:07PM *  0 points [-]

There are a number of problems with that:

1) You don't specify whether the bet is offered to all my copies or just to one of them, or if to just one of them, whether it is guaranteed to be the one in the Milky Way. Or if the one in the Milky Way knows he is in the Milky Way when taking the bet, and so on.

Suppose I am offered the bet before knowing whether I am in Andromeda or Milky Way. What odds should I accept on the coin toss: 50/50? Suppose I am then told I am in the Milky Way... what odds should I now accept on the coin-toss: still 50/50? If you say 50/50 in both cases then you are a "double-halfer" (in the terminology of Sleeping Beauty problems) and you can be Dutch-booked. If you answer other than 50/50 in one case or the other, then you are saying there are circumstances where you'd bet at odds different (probably very different) from the physical odds of a fair coin toss and without any context, that sounds rather crazy. So whatever you say, there is a bullet to bite.

2) I am, by the way, quite aware of the literature on Anthropic Decision Theory (especially Stuart Armstrong's paper) and since my utility function is roughly the average utility for my future copies (rather than total utility) I feel inclined to bet with the SSA odds. Yes, this will lead to the "me" in the Milky Way making a loss in the case of "H" but at that stage he counts for only a tiny slither of my utility function, so I think I'll take the risk and eat the loss. If I modify my reasoning now then there are other bets which will lead to a bigger expected loss (or even a guaranteed loss if I can be Dutch-booked).

Remember though that I only assigned 90% probability to the original hypothesis. Part of the remaining 10% uncertainty is that I am not fully confident that SSA odds are the right ones to use. So the anthropic mugger might not be able to make $500 off me (I'm likely to refuse the 1000:1 bet), but he probably could make $5 off me.

3) As in many such problems, you oversimplify by specifying in advance that the coin is fair, which then leads to the crazy-sounding betting odds (and the need to bite a bullet somewhere). But in the real world case, the coin has unknown bias (as we don't know the size of the future filter). This means we have to try to estimate the bias (size of filter) based on the totality of our evidence.

Suppose I'm doubtful about the fair coin hypothesis and have two other hypotheses: heavy bias towards heads or heavy bias towards tails. Then it seems very reasonable that under the "bias towards heads" hypothesis I would expect to be in Andromeda, and if I discover I am not, that counts as evidence for the "bias towards tails" hypothesis. So as I now suspect bias in one particular direction, why still bet on 50/50 odds?

Comment author: Squark 14 March 2014 02:59:08PM 0 points [-]

1) You don't specify whether the bet is offered to all my copies or just to one of them, or if to just one of them, whether it is guaranteed to be the one in the Milky Way. Or if the one in the Milky Way knows he is in the Milky Way when taking the bet, and so on.

I meant that the bet is offered to the copy in the Milky Way and that he knows he is in the Milky Way. This is the right analogy with the "large civilizations" problem since we know we're in a small civilization.

Suppose I am offered the bet before knowing whether I am in Andromeda or Milky Way. What odds should I accept on the coin toss: 50/50?

In your version of the problem the clones get to bet too, so the answer depends on how your utility is accumulated over clones.

So whatever you say, there is a bullet to bite.

If you have a well-defined utility function and you're using UDT, everything makes sense IMO.

Suppose I'm doubtful about the fair coin hypothesis and have two other hypotheses: heavy bias towards heads or heavy bias towards tails.

It doesn't change anything in principle. You just added another coin toss before the original coin toss which affects the odd of the latter.

Comment author: drnickbone 14 March 2014 03:59:01PM *  0 points [-]

I meant that the bet is offered to the copy in the Milky Way and that he knows he is in the Milky Way. This is the right analogy with the "large civilizations" problem since we know we're in a small civilization.

Well we currently observe that we are in a small civilisation (though we could be in a zoo or simulation or whatever). But to assess the hypotheses in question we have to (in essence) forget that observation, create a prior for small universe versus big universe hypotheses, see what the hypotheses predict we should expect to observe, and then update when we "notice" the observation.

Alternatively, if you adopt the UDT approach, you have to consider what utility function you'd have before knowing whether you are in a big civilization or not. What would the "you" then like to commit the "you" now to deciding?

If you think you'd care about average utility in that original situation then naturally the small civilisations will get less weight in outcomes where there are big civilisations as well. Whereas if there are only small civilisations, they get all the weight. No difficulties there.

If you think you'd care about total utility (so the small civs get equal weight regardless) then be carefully that it's bounded somehow. Otherwise you are going to have a known problem with expected utilities diverging (see http://lesswrong.com/lw/fg7/sia_fears_expected_infinity/).

It doesn't change anything in principle. You just added another coin toss before the original coin toss which affects the odd of the latter.

A metaphorical coin with unknown (or subjectively-assigned) odds is quite a different beast from a physical coin with known odds (based on physical facts). You can't create crazy-sounding conclusions with metaphorical coins (i.e. situations where you bet at million to 1 odds, despite knowing that the coin toss was a fair one.)

Comment author: Squark 14 March 2014 06:49:00PM 0 points [-]

If you think you'd care about average utility in that original situation then naturally the small civilisations will get less weight in outcomes where there are big civilisations as well. Whereas if there are only small civilisations, they get all the weight. No difficulties there.

If you think you'd care about total utility (so the small civs get equal weight regardless) then be carefully that it's bounded somehow. Otherwise you are going to have a known problem with expected utilities diverging (see http://lesswrong.com/lw/fg7/sia_fears_expected_infinity/).

I think that I care about a time-discounted utility integral within a future light-cone. Large civilizations entering this cone don't reduce the utility of small civilizations.

A metaphorical coin with unknown (or subjectively-assigned) odds is quite a different beast from a physical coin with known odds (based on physical facts).

I don't believe in different kinds of coins. They're all the same Bayesian probabilities. It's a meta-Occam razor: I don't see any need for introducing these distinct categories.

Comment author: Nisan 13 March 2014 10:17:41PM 0 points [-]

Oh I see, that makes sense.

Comment author: polymathwannabe 13 March 2014 04:17:33PM -1 points [-]

I agree with this counterargument, but this thread being what it is, in which direction should I vote sub-comments?

Comment author: Squark 14 March 2014 08:29:40AM 1 point [-]

Subcomments are voted by the ordinary rules

Comment author: Gunnar_Zarncke 13 March 2014 07:47:54AM 4 points [-]

Irrationality game:

Nice idea. This way I can safely test whether the Baseline of my opinion on LW topics is as contrarian as I think.

My proposition:

On The Simulation Argument I go for "(1) the human species is very likely to go extinct before reaching a “posthuman” stage" (80%)

Correspondingly on The Great Filter I go for failure to reach "9. Colonization explosion" (80%).

This is not because I think that humanity is going to self-annihilate soon (though this is a possibility).

Comment author: Squark 14 March 2014 12:25:17PM 0 points [-]

What is the extinction scenario you have in mind?

Comment author: Gunnar_Zarncke 14 March 2014 05:28:22PM 0 points [-]

Hopefully no extinction during the next many thousends of years. Which extinction afterwards is difficult to predict.

As I shortly argued in my baseline post I think that posthuman state is unlikely due to thermodynamics/complexity constraints.

Comment author: kokotajlod 13 March 2014 04:11:00AM *  26 points [-]

Irrationality Game: I am something ontologically distinct from my body; I am much simpler and I am not located in the same spacetime. 50%

EDIT: Upon further reflection, my probability assignment would be better represented as the range between 30% and 50%, after factoring in general uncertainty due to confusion. I doubt this will make a difference to the voting though. ;)

Comment author: [deleted] 16 March 2014 03:15:40AM 0 points [-]

Taboo “I”. For all the ways of interpreting that claim that I can come up with, I'd give a probability either much, much higher or much, much lower than 50%.

Comment author: kokotajlod 17 March 2014 04:35:05PM 1 point [-]

Could you list the ways? I'm interested to hear which ways you think would give a probability much higher than 50%.

Also, telling me to taboo "I" is telling me to give a successful analysis of consciousness; if I could already do that, I wouldn't assign 50% probability to it being one thing, and 50% probability to it being another.

Comment author: [deleted] 30 March 2014 07:33:13AM 0 points [-]

I mean, sometimes by the word “I” I mean my System 2 (as in “I'm not sure what the effect of stricter gun control on the homicide rate would be”), sometimes I mean my System 1 (as in “I'm not scared of spiders”), sometimes I mean my body (as in “I've got a backache”), sometimes I mean my car (as in “I'm parked over there”), etc. Which one did you mean there?

...Oh. Now I can see a reasonable non-tautological, non-tautologically false interpretation of your entry.

Comment author: Eugine_Nier 14 March 2014 01:44:36AM 1 point [-]

Why is this a 19? I thought this was a restatement of the "official LW position". Or would people argue that an uploaded kokotajlod wouldn't be the real kokotajlod?

Comment author: Randy_M 14 March 2014 01:13:35PM 3 points [-]

I guess if you read it loosely. I think the official LW position would be (correct me if I am wrong) an em of kokotajlod that has high enough fidelity to replicate his decision making process is him; what he is is a particular set of hueristics, instincts, etc, that accompany his body but could theoretically exist outside it. That does match his statement if one reads it as refering to something more like a platonic concept than a spiritual essence.

Comment author: kokotajlod 14 March 2014 12:14:28PM *  1 point [-]

I think the "official LW position" is more reductionist than my Irrationality Game statement.

Even if people think that I am a computation, they probably don't think I'm some sort of Platonic Form, but rather that I'm just a certain type of physical object that "implements" a computation.

That's my understanding of typical LW thought, at any rate. Which is why I chose the statement that I did. :)

Comment author: NancyLebovitz 13 March 2014 01:46:25PM 2 points [-]

Why would you be much simpler if you were ontologically distinct?

Comment author: kokotajlod 13 March 2014 04:19:12PM *  0 points [-]

The ";" was meant to be simply a "and also" rather than a "therefore."

I think that I'm much simpler than my body, and that is one of the reasons why I think I'm ontologically distinct. With 50% probability.

EDIT: Another answer I endorse: If I'm ontologically distinct from my body, then who knows how complicated I am--but apply Occam's Razor, such that I'm only as complicated as I need to be, and the result will be that I'm much simpler than my body. Anyone who believes that uploading would preserve consciousness should agree with this, since uploading (can) change the medium of computation to a simpler one.

Comment author: khafra 13 March 2014 10:41:13AM 1 point [-]

What does "spacetime" mean? Is the real "you" neither a causal descendent, nor a causal ancestor, of any of your body's actions? I'd have to put that down somewhere around argmin probability.

Or do you just mean that you consider the real you to be something like a platonic computation, which your material body instantiates? That's not too far off from some realms of LW semi-orthodoxy.

Comment author: kokotajlod 13 March 2014 01:41:07PM 2 points [-]

Good questions. I'll explain my reasoning:

Basically, after thinking about consciousness for a while, and personal identity, I've come to assign high probability to some sort of dualism/idealism being true. It might still be a sort of reductionist dualism, i.e. platonic computations.

So yes, the "platonic computation" theory would count. Do you think my original post ought to be revised given this information? I hope I haven't been misleading.

As for spacetime and causation: If I'm a platonic form, I'm not in spacetime, nor am I causally related to my body in any normal sense. It all depends on how we define causation, and I tend to be reductionist/eliminativist about causation.

Comment author: khafra 13 March 2014 03:33:10PM 0 points [-]

I hope I haven't been misleading.

I don't think you've been any more misleading than a dualist is pretty much required to be. The basic ambiguities of dualism do, of course, remain:

  1. How does the non-spacetime stuff produce subjective experience, when spacetime stuff can't?

  2. How does your subjective experience correlate with the environment and actions of your material body, just as if there were two-way causation going on? (even when you reduce causation to a Pearl-style net, or to the large-scale behavior of many individually time-reversible components, this question remains).

Comment author: kokotajlod 13 March 2014 04:15:15PM 1 point [-]

(1) It's not about producing subjective experience, it is about being subjective experience. The idea here is that massive, vague mereological fusions of subatomic stuff just aren't the sort of thing that can be subjective experiences. Just as no fundamental particle can be a chariot, since chariots have parts.

(2) I have no idea yet. I'm considering some sort of interactionist dualism, or some sort of idealism, or some sort of mathematical multiverse theory with self-contained mathematical structures that play the role of Platonic computations, with measure assigned by a simplicity weighting that generates the appearance of a physical world.

And of course I'm considering reductionist physicalism, reductionist mathematical multiverse theory, etc. as well. That's where the other 50% comes in.

Comment author: NoSuchPlace 12 March 2014 11:27:04PM *  34 points [-]

Irrationality game: Every thing which exists has subjective experience (80%). This includes things such as animals, plants, rocks, ideas, mathematics, the universe and any sub component of an aforementioned system.

Comment author: ThisSpaceAvailable 16 March 2014 01:10:24AM 1 point [-]

Besides the issue of "subjective experience" that has already been brought up, there's also the question of what "thing" and "exists" mean. Are abstract concepts "things"? Do virtual particles "exist"? By including ideas, you seem to be saying "yes" to the first question. So do subjective experiences have subjective experiences themselves?

Also, it's "an aforementioned". That's especially important when speaking.

Comment author: NoSuchPlace 18 March 2014 12:39:35AM 0 points [-]

Besides the issue of "subjective experience" that has already been brought up, there's also the question of what "thing" and "exists" mean.

I believe some form of MUH is correct so when I say exist I mean the same thing as in mathematics (in the sense of quantifying over various things). So by a thing I mean anything for which it is (at least in principle) possible to write down a mathematically precise definition.

Presumably abstract ideas and virtual particles fall under this category though in neither case am I sure because I don't know what you mean by abstract idea/I don't know enough physics. I not sure whether it possible to give a definition for subjective experience so I don't know whether subjective experiences have subjective experiences.

Also, it's "an aforementioned". That's especially important when speaking.

Substituted an a for an an.

Comment author: khafra 13 March 2014 10:46:08AM 4 points [-]

By "any subcomponent," do you mean that the powerset of the universe is composed of conscious entities, even when light speed and expansion preclude causal interaction within the conscious entity? Because, if the universe is indeed spatially infinite, that means that the set of conscious entities is the infinity of the continuum; and I'm really confused by what that does to anthropic reasoning.

Comment author: NoSuchPlace 13 March 2014 01:51:54PM 2 points [-]

By "any subcomponent," do you mean that the powerset of the universe is composed of conscious entities, even when light speed and expansion preclude causal interaction within the conscious entity?

If you replace consciousness with subjective experience I believe your statement is correct. Also once you have one infinity you can take power sets again and again.

I'm really confused by what that does to anthropic reasoning

As far as I understand it breaks anthropic reasoning because now your event space is to big to be able to define a probability measure. For the time being I have concluded that anthropic reasoning doesn't work because of a very similar argument though I will revise my argument once I have learned the relevant math.

Comment author: shminux 12 March 2014 11:43:31PM 1 point [-]

How would one define subjective experience for rocks and atoms?

Comment author: NoSuchPlace 13 March 2014 12:40:04AM 5 points [-]

Defining subjective experience is hard for the same reason that defining red is hard, since they are direct experiences. However in this case I can't get around this by pointing at examples. So the only thing I can do is offer an alternative phrasing which suffers from the same problem:

If you accept that our experiences are what an algorithm feels like from on the inside then I am saying that everything feels like something from the inside.

Comment author: Ixiel 12 March 2014 09:54:36PM *  16 points [-]

Irrationality game - there is a provident, superior entity that is in no way infinite (I wonder if people here would call that God. As a "superman theist" I had to put "odds of God (as defined in question)" at 5% but identify as strongly theist in the last census)

Edit: forgot odds. 80%

Comment author: gothgirl420666 13 March 2014 08:27:03AM 1 point [-]

What are your odds on this?

Comment author: Ixiel 13 March 2014 08:41:55AM 0 points [-]

Oh thanks. Fixed.

Comment author: NoSuchPlace 12 March 2014 10:24:53PM 2 points [-]

Could you explain your reasoning? I'm very curious about this.

Comment author: Ixiel 13 March 2014 08:58:49AM 5 points [-]

I was brought up Catholic, and quickly decided religion (later updated to human scribes millennia ago and blind faith therein) didn't really understand the difference between "bigger than I can understand" and "infinite." I also have a life so cartoonishly awesome (let me know if you have a solution to this, but I honestly believe if I laid down the facts people would think I'm lying), I figured what I called God not only exists but likes me more than everybody else. As I grew up, I "tested" the theory a few times, but never with any scientific rigor, and I think I'd have to call the results positive but not statistically significant. I have no problem assuming no god at the beginning of a discussion, and if I had strong enough evidence I'd like to think I'd admit I'm wrong. I also don't correlate anything about God with misunderstanding what "death" means - or as many Catholics call it life after death.

I know it's a minority view here and would never trot it out in normal discourse, but it seemed appropriate for the venue.

Comment author: Will_Newsome 14 March 2014 02:36:17AM 3 points [-]

I figured what I called God not only exists but likes me more than everybody else.

That's impossible, He likes me more than everybody else. (.000001% confident)

(Seriously though, I do believe there are god-like phenomena, and they seem to be suspiciously favorable toward me. (Excuse me while I knock on wood and stare plaintively at the fourth wall.))

Comment author: TsviBT 13 March 2014 04:45:53PM 0 points [-]

I too am curious.

Comment author: RowanE 13 March 2014 02:48:33PM 2 points [-]

I'm intruiged - a life so awesome that it's implausible for a member of an internet forum with thousands of members (especially this one, which is dedicated to the "science of winning at life" and has an average IQ so high it should be impossible) to actually be living it strikes me as a high bar, and additionally the idea of it being so awesome that it's (at least subjectively) convincing evidence for the hypothesis "god is real and he loves me". The only thing that comes to my mind that looks like it meets that criteria and isn't blatantly supernatural would be winning the lottery, but since lottery winners are often less happy in the end I don't expect that's it.

I can't promise I'll believe you, but I'd upvote just for sating my curiosity.

Comment author: Ixiel 13 March 2014 08:19:34PM *  5 points [-]

Nah, not implausible I exist, but I rarely post, so have no track record. It's amazing how many people are above average online...

Mentally, I'm materially above average intelligence, but understand that that only goes so far. And I cultivate rationality (I'm here aren't I?)

Socially, I'm reasonably well liked by everyone I know, people tell me I have a decent sense of humor. I'm engaged to a beautiful blonde doctor, who is eerily similar to the woman I prayed to meet as a teenager, and has been able to put up with my strangeness for four years.

Bodily, I have no known history of any genetic diseases and have never been dependant on prescription drugs. Though I admit the surgeon general would like me to lose a pound or two. Not "Mommy why's he like that?" fat though.

Financially, I own my own house, and if I (and they) decide to have kids, my kids and grandkids will never have to work, assuming I don't earn/inherit/win/cetera a penny, and my stocks gain 0% (Admittedly, they could crash). I tell people I have a Forrest Gump approach. "Lt. Dan said I didn't have to worry about money any more. And I said, well, that's good. One less thing."

Attitudinally, I'm hugely optimistic. Not every day, but more often than not, I wake up and am struck by the wonder of how unlikely my good fortune is.

I know it sounds out there, and it is, but it's also true. Hand to God. Or Bacon, or whomever you'd like, if you dig propriomanual verification.

Comment author: falenas108 14 March 2014 05:25:45PM -2 points [-]

I agree with RomanE in that this doesn't seem all that unusual. I'm an undergrad in college right now so I don't have the monetary security, fiancee, or house, but everything else applies to me as well. There are a couple of things that could help to explain this, in my cause and probably in yours.

Are you fairly neurotypical in a way that doesn't interfere with your social life or physical well-being? Did you grow up with a middle, upper middle, or upper class family? Do you grow up in a developed nation? Are you of a racial class that is generally privileged in your area? (E.g., white in America.)

If these are true, or even just the first three, I would say it's not all that unusual. Not lower than 5%, anyway. I don't think that, given the above, it is unusual enough to warrant an explanation outside the ordinary.

Comment author: Ixiel 14 March 2014 06:02:19PM 0 points [-]

Yes, upper, yes, and yes. As stated in the other reply, I do not currently count it as strong evidence.

Comment author: RowanE 14 March 2014 10:23:50AM 2 points [-]

The only parts of that list that seem out-there are the fiancee-eerily-similar-to-prayers (alternate explanations: The kind of people you actively seek out and the kind of people you pray to meet are going to resemble each other, human memories are fallible to the point that they can be completely fabricated, so a vague similarity might adjust to become an eerily close similarity) and the financial status (I'm not sure what my prior for this is supposed to be, since "rich enough never to have to work for three generations" varies a lot on expenditure, probably more than just the top 1% could get there if they were frugal).

Comment author: Ixiel 14 March 2014 11:01:40AM 1 point [-]

Oh agreed. My awesome life is not a good proof, but while I came with a high IQ out of the box, I hadn't learned the tools of thinking yet to the necessary degree. It's loosely confirmatory, but not a silver bullet. I was just saying that prompted me to have the idea a decade or two before (inadequitely still, but I knew it) testing the idea. My confidence may be too high, but really, it hasn't been a priority to test mostly because I can't think of a good one that doesn't come at too high a cost for too little benefit. I've never really tried to prove my Fiancé's aunt who I've never met exists either. Open to ideas.

And the grandkids thing is something I came up with to give my fiancé perspective when she was just my girlfriend. Anything with "illion" in it becomes "a bunch" to non math folk. Think about 4 mil. Now think about 8. I'd say (based on nothing but anecdote) if you said each to fifty men-on-the-street, you'd have at least 85 people thinking of pretty much the same pile of gold doubloons.

Comment author: Will_Newsome 21 March 2014 01:40:15AM 1 point [-]

Commit to not talk to anyone about the results of your test. No hinting, nothing. If you do this the experiment is worthless. Don't even mention it; if asked about it, say 'I promised beforehand that I would not give out information'. Take a coin, flip it twenty times or so, record heads and tails. This is not a good test but it's simple and easy and can at least theoretically provide some information.

Comment author: polymathwannabe 12 March 2014 08:17:23PM 2 points [-]

I'm 99% confident that dust specks in 3^^^3 eyes result in less lost utility than 50 years of torturing one person.

Comment author: [deleted] 16 March 2014 03:31:58AM 2 points [-]

Y'all know this already, but just a reminder: preferences ain't beliefs. Downvote preferences disguised as beliefs. Beliefs that include the word "should" are are almost always imprecise: avoid them.

Comment author: rthomas6 13 March 2014 03:31:31PM *  1 point [-]

I agree with you (maybe not 99% certainty though), and I'm surprised more people do not.That is, assuming the original stipulation of the dust specks causing only a "mild inconvenience" for everyone, and not some sort of bell curve of inconvenience with a mean of "mild". People around here seem to grok the idea of the hedonic treadmill, so why don't they apply that idea to this situation? Assuming all of those 3^^^3 people all truly only have a "mild inconvenience", I would argue that from a subjective point of view from each individual, the utility of their day as a whole has not been diminished at all.

Actually, the more I think about it, the idea itself is poorly formed. It depends a lot on what sort of inconvenience the dust causes. If it causes 0.00000001% of the people to decide to shoot up a store or something, then I guess the one person being tortured would be better. But if the dust does not cause any sort of cascading effect, if it's truly isolated to the lost utility of the dust itself, then I'd say the dust is better.

Comment author: polymathwannabe 13 March 2014 03:40:10PM *  1 point [-]

Elsewhere I argued that the pain from the dust specks doesn't add up (and is therefore not really comparable to one single person's torture) unless the victims are forming a hive mind. What the thought experiment is actually comparing is one instance of horrible pain versus many, many individual and not groupable instances of minor discomfort.

Comment author: Nornagest 13 March 2014 12:52:52AM 8 points [-]

Utility seems underspecified here.

Comment author: solipsist 12 March 2014 06:45:29PM *  28 points [-]

You (the reader) do not exist.

EDIT: That was too punchy and not precise. The reasoning behind the statement:

Most things which think they are me are horribly confused gasps of consciousness. Rational agents should believe the chances are small that their experiences are remotely genuine.

EDIT 2: After thinking about shminux's comment, I have to retract my original statement about you readers not existing. Even if I'm a hopelessly confused Boltzmann brain, the referent "you" might still well exist. At minimum I have to think about existence more. Sorry!

Comment author: [deleted] 16 March 2014 03:11:30AM 3 points [-]

Cogito, ergo upvoto. :-)

Comment author: rthomas6 12 March 2014 08:55:13PM *  0 points [-]

What degree of certainty do you place on that belief?

Comment author: solipsist 12 March 2014 10:16:28PM *  0 points [-]

I could put numbers to it, but it would really be pulling them out of my butt -- how certain are you that anthropic reasoning is valid? If it is valid (which seems more likely than not), then you quickly run into the problem of Boltzmann brains. Some people try to exorcise Boltzmann brains from their anthropic viewpoint, but I have no problem with biting the bullet that most brains are Boltzmann brains. The practical implications of that belief, assuming the world is as it appears to be, are (I believe) minimal.

Comment author: maia 12 March 2014 07:05:23PM 1 point [-]

Could you be more specific about what you mean by that?

Comment author: solipsist 12 March 2014 07:52:17PM *  1 point [-]

Of all possible minds thinking the thought that I am thinking right now, most aren't living on earth, posting to Less Wrong. Most are random fluctuations in high-entropy soup after the heat-death of the universe, or bizarre minds capable of belief but not thought, or other deluded or confused agents. In all but a negligible fraction of these, you, maia, do not exist.

Comment author: [deleted] 12 March 2014 06:58:05PM *  1 point [-]

.

Comment author: solipsist 12 March 2014 07:30:34PM *  2 points [-]

To quote the adage, I'm a solipsist, and am surprised everyone else isn't too. I think any intelligent agent should conclude that it is probably something akin to a Boltzmann brain. You could plausibly argue that I am cheating with pronoun references (other people might agree with the solipsistic logic, but centered around them). Is that what you are asking?

EDIT

Is there anything about the world that you expect would appear different to you because of this belief?

Not really. I think some of the problems with AIXI may be AIXI acting rationally where the desired behavior is irrational, but that's the only time I can think of it coming up outside of a philosophy discussion.

Comment author: Eitan_Zohar 12 March 2014 05:34:24PM 2 points [-]

Irrationality Game: Currently, understanding history or politics is a better avenue than studying AI or decision theory for dealing with existential risk. This is not because of the risk of total nuclear annihilation, but because of the possibility of political changes that result in setbacks to or an accelerated use and understanding of AI. 70%

Comment author: [deleted] 12 March 2014 04:44:17PM *  3 points [-]

.

Comment author: Will_Newsome 14 March 2014 02:44:28AM *  4 points [-]

If this train of thought continues along its natural course, you have to wonder why you are being "shown" the experience you have this moment as you read this, rather than some other more interesting or influential moment. Also, it is not clear that you would want to use this kind of anthropic reasoning to determine a policy; people that are not conscious but think they are would incorrectly draw the same conclusions and thus muck up the social commons with their undue senses of specialness.

ETA: There are a few other counterarguments similar to those in the previous paragraph. This has perturbed me for many years now, because the line of reasoning in the parent comment really does seem like the most intuitive approach to subjective anthropics. I'd be very satisfied to find a solution, but it seems equally likely that there's just something pretty wrong with our intuitions about (relative) existence, which has implications for which kinds of decision theories we should be willing to put our weight on.

ETA2: And the UDT pragmatist in me wonders whether it even means anything for a hypothesis to be true, if it rationally shouldn't affect your decisions. If anything I would lean toward decision theoretic epiphenomenalism implying falsehood.

Comment author: RowanE 13 March 2014 02:55:27PM 1 point [-]

The part of this I disagree with most is putting the probability as low as 10%. I up voted, since that seemed like just putting a number to the word "significant", and the other claims seem about right.

Comment author: trist 12 March 2014 03:00:19PM 35 points [-]

Irrationality Game: (meta, I like this idea)

Flush toilets are a horrible mistake. 7b/99%

Comment author: faul_sname 13 March 2014 07:04:07AM 0 points [-]

Upvoted for "horrible". I don't see how their impact is all that bad -- at 3.5 GPF (which is standard), that means that, for example, all of the flush toilets in California together use about 750,000 acre-feet of water per year. Compared to the 34 million acre feet used in the same state for agriculture, it's clear that flush toilets use a significant but still pretty small fraction of the water in the state, but "horrible" is an overstatement. (I choose California because it is a populous state that regularly has water shortages).

Comment author: trist 13 March 2014 03:07:56PM *  -1 points [-]

I admit to hyperbole, now, with a little more thought, I would have worded it differently. Both to clarify that it's pretty far down on our list of societal problems, and that it's more an individual level mistake rather than a systematic one (though there are systematic benefits to fewer flush toilets).

Comment author: Slackson 13 March 2014 02:55:18AM 2 points [-]

"If it's yellow let it mellow, if it's brown flush it down."

This is one of the first things I remember learning, growing up with tank water.

Comment author: ChristianKl 12 March 2014 04:46:16PM 3 points [-]

Based on what reasoning?

Comment author: rthomas6 12 March 2014 03:54:17PM 1 point [-]

How is this not just a preference?

Comment author: trist 12 March 2014 04:50:10PM 1 point [-]

I suppose my actual belief is that flush toilets are a mistake outside of urban areas, I don't have much experience with urban living or what other poop strategies could work with it.

Advantages, flush toilets:

  • Provide easy long distance transport of human waste in urban environments.
  • Exchanges weekly-to-yearly chores for purchased services.

Disadvantages, flush toilets:

  • Create additional dependency on water (and by extension outside water districts, electricity).
  • Turn (vast amounts of) drinking water into black water.
  • Create a waste product from human manure, which is a valuable resource (fertile soil) when dealt with properly.
  • Adds significantly to the cost of housing (especially outside sewer districts).
Comment author: Lumifer 12 March 2014 08:25:06PM 1 point [-]

You forgot the "not having the whole village die from a cholera outbreak" part :-/

Comment author: CellBioGuy 12 March 2014 08:29:51PM *  2 points [-]

The running water revolution came around the same time as the sanitation revolution. I'm not 100% sure you necessarily HAVE to have one to have the benefits of the other, though it helps. Modern composting toilets and hot-composting of human manure is quite safe if done properly. Flush toilets definitely get the sanitation thing done, but perhaps rather than 'mistake' we could call them an 'inefficient first draft that still works well'.

Comment author: Lumifer 12 March 2014 08:43:02PM *  7 points [-]

is quite safe if done properly

That's the thing -- it's basically an issue of idiot-proofing. Many things are "safe if done properly" and still are not a good idea because people in general are pretty bad at doing things properly.

Flush toilets are idiot-proof to a remarkable degree. Composting human manure, I have my doubts.

Comment author: [deleted] 16 March 2014 03:17:55AM 1 point [-]

Flush toilets are idiot-proof to a remarkable degree.

Almost all of them are, but I've seen a couple of them which are very easy to accidentally flush improperly in such a way that water will keep on running until someone notices and fixes it.

Comment author: rthomas6 12 March 2014 08:10:56PM *  1 point [-]

That clears things up a lot, and I changed my downvote to an upvote. EDIT: To be clear, I disagree with you.

My thoughts on your disadvantages list:

  • Flush toilets do create additional dependency on water, however if one already has running water and depends on it for drinking and washing, how significant is the additional water dependency for flush toilets?
  • The reason flush toilets use potable water is an economic one. It is simply cheaper to use one unified water system instead of two, when someplace already has running water. The cost of the wasted drinking water is negligible compared to the cost of building a second plumbing system.
  • This point is the most interesting to me. I have no information on the usefulness of human manure, and would be interested to know if human manure would have a comparable market value to cattle manure or synthetic fertilizer. I am skeptical because of the tendency for human waste to carry human diseases.
  • I have no disagreements with this disadvantage, but simply feel that the vast, vast majority of people would be willing to pay for the extra cost in housing if they already had indoor plumbing.
Comment author: NancyLebovitz 13 March 2014 01:10:43AM 1 point [-]

I expect this is too expensive to be worth it. but instead of a whole second water system, it's theoretically possible to use gray water from bathing and showering for flushing.

On second thought, this might actually make sense for apartment buildings and hotels, since some gray water could be stored and sent downhill for flushing-- you wouldn't need a pump in the bathroom.

Comment author: roystgnr 12 March 2014 09:43:07PM 1 point [-]

Austin's "Dillo Dirt" is made from yard waste and treated human sewage. Less-treated sewage gets used to fertilize ranchland. As you suspected, there's more than a little controversy over whether the result is well-composted enough for health and aesthetics, but it's mixed up with concern over the standards for various non-fecal pollutants. Presumably whatever closed loop fertilization trist is advocating wouldn't have to worry so much about the various kinds of industrial and medical waste people dump down their drains.

Comment author: gothgirl420666 12 March 2014 09:27:23PM 0 points [-]

I changed my downvote to an upvote.

I think you're playing it wrong? You upvote if you disagree.

Comment author: rthomas6 12 March 2014 09:40:09PM 0 points [-]

I do disagree. Did you read the rest of my comment? I originally downvoted because the rules also say to downvote if someone expresses a preference disguised as a belief.

Comment author: Error 12 March 2014 06:53:02PM 3 points [-]

I still think you're overconfident, so upvoting, but the justification is convincing enough to make me update from near-zero to something noticeably above zero. I never thought of it quite that way.

Comment author: trist 12 March 2014 09:59:01PM 0 points [-]

Translate it to "In x% of new non-urban houses, there are options better than flush toilets." My confidence in my confidence assignment isn't very high yet though, so I am quite open to being overconfident.

And obviously both lists are non-exhaustive.

  • Flush toilets handle large numbers of people for a long time fairly easily.
  • Flush toilets get clogged.
Comment author: CellBioGuy 12 March 2014 02:03:05PM *  5 points [-]

Irrationality game:

There are other 'technological civilizations' (in the sense of intelligent living things that have learned to manipulate matter in a complicated way) in the observable universe: 99%

There are other 'technological civilizations' in our own galaxy: 75% with most of the probability mass in regimes where there are somewhere between dozens and thousands.

Conditional on these existing: Despite some being very old, they are limited by the hostile nature of the universe and the realities of practical manipulation of matter and energy to never controlling much matter outside the surfaces of life-bearing worlds, and either never leave their solar systems of origin with anything self-replicating or their replicators on average produce less than 1 seed to continue. 95%

Humanity has already received and recorded a radio signal from another thing-analogous-to-a-technological-civilization. This was either unnoticed or not unequivocally recognized as such due to some combination of very short duration, being a one-off event that was never repeated, being modulated in a way that the receiver was not looking for, or being indistinguishable from terrestrial radio noise. 20%.

Conditional on the above, the "Wow!" signal was such a signal. 20%.

Comment author: [deleted] 16 March 2014 03:29:28AM 1 point [-]

There are other 'technological civilizations' (in the sense of intelligent living things that have learned to manipulate matter in a complicated way) in the observable universe: 99%

What do you mean by “complicated”? Were humans a technological civilization in 1900? In 1700? In 10,000 BC? In 50,000 BC?

Comment author: shokwave 13 March 2014 04:25:46PM 0 points [-]

Even given other technological civilisations existing, putting "matter and energy manipulation tops out a little above our current cutting edge" at 5% is way off.

Comment author: ThisSpaceAvailable 16 March 2014 02:29:57AM 1 point [-]

Way off in which direction?

Comment author: CellBioGuy 14 March 2014 02:00:13AM 0 points [-]

There's a lot you can do on the surface of a clement planet, and a lot you can do in a solar system without replicators that eat everything. Also depends on what you mean by 'above'.

Comment author: Eugine_Nier 13 March 2014 12:27:56AM 4 points [-]

One statement per comment please.

Comment author: CellBioGuy 13 March 2014 01:29:14AM 1 point [-]

Probably overdid it with that one. Will split things up more in subsequent comments.

Comment author: CellBioGuy 12 March 2014 01:51:29PM 1 point [-]

Meta thread

Comment author: Ixiel 13 March 2014 09:16:39AM 3 points [-]

Should we down vote posts with many propositions if we agree with a majority? One? All? There are already two split clusters for me.

Comment author: CellBioGuy 14 March 2014 02:09:33AM 0 points [-]

Hmm. I'd recommend if the split has one that's much stronger go with that vote, otherwise leave it at zero and explain in a comment.

Comment author: Douglas_Knight 13 March 2014 05:22:22AM 5 points [-]

You could do this with polls instead of karma. The advantage of karma is that it provides an incentive for people to play to win. The disadvantage is hiding comments.

Comment author: Fhyve 14 March 2014 04:31:47AM 2 points [-]

I agree. I want to comment on some of the downvoted posts, but I don't want to pay the karma

Comment author: CellBioGuy 14 March 2014 02:10:14AM 1 point [-]

Great idea. I'll put a note in the post so that if anyone ever resurrects this in the future they'll do it that way.

Comment author: John_Maxwell_IV 13 March 2014 11:22:56PM *  1 point [-]

"provides an incentive for people to play to win"

You mean an incentive to hold irrational beliefs? Is that something we want to incentivize?

Comment author: ThisSpaceAvailable 16 March 2014 02:41:34AM 1 point [-]

No, not to hold irrational beliefs, but to admit to holding irrational beliefs.

Comment author: Eugine_Nier 13 March 2014 12:26:13AM 2 points [-]

Warning: the comments section of this post will look odd. The most reasonable comments will have lots of negative karma. Do not be alarmed, it's all part of the plan. In order to participate in this game you should disable any viewing threshold for negatively voted comments.

Unfortunately, since the first irrationality game, the hiding code was changed so that this is no longer possible.

Comment author: falenas108 12 March 2014 04:15:05PM 4 points [-]

This is a time that the system of hiding votes less than -3 is a bad thing. In this thread, downvotes indicate that a belief that people may have thought was rare is actually pretty common on LW, which is something I'm interested in seeing.

Comment author: gwern 12 March 2014 03:43:29PM 8 points [-]

Why are we reviving this at all?

Comment author: bramflakes 12 March 2014 03:42:15PM *  2 points [-]

Am I allowed to post about whether a counterfactual world would be "better" in some sense, if I specify something like "If Y had happened instead of X, the number of excess deaths from then till now would be lower / economic growth would have been better" ? I don't know whether that falls under preferences disguised as beliefs.

Comment author: CellBioGuy 14 March 2014 02:18:56AM 0 points [-]

Perhaps you can try to turn it in a more generalized form?

Comment author: bramflakes 14 March 2014 11:19:11AM 0 points [-]

How do you mean?