All of rstarkov's Comments + Replies

Indeed, terse "explanations" that handwave more than explain are a pet peeve of mine. They can be outright confusing and cause more harm than good IMO. See this question on phrasing explanations in physics for some examples.

One useful definition of Bayesian vs Frequentist that I've found is the following. Suppose you run an experiment; you have a hypothesis and you gather some data.

  • if you try to obtain the probability of the data, given your hypothesis (treating the hypothesis as fixed), then you're doing it the frequentist way
  • if you try to obtain the probability of the hypothesis, given the data you have, then you're doing it the Bayesian way.

I'm not sure whether this view holds up to criticism, but if so, I sure find the latter much more interesting than the former.

4waveman
Interesting that, very often, people interpret a frequentist result as though it were Bayesian. E.g. that there is a 90% chance the true value is within the confidence interval. This is so common in medical research that it may possibly be the majority interpretation.
rstarkov180

This has been the most fun, satisfying survey I've ever been part of :) Thanks for posting this. Can't wait to see the results!

One question I'd find interesting is closely related to the probability of life in the universe. Namely, what are the chances that a randomly sampled spacefaring lifeform would have an intelligence similar enough to ours for us to be able to communicate meaningfully, both in its "ways" and in general level of smarts, if we were to meet.

Given that I enjoyed taking part in this, may I suggest that more frequent and in-depth surveys on specialized topics might be worth doing?

Maybe we've finally reached the point where there's no work left to be done

If so, this is superb! This is the end goal. A world in which there is no work left to be done, so we can all enjoy our lives, free from the requirement to work.

The thought that work is desirable has been hammered into our heads so hard that it's a really, really dubious proposition that actually a world where nobody has to work is the ultimate goal. Not one in which everyone works. That world sucks. That's world in which 85% of us live today.

3ikrase
That, and that people are so used to the case in which the only way to earn money is to work for it, that they don't think of the thousand alternatives. (The one that comes most readily to mind is living like a king off the labor of a small family business administering thousands of AI-serfs).

I've first read this about two years ago and it has been an invaluable tool. I'm sure it has saved countless hours of pointless arguments around the world.

When I realise that an inconsistency in how we interpret a specific word is a problem in a certain argument and apply this tool, it instantly transforms arguments which actually are about the meaning of the word to make them a lot more productive (it turns out it can be unobvious that the actual disagreement is about what a specific word means). In other cases it just helps get back on the right track in... (read more)

Addressed by making a few edits to the "Solution" section. Thank you!

All fair points. I did want to post this to main, but decided against it in the end. Didn't know I could move it to main afterwards. Will work on the title, after I've fixed the error pointed out by VincentYu.

I've reviewed the language of the original statement and it seems that the puzzle is set in essentially the real world with two major givens, i.e. facts in which you have 100% confidence.

Given #1: Omega was correct on the last 100 occurrences.

Given #2: Box B is already empty or already full.

There is no leeway left for quantum effects, or for your choice affecting in any way what's in box B. You cannot make box B full by consciously choosing to one-box. The puzzle says so, after all.

If you read it like this, then I don't see why you would possibly one-box.... (read more)

I'm not sure I understand correctly, but let me phrase the question differently: what sort of confidence do we have in "99.9%" being an accurate value for Omega's success rate?

From your previous comment I gather the confidence is absolute. This removes one complication while leaving the core of the paradox intact. I'm just pointing out that this isn't very clear in the original specification of the paradox, and that clearing it up is useful.

To explain why it's important, let me indeed think of an AI like hairyfigment suggested. Suppose someone sa... (read more)

0hairyfigment
Seems to me the language of this rules out faked video. And to explain it as a newsletter scam would, I think, require postulating 2^100 civilizations that have contact with Omega but not each other. Note that we already have some reason to believe that a powerful and rational observer could predict our actions early on. So you tell me what we should expect here.

While I disagree that one-boxing still wins, I'm most interested in seeing the "no future peeking" and the actual Omega success rate being defined as givens. It's important that I can rely on the 99.9% value, rather than wondering whether it is perhaps inferred from their past 100 correct predictions (which could, with a non-negligible probability, have been a fluke).

0FAWS
Why is it important to you that the success rate be a frequentialist probability rather than just a bayesian one?
2hairyfigment
That does indeed seem like the standard version of Newcomb's. (Though I don't understand your last sentence, assuming "non-negligible" does not mean 1/2 to the power of 100.) Can you spell out what you mean by "if" in this context? Because a lot of us are explicitly talking about the best algorithm to program into an AI.
1hairyfigment
As near as I can tell, this depends on dubious assumptions about a mathematical universe. You appear to treat time as fundamental, and yet reject the possibility that reality (or the Matrix) simulates a certain outcome happening at a certain time, not before (as we'd expect if reality calculated the output of a time-dependent wavefunction). In addition, you seem to assume that reality cares about the same aspects of the situation that interest Omega. Otherwise it seems clear that Omega could get an answer sooner by leaving out all the details which don't affect the human-level outcome.
1FAWS
Assume no "future peeking" and Omega only correctly predicting people as difficult to predict as you with 99.9% probability. One-boxing still wins.

To expand a bit on the first paragraph, I feel that such reasonable arguments are to many people about the same as the proof of Poincaré conjecture is to me: I fully understand the proposition, but I'm not nearly smart enough to follow the proof sufficiently well to be confident it's right.

Importantly, I can also follow the outline of the proof, to see how it's intended to work, but this is of course insufficient to establish the validity of the proof.

So the only real reason I happen to trust this proof is that I already have a pre-established trust in the... (read more)

I have found that the logical approach like this one works much more rarely than it doesn't, simply because it appears that people can manage not to trust reason, or to doubt the validity of the (more or less obvious) inferences involved.

Additionally, belief is so emotional that even people who see all the logic, and truly seem to appreciate that believing in God is completely silly, still can't rid themselves of the belief. It's like someone who knows household spiders are not dangerous in any way and yet are more terrified of them than, say, an elephant.... (read more)

2rstarkov
To expand a bit on the first paragraph, I feel that such reasonable arguments are to many people about the same as the proof of Poincaré conjecture is to me: I fully understand the proposition, but I'm not nearly smart enough to follow the proof sufficiently well to be confident it's right. Importantly, I can also follow the outline of the proof, to see how it's intended to work, but this is of course insufficient to establish the validity of the proof. So the only real reason I happen to trust this proof is that I already have a pre-established trust in the community who reviewed the proof. But of course the same is also true of a believer who has a pre-established trust in the theist community. So the guide would require a section on "how to pick authorities to trust", which would explain why it's necessary (impractical to verify everything yourself) and why the scientific community is the best one to trust (highest rate of successful predictions and useful conclusions).

Of course I'd argue that the game of life is not an isolated universe if one can toggle cells in it, and if you consider the whole lot then there's nothing supernatural about the process of cells being toggled.

But this is a good example. I asked about what others mean by "supernatural" and this sounds very close indeed!

Sounds like a reasonable way of putting it. So a weapon shooting invisible (to the human eye) bullets would be classified as "supernatural" by someone from the stone age, because to them, killing someone requires direct contact with a visible weapon or projectile, that has appreciable travel time. Right?

Although "hard science" would have to be excluded from this, even though it contains lots of stuff that doesn't obey the same laws as most stuff we see.

I suppose it's not the most concise post I've ever written. Thanks for the feedback!

So from the negative votes I'm guessing that this is not something you guys find appropriate in "discussion"? It would help me as a newcomer if you also suggested what makes it bad :)

3Normal_Anomaly
I liked it, and it inspired an interesting conversation. It is a bit rambling, but I for one like the format of going step-by-step through the thought process that inspired the post. Tighten up your posts in the future, but don't feel compelled to lose that feature in doing so. Voted up.
3ShardPhoenix
I didn't downvote and it's not an inappropriate topic, but the post is a bit rambling. More thought and editing could allow you to figure out what you're really trying to say, and hence tighten up the post.
2knb
I think it is quite relevant for LW. Voted up.
rstarkov110

Even more important, I think, is the realization that, to decide how much you're willing to bet on a specific outcome, all of the following are essentially the same:

  • you do have the information to calculate it but haven't calculated it yet
  • you don't have the information to calculate it but know how to obtain such information.
  • you don't have the information to calculate it

The bottom line is that you don't know what the next value will be, and that's the only thing that matters.

Thanks for this, it really helped.

it doesn't guarantee that we have time, resources, or inclination to actually calculate it

Here's how I understand this point, that finally made things clearer:

Yes, there exists a more accurate answer, and we might even be able to discover it by investing some time. But until we do, the fact that such an answer exists is completely irrelevant. It is orthogonal to the problem.

In other words, doing the calculations would give us more information to base our prediction on, but knowing that we can do the calculation doesn't... (read more)

0Alex Flint
Cool, glad it was helpful :) Here is one interesting post about how to encourage our brains to output specific probabilities: http://lesswrong.com/lw/3m6/techniques_for_probability_estimates/

Perhaps - obviously each coin is flipped just once, i.e. Binomial(n=1,p), which is the same thing as Bernoulli(p). I was trying to point out that for any other n it would work the same as a normal coin, if someone were to keep flipping it.

And just as it gets really interesting, that chapter ends. There is no solution provided for stage 4 :/

0Perplexed
Odd. The printed book has another page and a half for that chapter, including the solution to Stage 4. (No surprises in the solution - same as stage 3 except you start with 40 fewer Green widgets.)

Bayesianism tells us that there is a unique answer in the form of a probability for the next coin to be heads

I'm obviously new to this whole thing, but is this a largely undebated, widely accepted view on probabilities? That there are NO situations in which you can't meaningfully state a probability?

For example, let's say we have observed 100 samples of a real-valued random variable. We can use the maximum entropy principle, and thus use the normal distribution (whcih is maximal-entropy for unbounded reals). We then use standard methods to estimate popu... (read more)

7Richard_Kennaway
It does seem to be widely accepted and largely undebated. However, it is also widely rejected and largely undebated, for example by Andrew Gelman, Cosma Shalizi, Ken Binmore, and Leonard Savage (to name just the people I happen to have seen rejecting it -- I am not a statistician, so I do not know how representative these are of the field in general, or if there has actually been a substantial debate anywhere). None of them except Ken Binmore actually present arguments against it in the material I have read, they merely dismiss the idea of a universal prior as absurd. But in mathematics, only one thing is absurd, a contradiction, and by that standard only Ken Binmore has offered any mathematical arguments. He gives two in his book "Rational Decisions": one based on Gödel-style self-reference, and the other based on a formalisation of the concept of "knowing that" as the box operator of S5 modal logic. I haven't studied the first but am not convinced by the second, which fails at the outset by defining "I know that" as an extensional predicate. (He identifies a proposition P with the set of worlds in which it is true, and assumes that "I know that P" is a function of the set representing P, not of the syntactic form of P. Therefore by that definition of knowing, since I know that 2+2=4, I know every true statement of mathematics, since they are all true in all possible worlds.) (ETA: Binmore's S5 argument can also be found online here.) (ETA2: For those who don't have a copy of "Rational Decisions" to hand, here's a lengthy and informative review of it.) These people distinguish "small-world" Bayesianism from "large-world" Bayesianism, they themselves being small-worlders. Large-worlders would include Eliezer, Marcus Hutter, and everyone else who believes in the possibility of a universal prior. A typical small-world Bayesian argument would be: I hypothesise that a certain variable has a Gaussian distribution with unknown parameters over which I have a prior dist
2Alex Flint
Actually, yes, but you're right to be surprised because it's (to my mind at least) an incredible result. Cox's theorem establishes this as a mathematical result from the assumption that you want to reason quantitatively and consistently. Jaynes gives a great explanation of this in chapters 1 and 2 of his book "Probability Theory". The short answer is that a probability always reflects your current state of knowledge. If I told you absolutely nothing about the coin or the distribution, then you would be entirely justified in assigning 50% probability to heads (on the basis of symmetry). If I told you the exact distribution over p then you would be justified in assigning a different probability to heads. But in both cases I carried out the same experiment -- it's just that you had different information in the two trials. You are justified in assigning different probabilities because Probability is in the mind. The knowledge you have about the distribution over p is just one more piece of information to roll into your probability. That depends on the probability that the coin flipper chooses a Cauchy distribution. If this were a real experiment then you'd have to take into account unwieldy facts about human psychology, physics of coin flips, and so on. Cox's theorem tells us that in this case there is a unique answer in the form of a probability, but it doesn't guarantee that we have time, resources, or inclination to actually calculate it. If you want to avoid all those kinds of complicated facts then you can start from some reasonable mathematical assumptions such as a normal distribution over p - but if your assumptions are wrong then don't be surprised when your conclusions turn out wrong.

I read this to say that you can't calculate a value that is guaranteed to break even in the long term, because there isn't enough information to do this. (which I tend to agree with)

If I were trying to make a profit then I'd need to know how much to charge for entry. If I could calculate that then yes, I'd offer the bet regardless of how many heads came out of 100 trials.

But this is entirely beside the point; the purpose of this thought experiment is for me to show which parts of bayesianism I don't understand and solicit some feedback on those parts.

In particular, a procedure that I could use to actually pick a break-even price of entry would be very helpful.

0endoself
This is actually essential to the problem. If you would only bet me if the coins came up heads, you could make all the coins heavily biased toward tales. In the rare scenario that 100 coins happened to come up anyways, you could show this to me to try to trick me into accepting, when you know that the next coin, like all the others, is guaranteed to be biased toward tales. For actually solving the problem, I am no expert, but I think Laplace's law of succession applies here. Laplace's law states that when the only information that you have is the fact that there are only two possible results to a process (such as heads and tales) and the results to a number of trials that have already been done, the probability that the next result turns out a specific way is (s+1)/(n+2), where s is the number of times that is happened that way in the past and n is the total number of trials so far. I am not sure if that applies here because we may be working with a bit more information than this, but it might be correct. In this case: P(heads) = 101/102 EV(heads) = 101/102*V(heads) + 1/102*V(tails) = (101*$1000+1*$10)/102 = $101010/102 = $990 5/17 You can read more about Laplace's law, including a derivation, in chapter 6 of Probability Theory: the Logic of Science by Edwin T. Janes.
0[anonymous]
This is actually essential to the problem. If you would only bet me if the coins came up heads, you could make all the coins heavily biased toward tales. In the rare scenario that 100 coins happened to come up anyways, you could show this to me to try to trick me into accepting, when you know that the next coin, like all the others, is guaranteed to be biased toward tales.

You take the evidence, and you decide that you pay X. Then we run it lots of times. You pay X, I pick a random coin and flip it. I pay your winnings. You pay X again, I pick again, etc. X is fixed.

Preferably, let other people play the game first to gather the evidence at no cost to myself.

For the record, this is not permitted.

My take at it is basically this: average over all possible distributions

It's easy to say this but I don't think this works when you start doing the maths to get actual numbers out. Additionally, if you really take ALL possible distributions then you're already in trouble, because some of them are pretty weird - e.g. the Cauchy distribution doesn't have a mean or a variance.

distribution about which we initially don’t kn

... (read more)
0Perplexed
You should probably look at Jaynes's book "Probability Theory: the Language of Science". In particular, I think that the discussion there dealing with the Widget Problem and with Laplace's Rule of Succession may be relevant to your question.
2saturn
If I'm not mistaken, the Cauchy distribution wouldn't be included because it's not supported on a bounded interval.

The properties of the pool are unknown to you, so you have to take into account the possibility that I've tuned them somehow. But you do know that the 100 coins I drew from that pool were drawn fairly and randomly.

0Dorikka
Okay. I retract my permanent answer of $10.
0endoself
Would you have tried to bet us anyways if you had not landed heads 100 times?

I have clarified my post to specify that for each flip, I pick a coin from this infinite pool at random. Suppose you also magically know with absolute certainty that these givens are true. Still $10?

0Dorikka
You still decide what coins are in the infinite pool, right?

This is a good point, and I've pondered on this for a while.

Following your logic: we can observe that I'm not spending all my waking time caring about A (people dying somewhere for some reason). Therefore we can conclude that the death of those people is comparable to mundane things I choose to do instead - i.e. the mundane things are not infinitely less important than someone's death.

But this only holds if my decision to do the mundane things in preference to saving someone's life is rational.

I'm still wondering whether I do the mundane things by rational... (read more)

The original description of the problem doesn't mention if you know of Omega's strategy for deciding what to place in box B, or their success history in predicting this outcome - which is obviously a very important factor.

If you know these things, then the only rational choice, obviously and by a huge margin, is to pick only box B.

If you don't know anything other than box B may or may not contain a million dollars, and you have no reasons to believe that it's unlikely, like in the lottery, then the only rational decision is to take both. This also seems to... (read more)

3sriku
It looks like you just restated the "paradox" - using one argument, it is "obvious" to pick B and using another argument, it is "obvious" to pick both. Also, in general, do try to avoid saying something is "obvious". It usually throws a lot of complexity and potential faults into a black box and worsens your chances of uncovering those faults by intimidating people.
rstarkov-10

I don't know. I don't suppose you claim to know at which point the number of dust specks is small enough that they are preferable to 50 years of torture?

(which is why I think that Idea 2 is a better way to reason about this)

Argh, I have accidentally reported your comment instead of replying. I did wonder why it asks me if I'm sure... Sorry.

It does indeed appear that the only rational approach is for them to be treated as comparable. I was merely trying to suggest a possible underlying basis for people consistently picking dust specks, regardless of the hugeness of the numbers involved.

0Alicorn
You did report it; I've ignored the report and now it is gone.

I think Torture vs Dust Specks makes a hidden assumption that the two things are comparable. It appears that people don't actually think like that; even an infinite amount of dust specks are worse than a single person being tortured or dying. People arbitrarily place some bad things into a category that's infinitely worse than another category.

So, I'd say that you aren't preferring morality; you are simply placing 50 years of torture as infinitely worse than a dust speck; no number people getting dust specks can possibly be worse than 50 years of torture.

6MBlume
The thing is, if you think that A and B aren't comparable, with A>B, and if you don't make some simplifying assumption like "any event with P < 0.01 is unworthy of consideration, no matter how great or awful" or something, then you don't get to ever care about B for a moment. There's always some tiny chance of A that has to completely dominate your decision-making.
4endoself
We defined a dust speck to have nonzero negative utility. If you don't think this describes reality, then you can substitute something else, like a stubbed toe. As long as we can make a series of things, none of which is infinitely worse than the next, we can prove that nothing in the list is infinitely worse than any other. http://lesswrong.com/lw/n3/circular_altruism/u7x presents this well.
2Nornagest
It absolutely assumes that the two are comparable, and most of the smarter objections to it that I've seen invoke some kind of filtering function to zero out the impact of any particular dust speck on some level of comparison. There are a number of objections to this that you could raise in practice: given a random distribution of starting values, for example, an additional dust speck would be sufficient to push a small percentage, but an unimaginably huge quantity, of victims' subjective suffering over any threshold of significance we feel like choosing. I'm not too impressed with any of these responses -- they generally seem to leverage special pleading on some level -- but I've got to admit that they don't have anything wrong with them that the filtering argument doesn't. Welcome to Less Wrong, by the way.
rstarkov-10

Idea 1: dust specks, because on a linear scale (which seems to be always assumed in discussions of utility here) I think 50 years of torture is more than 3^^^3 times worse than a dust speck in one's eye.

Idea 2: dust specks, because most people arbitrarily place bad things into incomparable categories. The death of your loved one is deemed to be infinitely worse than being stuck in an airport for an hour. It is incomparable; any amount of 1 hour waits are less bad than a single loved one dying.

0ata
How much would you have to decrease the amount of torture, or increase the number of dust specks, before the dust specks would be worse?