In these spheres people generally understand that heuristics optimize for something. Frequently people think they optimize for some ancestral environment that's quite unlike the world we are living in at the moment. I think that's a question where a well written post would be very useful.

This is probably not a novel analogy, but the surprising thing to me is that social psychology tends to frame any "reticle adjustment" as a bias against which we must fight without testing its performance in the contexts under which the adjustment was made.

I would think that many sociologists would say that many people who are racist and look down on Blacks are racists because they don't interact much with Blacks. If the adjustment was made during a time where the person was at an all-White school, the interesting question isn't whether the adjustment performs well within the context of the all-White school but whether it also performs well at decisions made later outside of that heterogeneous environment.

Yup! I agree completely.

If you were modeling an octopus-based sentient species, for the purposes of writing some interesting fiction, then this would be a nice detail to add.

Lottery income is most definitely taxed, although this likely makes little difference to your point.

the fact that I feel that my field is an unfriendly environment for the free exploration of novel or uncommon ideas ... "stereotype accuracy"

Since you are going to spend a lifetime working in this field, you... may have problems.

Ah yes, pausing ghostery seems to fix it.

Clicking the "Donate now" button under "PayPal or Credit Card" does not seem to do anything other than refresh the page.

(browser Firefox 48.0 , OS Ubuntu)

That sounds like "thesis is true" or "thesis is not true" are reasonable positions. Bayesian beliefs have probabilities attached to them.

Sometimes, even people who understand Bayesian reasoning use idiomatic phrases like "believe is true" as a convenient shorthand for "assign a high probability to"! I can see how that might be confusing!

Like I just said, modern science started with an extreme outlier.

There's a lot of history of science and it generally doesn't find that it all hinges on one event like Newton.

My favorite one: burning wood for heat. Better than fossil fuels for the GW problem, but really bad for local air quality.

Does anyone have any insight into VoI plays with Bayesian reasoning?

At a glance, it looks like the VoI is usually not considered from a Bayesian viewpoint, as it is here. For instance, wikipedia says:

""" A special case is when the decision-maker is risk neutral where VoC can be simply computed as; VoC = "value of decision situation with perfect information" - "value of current decision situation" """

From the perspective of avoiding wireheading, an agent should be incentivized to gain information even when this information decreases its (subjective) "value of decision situation". For example, consider a bernoulli 2-armed bandit:

If the agent's prior over the arms is uniform over [0,1], so its current value is .5 (playing arm1), but after many observations, it learns that (with high confidence) arm1 has reward of .1 and arm2 has reward of .2, it should be glad to know this (so it can change to the optimal policy, of playing arm2), BUT the subjective value of this decision situation is less than when it was ignorant, because .2 < .5.

Let E stand for the observation of sabotage

Didn't you mean "the observation of no sabotage"?

This tends to be very context dependent; I don't know enough about biology to estimate. The main caution here is that people tend to forget about regression to the mean (if you have a local measurement X that's only partly related to Y, you should not just port your estimate from X over to Y, but move it closer to what you would have expected from Y beforehand).

You should play if the expected value is positive, and not if it's negative. If the test run results in heads, then the posterior probability is 2/3rds and 24*2/3-12=4, which is positive. If the test run results in tails, then the posterior probability is 1/3rd and 24*1/3-12=-4, which is negative.

(Why is the posterior probability 2/3 or 1/3? Check out footnote 3, or Laplace's Rule of Succession.)

I did not end up using it, although I periodically stumble upon this again and still think it's a neat way of thinking

I certainly wouldn't defend the [...] thesis

"Wouldn't defend" is an interestingly ambiguous phrase!—it could mean "I don't think the thesis is true," or it could mean "I think the thesis is true, but I'm not going to argue for it here." The thing to remember is that the ambiguity is meant for the listener, not the speaker; it's important not to let your sensible caution about what beliefs you're willing to argue for under your True Name distort your model of the true state of reality. And precisely because other people are also cautious about what they're willing to argue for, there could be all sorts of important truths—actionable information that you can use to make important life decisions better—that take special rationality skills to discover, that you won't automatically learn about just by reading what almost everyone says, because almost everyone is too cowardly to just say the Really Obvious Thing.

This, unfortunately, is why you probably won't understand what I'm talking about for another seven years and eight months.

The previous stories are here.

We are told no such thing. We are told it's a fair coin and that can only mean that if you divide up worlds by their probability density, you win in half of them. This is defined.

No, take another look:

in the overwhelming measure of the MWI worlds it gives the same outcome. You don't care about a fraction that sees a different result, in all reality the result is that Omega won't even consider giving you $10000, it only asks for your $100.

So if all pirates implement TDT, what happens?

Necro - I know. However, I'd be willing to bet that few current readers have seen it and we're kind of hurting for new content, so it's probably fine to mine the archives a bit.

That being said, I really enjoyed this article. It seems to check with my own experiences reasonably well and shed some new light on the subject (for me at least). I hadn't really looked at the rack and stack of power to this level of detail nor considered closely where the power of voters really lies. It's also one of the few places where there's a good rational argument for why "Blue Team" "Green Team" is destructive (most of the other content on the site - including the fable of Green and Blue - seem to focus more heavily on the fact that it's annoying when people act irrationally rather than discussing a specific situation where that irrationality is actually harming them).

Interesting stuff.

I'm going to go off the assumption that this post is deliberate satire, and say it's brilliant.

"Even if it's not true, I'm going to decide to believe that people can't sincerely self-deceive."