This is a simple transformation of standard expected utility formula that I found conceptually interesting.

For simplicity, let's consider a finite discrete probability space with non-zero probability at each point p(x), and a utility function u(x) defined on its sample space. Expected utility of an event A (set of the points of the sample space) is the average value of utility function weighted by probability over the event, and is written as

Related to: The Simple Math of Everything, Your Strength as a Rationalist, Teaching the Unteachable.

Eric Drexler wrote a couple of articles on the importance and methods of obtaining interdisciplinary knowledge:

• How to Understand Everything (and Why)
• How to Learn About Everything

Note that the title above isn't "how to learn everything", but "how to learn about everything". The distinction I have in mind is between knowing the inside of a topic in deep detail — many facts and problem-solving skills — and knowing the structure and context of a topic: essential facts, what problems can be solved by the skilled, and how the topic fits with others.

This knowledge isn't superficial in a survey-course sense: It is about both deep structure and practical applications. Knowing about, in this sense, is crucial to understanding a new problem and what must be learned in more depth in order to solve it.

This topic was discussed intermittently on Overcoming Bias. Basic understanding of many fields allows to recognize how well-understood by science a problem is and to see its place in the structure of scientific knowledge; to develop better intuitive grasp on what's possible and what's not; and to adequately perceive the natural world.

The advice he gives for obtaining general knowledge feels right, even for studying the topics that you intend to eventually understand in depth:

Don't drop a subject because you know you'd fail a test — instead, read other half-understandable journals and textbooks to accumulate vocabulary, perspective, and context.

In this article, I consider the standard biases in drawing factual conclusions that are not related to emotional reactions, and describe a simple model summarizing what goes wrong with the reasoning in these cases, that in turn suggests a way of systematically avoiding this kind of problems.

The following model is used to describe the process of getting from a question to a (potentially biased) answer for the purposes of this article. First, you ask yourself a question. Second, in the context of the question, a data set is presented before your mind, either directly, by you looking at the explicit statements of fact, or indirectly, by associated facts becoming salient to your attention, triggered by the explicit data items or by the question. Third, you construct an intuitive model of some phenomenon, that allows to see its properties, as a result of considering the data set. And finally, you pronounce the answer, that is read out as one of the properties of the model you've just constructed.

This description is meant to present mental paintbrush handles, to refer to the things you can see introspectively, and things you could operate consciously if you choose to.

Most of the biases in the considered class may be seen as particular ways in which you pay attention to a wrong data set, not representative of the phenomenon you model to get to the answer you seek. As a result, the intuitive model gets systematically wrong, and the answer read out from it gets biased. Below I review the specific biases, to identify the ways in which things go wrong in each particular case, and then I summarize the classes of mistakes of reasoning playing major roles in these biases and correspondingly the ways of avoiding those mistakes.

Related to: Well-Kept Gardens Die By Pacifism.

I wrote a script for the greasemonkey extension for Firefox, implementing less painful downvoting. It inserts a button "Vote boo" in addition to "Vote up" and "Vote down" for each comment. Pressing this button has 30% chance of resulting in downvoting the comment, which is on average equivalent to taking 0.3 points of rating. If pressing the button once has no effect, don't press it twice: the action is already performed, resulting in one of the two possible outcomes.

The idea is to lower the level of punishment from downvoting, thus making it easier to downvote average mediocre comments, not just remarkably bad ones. Systematically downvoting mediocre comments should make their expected rating negative, creating an incentive to focus more on making high-quality comments, and punishing systematic mediocrity. At the same time, low penalty for average comments (implemented through stochastic downvoting) allows to still make them freely, which is essential for supporting a discussion. Contributors may see positive rating of good comments as currency for which they can buy a limited number of discussion-supporting average comments.

The "Vote boo" option is not to be taken lightly, one should understand a comment before declaring it mediocre. If you are not sure, don't vote. If comment is visibly a simple passing remark, or of mediocre quality, press the button.

Related to: Dissolving the Question, The Second Law of Thermodynamics, and Engines of Cognition, The Meditation on Curiosity.

The sentence "snow is white" is true if, and only if, snow is white.

-- A. Tarski

Several days ago I've spent a couple of hours trying to teach my 15 year old brother how to properly construct Tarski statements. It's quite nontrivial to get right. Learning to place facts and representations in the separate mental buckets is one of the fundamental tools for a rationalist. In our model of the world, information propagates from object to object, from mind to mind. To ascertain the validity of your belief, you need to research the whole network of factors that led you to attain the belief. The simplest relation is between a fact and its representation, idealized to represent correctness or incorrectness only, without yet worrying about probabilities. The same object or the same property can be interpreted to mean different things in different relations and contexts, indicating the truth of one statement or another, and it's important not to conflate those.

Let's say you are watching news on TV and the next item is an interview with a sasquatch. The sasquatch answers the questions about his family in decent English, with a slight British accent.

What do you actually observe, how should you interpret the data? Did you "see a sasquatch"? Did you learn the facts about sasquatch's family? Is there a fact of the matter, as to whether the sasquatch's daughter is 5 years old, as opposed to 4 or 6?

I'm sure many of you have already seen this performance. Tim Minchin's beat poem "Storm" is about the sceptical, secular understanding of the world, stupidity of quackery and supernatural, weight of dishonesty, and joy in the merely real. Contains strong language.