On Sunday, April 14th, the Boston group held our first Schelling Day celebration. The idea was to open up and share our private selves. It was a rousing success.

 

That doesn't do it justice. Let me try again.

 

By all the stars, you guys. This was beautiful.

 

About fifteen people showed up. Most of us were from the hard core of Boston's rationalist community. Two of us were new to the group. (I'm hopeful this will convince them to start attending our regular meetups.) There was a brief explanation and a few vital clarifying questions before we began the ritual, which went for maybe 90-120 minutes, including a couple of short breaks. All of us spoke at least once.

 

I don't want to go into specifics about what people said, but it was powerful. I learned about sides of my friends I would never have guessed at. People went into depth about issues I had only seen from the surface. I heard things that will make me change my behavior towards my friends. I saw angst and guilt and hope and pain and wild joy. I saw compassion and uncertainty and courage. People said things they had never said before, things I might not have been brave enough even to think in their position. I had tears in my eyes more than once.

 

Speaking went remarkably smoothly. I set a timer for five minutes for each speaker, but it never ran out. (Five minutes is a surprisingly long time.) Partway through, Julia suggested we leave a long moment of silence between speakers, which was a very good idea and I wish I'd done a better job of enforcing it.

 

Afterwards, we had a potluck and mingled in small groups. At first we talked about our revelations, but over time our conversation started drifting towards our usual topics. Next time, in order to keep us on topic, I'll probably try adding more structure to this stage.

 

The other area I wanted to improve was the ritual with the snacks. We had five categories: Struggles, Confessions, Hopes, Joys, and Other. There weren't many Hopes, and there wasn't much distinction between Struggles and Confessions. I'll change this for next time, possibly to Hardships, Joys, Histories, and Other. There's room for improvement in the specific snacks I picked, too.

 

This celebration was the most powerful thing I've experienced since the Solstice megameetup. I don't think I want to do this again soon—it was one of the most exhausting things I've ever done, even if I didn't notice until after I'd left—but I know I want to do it again sometime.

 

 

To everyone who came: I'm so proud of what you did and who you are. Thank you for your courage and sincerity.

After trying to discover why the LW wiki “definition” of Reductionism appeared so biased, I concluded from the responses that it was never really intended as a definition of the Reductionist position itself, but as a summary of what is considered to be wrong with positions critical of Reductionism.

The argument goes like this. “Emergentism”, as the critical view is often called, points out the properties that emerge from a system when it is assembled from its elements, which do not themselves show such a property. From such considerations it points out various ways in which research programmes based on a reductionist approach may distort priorities and underestimate difficulties. So far, this is all a matter of degree and eventually each case must be settled on its merits. However, it gets philosophically sensitive when Emergentists claim that a Reductionist approach may be unable in principle to 'explain' certain emergent properties.

The reponse to this claim (I think) goes like this. (1) The explanatory power of a model is a function of its ingredients. (2) Reductionism includes all the ingredients that actually exist in the real world. Therefore (3) Emergentists must be treating the “emergent properties” as extra ingredients, thereby confusing the “map” with the “territory”. So Reductionism is defined by EY and others as not treating emergent properties as extra ingredients (in effect).

At this point it is important to distinguish “Mind theory” from other fields where Reductionism is debated. In this field, Reductionists apparently regard Emergentism as a form of disguised Vitalism/Dualism - if emergent properties can’t be explained by the physical ingredients, they must exist in some non-physical realm. However, Emergentism can apply equally well to everything from chess playing programs to gearbox vibrations, neither of which involve anything like mysterious spiritual substances, so this can hardly be the whole story. And in fact I would argue that the reverse is the case: Vitalists or “substance Dualists” are actually unconscious Reductionists as well: when they assume an extra ingredient is necessary to account for the things which they believe Physicalism cannot explain, they are still reducing a system to its ingredients. Emergentists by contrast reject premise (1) of the previous paragraph, that the explanatory power of a model is a function of its ingredients. Thus it seems to me that the real difference between Reductionists & Emergentists is a difference over the nature of explanation. So it seems worthwhile looking into some of the different things that can be meant by “explanation”.

For simplicity, let us illustrate this by the banal example of a brickwork bridge. The elements are the bricks and their relative positions. Our reductionist R points out that these are the only elements you need - after all, if you remove all the bricks there is nothing left - and so proposes to become an expert in bricks. Our (Physicalist) Emergentist E suggests that this won’t be of much use without a knowledge of the Arch (an emergent feature). R isn't stupid and agrees that this would be extremely useful but points out that if no expert in Arch Theory is to hand, given the very powerful computer available, such expertise isn’t strictly necessary: it's not an inherent requirement. Simply solving the force balance equations for each brick will establish whether a given structure will fall into the river. Isn’t that an explanation?

Not in my sense, says E, as to start with it doesn’t tell me how the bridge will be designed, only how an existing design will be analysed. So R explains that the computer will generate structures randomly until one is found that satisfies the requirements of equilibrium. When E enquires how stability will be checked. R replies that the force balance will be checked under all possible small deviations from the design position.

E isn’t satisfied. To claim understanding, R must be able to apply the results of the first design to new bridges of different span, but all (s)he can do is repeat the process again every time.

On the contrary, replies R, this being the age of Big Data, the computer can generate solutions in a large number of cases and then use pattern recognition software to extract rules that can be applied to new cases.

Ah, says E, but explaining these rules means hypothesing more general rules from which these rules can be derived, using appropriate Bayesian reasoning to confirm your hypothesis.

OK, replies R, my program has a heuristic feature that has passed the Turing Test. So anything you can do along these lines, it can do just as well.

So using R’s approach, explanation even in E’s most general sense can always be arrived at by a four-stage process: (1) construct a model using the basic elements applicable to the situation, (2) fill a substantial chunk of solution space, (3) use pattern recognition to extract pragmatic rules, (4) use hypothesis generation and testing to derive general principles from the rules. It may be a trivial illustration, but it seems to me that in a broad sense this sort of process must be applicable in almost any situation.

How should we interpret this conclusion? R would say that it proves that “explanation” can be arrived it using a Reductionist model. E would say it proves the inadequacy of Reductionism, since Reductionist steps (1) & (2) have to be supplemented by Integrationist steps (3) & (4): the rules found at step (3) are precisely “emergent features” of the solution space. Moreover, pattern recognition is not a closed-form process with repeatable results. (Is it?) On the other hand the patterns identified in solution space might well be derivable in closed form directly from higher-level characteristics of the system in question (such as constraints in the system).

I would say that the choice of interpretation is a matter of convention, though I own up that I find the Emergentist mind-set more helpful in the fields I have learnt something about. What really matters is a recognition of the huge difference between “providing a solution” and “generalising from solution space” as types of explanation. The “Emergentist” label is a reminder of that difference. But call yourself a “Reductionist” if you like so long as you acknowledge the difference.

It seems to me that the sort of argument sketched here provides useful pointers to help recognize when “Reductionism” becomes “Greedy Reductionism”(A). For example, consider the claim that mapping the Human Connectome will enable the workings of the brain to be explained. Clearly, the mapping is just step (1). Consider the size of the Connectome, and then consider the size of the solution space of its activity. That makes step (1) sound utterly trivial compared with step (2). This leaves the magnitude of steps (3) & (4) to be evaluated. That doesn’t mean the project won’t be extremely valuable, but it puts the time-frame of the claim to provide real “understanding” into a very different light, and underlines the continued value of working at other scales as well.

(A): See e.g. fubarobfusco's comment on my earlier discussion.

Related: Thick and Thin, Loss of local knowledge affecting intellectual trends

An entry I found in the archives on Gregory Cochran's and Henry Harpending's blog West Hunter.

In yet another example of  long-delayed discovery, forms of high-altitude lightning were observed for at least a century before becoming officially real (as opposed to really real).

Some thunderstorms manage to generate blue jets shooting out of their thunderheads, or  glowing red rings and associated tentacles around 70 kilometers up.   C T R Wilson predicted this long ago, back in the 1920s.  He had a simple model that gets you started.

You see, you can think of the thunderstorm, after a ground discharge,  as a vertical dipole. Its electrical field drops as the cube of altitude.  The threshold voltage for atmospheric breakdown is proportional to pressure, while pressure drops exponentially with altitude: and as everyone knows, a negative exponential drops faster than any power.

The curves must cross.   Electrical breakdown occurs.  Weird lightning, way above the clouds.

As I said, people reported sprites at least a hundred years ago, and they have probably been observed occasionally since the dawn of time. However, they’re far easier to see if you’re above the clouds – pilots often do.

Pilots also learned not to talk about it, because nobody listened.   Military and commercial pilots have to pass periodic medical exams known as ‘flight physicals’,  and there was a suspicion that reporting glowing red cephalopods in the sky might interfere with that.  Generally, you had to see the things that were officially real (whether they were really real or not), and only those things.

Sprites became real when someone recorded one by accident on a fast camera in 1989.  Since then it’s turned into a real subject, full of strangeness: turns out that thunderstorms  sometimes generate gamma-rays and even antimatter.

Presumably we’ve gotten over all that ignoring your lying eyes stuff by now.

May you tell others what you see. (~_^)

 

Today's post, Your Price for Joining was originally published on 26 March 2009. A summary (taken from the LW wiki):

 

The game-theoretical puzzle of the Ultimatum game has its reflection in a real-world dilemma: How much do you demand that an existing group adjust toward you, before you will adjust toward it? Our hunter-gatherer instincts will be tuned to groups of 40 with very minimal administrative demands and equal participation, meaning that we underestimate the inertia of larger and more specialized groups and demand too much before joining them. In other groups this resistance can be overcome by affective death spirals and conformity, but rationalists think themselves too good for this - with the result that people in the nonconformist cluster often set their joining prices way way way too high, like an 50-way split with each player demanding 20% of the money. Nonconformists need to move in the direction of joining groups more easily, even in the face of annoyances and apparent unresponsiveness. If an issue isn't worth personally fixing by however much effort it takes, it's not worth a refusal to contribute.

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Sacred Mundane, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

Why give globally? Why give money? Why health charities? Why single-issue organizations? At first glance these all seem like arbitrary choices: what if I would rather volunteer, or donate to local charities? Why does it matter? It comes down to two distributions: cost-effectiveness and income.

 

DALYs per $1000

This shows the cost-effectiveness of a large number of health interventions, with taller bars in cases where we can avert more death and suffering per dollar. The shape of this chart is important: while we can do a lot of good if we pick an intervention at random or support a 'horizontal' effort that works on everything, we can do 300 times better by picking one in the top 10%. This is why single-issue charities make sense: you can pick one that focuses on a top intervention.

(Don't let the small bars on the left fool you: nearly every intervention on that chart is worth doing [1], some are just far more valuable than others.)

This only considers health: what about other ways of helping people? Political advocacy, development, literacy, human rights, why not them? The big thing health has going for it is that we can measure impact, which lets us choose only the best options. In other fields where we can't measure we could end up anywhere on the impact curve.

Let's look at another distribution:

So some people have a lot more money that other people, we knew that, right? But have a look at the scale. Someone earning at the poverty line in the US is richer than 90% of people. This is why giving globally is so powerful: small amounts of your money can mean a huge amount to people who have so much less.

Neither of these distributions are intuitive: we don't feel that rich, and charities all seem kind of interchangeable. But understanding them can make the difference between trying to do good and really succeeding.

(I first saw these charts in a talk by Toby Ord of Giving What We Can (GWWC). The data for the first chart, DALYs per $1000, comes from the DCP2. This was a project that, among other things, compiled cost effectiveness estimates for a very wide range of health interventions. I made the chart from the csv version of the data from here, excluding the ~60 interventions (of 171) that didn't have estimates. The second chart is straight from GWWC's website, and you can read the details there by clicking on footnote 4.)

[1] The median intervention there is $207/DALY, which roughly means it can give someone an extra year of healthy life for $207. Which is an incredible deal, that I think most of us would jump at. And it's less than 5 pixels high.

I also posted this on my blog.

Related: Pinpointing Utility

If I ever say "my utility function", you could reasonably accuse me of cargo-cult rationality; trying to become more rational by superficially immitating the abstract rationalists we study makes about as much sense as building an air traffic control station out of grass to summon cargo planes.

There are two ways an agent could be said to have a utility function:

1. It could behave in accordance with the VNM axioms; always choosing in a sane and consistent manner, such that "there exists a U". The agent need not have an explicit representation of U.

2. It could have an explicit utility function that it tries to expected-maximize. The agent need not perfectly follow the VNM axioms all the time. (Real bounded decision systems will take shortcuts for efficiency and may not achieve perfect rationality, like how real floating point arithmetic isn't associative).

Neither of these is true of humans. Our behaviour and preferences are not consistent and sane enough to be VNM, and we are generally quite confused about what we even want, never mind having reduced it to a utility function. Nevertheless, you still see the occasional reference to "my utility function".

Sometimes "my" refers to "abstract me who has solved moral philosophy and or become perfectly rational", which at least doesn't run afoul of the math, but is probably still wrong about the particulars of what such an abstract idealized self would actually want. But other times it's a more glaring error like using "utility function" as shorthand for "entire self-reflective moral system", which may not even be VNMish.

But this post isn't really about all the ways people misuse terminology, it's about where we're actually at on the whole problem for which a utility function might be the solution.

As above, I don't think any of us have a utility function in either sense; we are not VNM, and we haven't worked out what we want enough to make a convincing attempt at trying. Maybe someone out there has a utility function in the second sense, but I doubt that it actually represents what they would want.

Perhaps then we should speak of what we want in terms of "terminal values"? For example, I might say that it is a terminal value of mine that I should not murder, or that freedom from authority is good.

But what does "terminal value" mean? Usually, it means that the value of something is not contingent on or derived from other facts or situations, like for example, I may value beautiful things in a way that is not derived from what they get me. The recursive chain of valuableness terminates at some set of values.

There's another connotation, though, which is that your terminal values are akin to axioms; not subject to argument or evidence or derivation, and simply given, that there's no point in trying to reconcile them with people who don't share them. This is the meaning people are sometimes getting at when they explain failure to agree with someone as "terminal value differences" or "different set of moral axioms". This is completely reasonable, if and only if that is in fact the nature of the beliefs in question.

About two years ago, it very much felt like freedom from authority was a terminal value for me. Those hated authoritarians and fascists were simply wrong, probably due to some fundamental neurological fault that could not be reasoned with. The very prototype of "terminal value differences".

And yet here I am today, having been reasoned out of that "terminal value", such that I even appreciate a certain aesthetic in bowing to a strong leader.

If that was a terminal value, I'm afraid the term has lost much of its meaning to me. If it was not, if even the most fundamental-seeming moral feelings are subject to argument, I wonder if there is any coherent sense in which I could be said to have terminal values at all.

The situation here with "terminal values" is a lot like the situation with "beliefs" in other circles. Ask someone what they believe in most confidently, and they will take the opportunity to differentiate themselves from the opposing tribe on uncertain controversial issues; god exists, god does not exist, racial traits are genetic, race is a social construct. The pedant answer of course is that the sky is probably blue, and that that box over there is about a meter long.

Likewise, ask someone for their terminal values, and they will take the opportunity to declare that those hated greens are utterly wrong on morality, and blueness is wired into their very core, rather than the obvious things like beauty and friendship being valuable, and paperclips not.

So besides not having a utility function, those aren't your terminal values. I'd be suprised if even the most pedantic answer weren't subject to argument; I don't seem to have anything like a stable and non-negotiable value system at all, and I don't think that I am even especially confused relative to the rest of you.

Instead of a nice consistent value system, we have a mess of intuitions and hueristics and beliefs that often contradict, fail to give an answer, and change with time and mood and memes. And that's all we have. One of the intuitions is that we want to fix this mess.

People have tried to do this "Moral Philosophy" thing before, myself included, but it hasn't generally turned out well. We've made all kinds of overconfident leaps to what turn out to be unjustified conclusions (utilitarianism, egoism, hedonism, etc), or just ended up wallowing in confused despair.

The zeroth step in solving a problem is to notice that we have a problem.

The problem here, in my humble opinion, is that we have no idea what we are doing when we try to do Moral Philosophy. We need to go up a meta-level and get a handle on Moral MetaPhilosophy. What's the problem? What are the relevent knowns? What are the unknowns? What's the solution process?

Ideally, we could do for Moral Philosphy approximately what Bayesian probability theory has done for Epistemology. My moral intuitions are a horrible mess, but so are my epistemic intuitions, and yet we more-or-less know what we are doing in epistemology. A problem like this has been solved before, and this one seems solvable too, if a bit harder.

It might be that when we figure this problem out to the point where we can be said to have a consistent moral system with real terminal values, we will end up with a utility function, but on the other hand, we might not. Either way, let's keep in mind that we are still on rather shaky ground, and at least refrain from believing the confident declarations of moral wisdom that we so like to make.

Moral Philosophy is an important problem, but the way is not clear yet.

This is a summary of an article I'm writing on consciousness, and I'd like to hear opinions on it.  It is the first time anyone has been able to defend a numeric claim about subjective consciousness.

ADDED:  Funny no one pointed out this connection, but the purpose of this article is to create a nonperson predicate.

1. Overview

I propose a test for the absence of consciousness, based on the claim that a necessary, but not sufficient, condition for a symbol-based knowledge system to be considered conscious is that it has exactly one possible symbol grounding, modulo symbols representing qualia. This supposition, plus a few reasonable assumptions, leads to the conclusion that a symbolic artificial intelligence using Boolean truth-values and having an adult vocabulary must have on the order of 10⁶ assertions before we need worry whether it is conscious.

Section 2 will explain the claim about symbol-grounding that this analysis is based on. Section 3 will present the math and some reasonable assumptions for computing the expected number of randomly-satisfied groundings for a symbol system. Section 4 will argue that a Boolean symbol system with a human-level vocabulary must have millions of assertions in order for it to be probable that no spurious symbols groundings exist.

Today's post, Don't Believe You'll Self-Deceive was originally published on 09 March 2009. A summary (taken from the LW wiki):

 

It may be wise to tell yourself that you will not be able to successfully deceive yourself, because by telling yourself this, you may make it true.

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Moore's Paradox, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

The chapter on judgment under uncertainty in the (excellent) new Oxford Handbook of Cognitive Psychology has a handy little section on recent critiques of the "heuristics and biases" tradition. It also discusses problems with the somewhat-competing "fast and frugal heuristics" school of thought, but for now let me just quote the section on heuristics and biases (pp. 608-609):

The heuristics and biases program has been highly influential; however, some have argued that in recent years the influence, at least in psychology, has waned (McKenzie, 2005). This waning has been due in part to pointed critiques of the approach (e.g., Gigerenzer, 1996). This critique comprises two main arguments: (1) that by focusing mainly on coherence standards [e.g. their rationality given the subject's other beliefs, as contrasted with correspondence standards having to do with the real-world accuracy of a subject's beliefs] the approach ignores the role played by the environment or the context in which a judgment is made; and (2) that the explanations of phenomena via one-word labels such as availability, anchoring, and representativeness are vague, insufficient, and say nothing about the processes underlying judgment (see Kahneman, 2003; Kahneman & Tversky, 1996 for responses to this critique).

The accuracy of some of the heuristics proposed by Tversky and Kahneman can be compared to correspondence criteria (availability and anchoring). Thus, arguing that the tradition only uses the “narrow norms” (Gigerenzer, 1996) of coherence criteria is not strictly accurate (cf. Dunwoody, 2009). Nonetheless, responses in famous examples like the Linda problem can be reinterpreted as sensible rather than erroneous if one uses conversational or pragmatic norms rather than those derived from probability theory (Hilton, 1995). For example, Hertwig, Benz and Krauss (2008) asked participants which of the following two statements is more probable:

[X] The percentage of adolescent smokers in Germany decreases at least 15% from current levels by September 1, 2003.

[X&Y] The tobacco tax in Germany is increased by 5 cents per cigarette and the percentage of adolescent smokers in Germany decreases at least 15% from current levels by September 1, 2003.

According to the conjunction rule, [X&Y cannot be more probable than X] and yet the majority of participants ranked the statements in that order. However, when subsequently asked to rank order four statements in order of how well each one described their understanding of X&Y, there was an overwhelming tendency to rank statements like “X and therefore Y” or “X and X is the cause for Y” higher than the simple conjunction “X and Y.” Moreover, the minority of participants who did not commit the conjunction fallacy in the first judgment showed internal coherence by ranking “X and Y” as best describing their understanding in the second judgment.These results suggest that people adopt a causal understanding of the statements, in essence ranking the probability of X, given Y as more probable than X occurring alone. If so, then arguably the conjunction “error” is no longer incorrect. (See Moro, 2009 for extensive discussion of the reasons underlying the conjunction fallacy, including why “misunderstanding” cannot explain all instances of the fallacy.)

The “vagueness” argument can be illustrated by considering two related phenomena: the gambler’s fallacy and the hot-hand (Gigerenzer & Brighton, 2009). The gambler’s fallacy is the tendency for people to predict the opposite outcome after a run of the same outcome (e.g., predicting heads after a run of tails when flipping a fair coin); the hot-hand, in contrast, is the tendency to predict a run will continue (e.g., a player making a shot in basketball after a succession of baskets; Gilovich, Vallone, & Tversky, 1985). Ayton and Fischer (2004) pointed out that although these two behaviors are opposite - ending or continuing runs - they have both been explained via the label “representativeness.” In both cases a faulty concept of randomness leads people to expect short sections of a sequence to be “representative” of their generating process. In the case of the coin, people believe (erroneously) that long runs should not occur, so the opposite outcome is predicted; for the player, the presence of long runs rules out a random process so a continuation is predicted (Gilovich et al., 1985). The “representativeness” explanation is therefore incomplete without specifying a priori which of the opposing prior expectations will result. More important, representativeness alone does not explain why people have the misconception that random sequences should exhibit local representativeness when in reality they do not (Ayton & Fischer, 2004).

 

My thanks to MIRI intern Stephen Barnes for transcribing this text.