New meetups (or meetups with a hiatus of more than a year) are happening in:

• First Bristol meetup: 25 May 2013 03:00PM
• Tel Aviv, Israel Meetup - Goal Clarification with special guest Cat from CFAR: 23 May 2013 07:00PM

Other irregularly scheduled Less Wrong meetups are taking place in:

• Atlanta Lesswrong's May Meetup: The Rationality of Social Relationships, Friendship, Love, and Family.: 17 May 2013 07:00PM
• Bielefeld Meetup May 22nd: 22 May 2013 07:00PM
• Berlin Social Meetup: 15 June 2013 05:00PM
• Bratislava lesswrong meetup III: 20 May 2013 06:30PM
• Brussels meetup: 18 May 2013 01:00PM
• Durham/RTLW HPMoR discussion, ch. 65-68: 18 May 2013 12:30PM
• London Meetup: 26th May: 26 May 2013 02:00PM
• [Moscow] Belief cleaning: 26 May 2013 04:00PM
• Paris Meetup: Sunday, May 26.: 26 May 2013 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

• Austin, TX: 18 May 2019 01:30PM
• Seattle-Vancouver Kilomeetup: 18 May 2013 11:54AM
• Vienna meetup #3: 18 May 2013 04:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, Madison WI, Melbourne, Mountain View, New York, Ohio, Portland, Salt Lake City, Seattle, Toronto, Vienna, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

Thus spake Eliezer:

A Munchkin is the sort of person who, faced with a role-playing game, reads through the rulebooks over and over until he finds a way to combine three innocuous-seeming magical items into a cycle of infinite wish spells.  Or who, in real life, composes a surprisingly effective diet out of drinking a quarter-cup of extra-light olive oil at least one hour before and after tasting anything else.  Or combines liquid nitrogen and antifreeze and life-insurance policies into a ridiculously cheap method of defeating the invincible specter of unavoidable Death.  Or figures out how to build the real-life version of the cycle of infinite wish spells.

It seems that many here might have outlandish ideas for ways of improving our lives. For instance, a recent post advocated installing really bright lights as a way to boost alertness and productivity. We should not adopt such hacks into our dogma until we're pretty sure they work; however, one way of knowing whether a crazy idea works is to try implementing it, and you may have more ideas than you're planning to implement.

So: please post all such lifehack ideas! Even if you haven't tried them, even if they seem unlikely to work. Post them separately, unless some other way would be more appropriate. If you've tried some idea and it hasn't worked, it would be useful to post that too.

Followup to:  Pascal's Mugging: Tiny Probabilities of Vast Utilities, The Pascal's Wager Fallacy Fallacy, Being Half-Rational About Pascal's Wager Is Even Worse

Short form:  Pascal's Muggle

tl;dr:  If you assign superexponentially infinitesimal probability to claims of large impacts, then apparently you should ignore the possibility of a large impact even after seeing huge amounts of evidence.  If a poorly-dressed street person offers to save 10^{(10^100)} lives (googolplex lives) for $5 using their Matrix Lord powers, and you claim to assign this scenario less than 10^{-(10^100)} probability, then apparently you should continue to believe absolutely that their offer is bogus even after they snap their fingers and cause a giant silhouette of themselves to appear in the sky.  For the same reason, any evidence you encounter showing that the human species could create a sufficiently large number of descendants - no matter how normal the corresponding laws of physics appear to be, or how well-designed the experiments which told you about them - must be rejected out of hand.  There is a possible reply to this objection using Robin Hanson's anthropic adjustment against the probability of large impacts, and in this case you will treat a Pascal's Mugger as having decision-theoretic importance exactly proportional to the Bayesian strength of evidence they present you, without quantitative dependence on the number of lives they claim to save.  This however corresponds to an odd mental state which some, such as myself, would find unsatisfactory.  In the end, however, I cannot see any better candidate for a prior than having a leverage penalty plus a complexity penalty on the prior probability of scenarios.

In late 2007 I coined the term "Pascal's Mugging" to describe a problem which seemed to me to arise when combining conventional decision theory and conventional epistemology in the obvious way.  On conventional epistemology, the prior probability of hypotheses diminishes exponentially with their complexity; if it would take 20 bits to specify a hypothesis, then its prior probability receives a 2^-20 penalty factor and it will require evidence with a likelihood ratio of 1,048,576:1 - evidence which we are 1048576 times more likely to see if the theory is true, than if it is false - to make us assign it around 50-50 credibility.  (This isn't as hard as it sounds.  Flip a coin 20 times and note down the exact sequence of heads and tails.  You now believe in a state of affairs you would have assigned a million-to-one probability beforehand - namely, that the coin would produce the exact sequence HTHHHHTHTTH... or whatever - after experiencing sensory data which are more than a million times more probable if that fact is true than if it is false.)  The problem is that although this kind of prior probability penalty may seem very strict at first, it's easy to construct physical scenarios that grow in size vastly faster than they grow in complexity.

I originally illustrated this using Pascal's Mugger:  A poorly dressed street person says "I'm actually a Matrix Lord running this world as a computer simulation, along with many others - the universe above this one has laws of physics which allow me easy access to vast amounts of computing power.  Just for fun, I'll make you an offer - you give me five dollars, and I'll use my Matrix Lord powers to save 3↑↑↑↑3 people inside my simulations from dying and let them live long and happy lives" where ↑ is Knuth's up-arrow notation.  This was originally posted in 2007, when I was a bit more naive about what kind of mathematical notation you can throw into a random blog post without creating a stumbling block.  (E.g.:  On several occasions now, I've seen someone on the Internet approximate the number of dust specks from this scenario as being a "billion", since any incomprehensibly large number equals a billion.)  Let's try an easier (and way smaller) number instead, and suppose that Pascal's Mugger offers to save a googolplex lives, where a googol is 10¹⁰⁰ (a 1 followed by a hundred zeroes) and a googolplex is 10 to the googol power, so 10¹⁰¹⁰⁰ or 10^{10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000} lives saved if you pay Pascal's Mugger five dollars, if the offer is honest.

Here's another installment of rationality quotes. The usual rules apply:

Summary: Intelligence Explosion Microeconomics (pdf) is 40,000 words taking some initial steps toward tackling the key quantitative issue in the intelligence explosion, "reinvestable returns on cognitive investments": what kind of returns can you get from an investment in cognition, can you reinvest it to make yourself even smarter, and does this process die out or blow up? This can be thought of as the compact and hopefully more coherent successor to the AI Foom Debate of a few years back.

(Sample idea you haven't heard before:  The increase in hominid brain size over evolutionary time should be interpreted as evidence about increasing marginal fitness returns on brain size, presumably due to improved brain wiring algorithms; not as direct evidence about an intelligence scaling factor from brain size.)

I hope that the open problems posed therein inspire further work by economists or economically literate modelers, interested specifically in the intelligence explosion qua cognitive intelligence rather than non-cognitive 'technological acceleration'.  MIRI has an intended-to-be-small-and-technical mailing list for such discussion.  In case it's not clear from context, I (Yudkowsky) am the author of the paper.

Abstract:

I. J. Good's thesis of the 'intelligence explosion' is that a sufficiently advanced machine intelligence could build a smarter version of itself, which could in turn build an even smarter version of itself, and that this process could continue enough to vastly exceed human intelligence.  As Sandberg (2010) correctly notes, there are several attempts to lay down return-on-investment formulas intended to represent sharp speedups in economic or technological growth, but very little attempt has been made to deal formally with I. J. Good's intelligence explosion thesis as such.

I identify the key issue as returns on cognitive reinvestment - the ability to invest more computing power, faster computers, or improved cognitive algorithms to yield cognitive labor which produces larger brains, faster brains, or better mind designs.  There are many phenomena in the world which have been argued as evidentially relevant to this question, from the observed course of hominid evolution, to Moore's Law, to the competence over time of machine chess-playing systems, and many more.  I go into some depth on the sort of debates which then arise on how to interpret such evidence.  I propose that the next step forward in analyzing positions on the intelligence explosion would be to formalize return-on-investment curves, so that each stance can say formally which possible microfoundations they hold to be falsified by historical observations already made.  More generally, I pose multiple open questions of 'returns on cognitive reinvestment' or 'intelligence explosion microeconomics'.  Although such questions have received little attention thus far, they seem highly relevant to policy choices affecting the outcomes for Earth-originating intelligent life.

The dedicated mailing list will be small and restricted to technical discussants.

Related to: Privileging the Hypothesis

Remember the exercises in critical reading you did in school, where you had to look at a piece of writing and step back and ask whether the author was telling the whole truth? If you really want to be a critical reader, it turns out you have to step back one step further, and ask not just whether the author is telling the truth, but why he's writing about this subject at all.

-- Paul Graham

There's an old saying in the public opinion business: we can't tell people what to think, but we can tell them what to think about.

-- Doug Henwood

Many philosophers—particularly amateur philosophers, and ancient philosophers—share a dangerous instinct: If you give them a question, they try to answer it.

-- Eliezer Yudkowsky

Here are some political questions that seem to commonly get discussed in US media: should gay marriage be legal? Should Congress pass stricter gun control laws? Should immigration policy be tightened or relaxed? 

These are all examples of what I'll call privileged questions (if there's an existing term for this, let me know): questions that someone has unjustifiably brought to your attention in the same way that a privileged hypothesis unjustifiably gets brought to your attention. The questions above are probably not the most important questions we could be answering right now, even in politics (I'd guess that the economy is more important). Outside of politics, many LWers probably think "what can we do about existential risks?" is one of the most important questions to answer, or possibly "how do we optimize charity?" 

Why has the media privileged these questions? I'd guess that the media is incentivized to ask whatever questions will get them the most views. That's a very different goal from asking the most important questions, and is one reason to stop paying attention to the media. 

The problem with privileged questions is that you only have so much attention to spare. Attention paid to a question that has been privileged funges against attention you could be paying to better questions. Even worse, it may not feel from the inside like anything is wrong: you can apply all of the epistemic rationality in the world to answering a question like "should Congress pass stricter gun control laws?" and never once ask yourself where that question came from and whether there are better questions you could be answering instead.

I suspect this is a problem in academia too. Richard Hamming once gave a talk in which he related the following story:

Over on the other side of the dining hall was a chemistry table. I had worked with one of the fellows, Dave McCall; furthermore he was courting our secretary at the time. I went over and said, "Do you mind if I join you?" They can't say no, so I started eating with them for a while. And I started asking, "What are the important problems of your field?" And after a week or so, "What important problems are you working on?" And after some more time I came in one day and said, "If what you are doing is not important, and if you don't think it is going to lead to something important, why are you at Bell Labs working on it?" I wasn't welcomed after that; I had to find somebody else to eat with!

Academics answer questions that have been privileged in various ways: perhaps the questions their advisor was interested in, or the questions they'll most easily be able to publish papers on. Neither of these are necessarily well-correlated with the most important questions. 

So far I've found one tool that helps combat the worst privileged questions, which is to ask the following counter-question:

What do I plan on doing with an answer to this question?

With the worst privileged questions I frequently find that the answer is "nothing," sometimes with the follow-up answer "signaling?" That's a bad sign. (Edit: but "nothing" is different from "I'm just curious," say in the context of an interesting mathematical or scientific question that isn't motivated by a practical concern. Intellectual curiosity can be a useful heuristic.)

(I've also found the above counter-question generally useful for dealing with questions. For example, it's one way to notice when a question should be dissolved, and asked of someone else it's one way to help both of you clarify what they actually want to know.)

Just before the Trinity test, Enrico Fermi decided he wanted a rough estimate of the blast's power before the diagnostic data came in. So he dropped some pieces of paper from his hand as the blast wave passed him, and used this to estimate that the blast was equivalent to 10 kilotons of TNT. His guess was remarkably accurate for having so little data: the true answer turned out to be 20 kilotons of TNT.

Fermi had a knack for making roughly-accurate estimates with very little data, and therefore such an estimate is known today as a Fermi estimate.

Why bother with Fermi estimates, if your estimates are likely to be off by a factor of 2 or even 10? Often, getting an estimate within a factor of 10 or 20 is enough to make a decision. So Fermi estimates can save you a lot of time, especially as you gain more practice at making them.

Estimation tips

These first two sections are adapted from Guestimation 2.0.

Dare to be imprecise. Round things off enough to do the calculations in your head. I call this the spherical cow principle, after a joke about how physicists oversimplify things to make calculations feasible:

Milk production at a dairy farm was low, so the farmer asked a local university for help. A multidisciplinary team of professors was assembled, headed by a theoretical physicist. After two weeks of observation and analysis, the physicist told the farmer, "I have the solution, but it only works in the case of spherical cows in a vacuum."

By the spherical cow principle, there are 300 days in a year, people are six feet (or 2 meters) tall, the circumference of the Earth is 20,000 mi (or 40,000 km), and cows are spheres of meat and bone 4 feet (or 1 meter) in diameter.

Decompose the problem. Sometimes you can give an estimate in one step, within a factor of 10. (How much does a new compact car cost? $20,000.) But in most cases, you'll need to break the problem into several pieces, estimate each of them, and then recombine them. I'll give several examples below.

Estimate by bounding. Sometimes it is easier to give lower and upper bounds than to give a point estimate. How much time per day does the average 15-year-old watch TV? I don't spend any time with 15-year-olds, so I haven't a clue. It could be 30 minutes, or 3 hours, or 5 hours, but I'm pretty confident it's more than 2 minutes and less than 7 hours (400 minutes, by the spherical cow principle).

Can we convert those bounds into an estimate? You bet. But we don't do it by taking the average. That would give us (2 mins + 400 mins)/2 = 201 mins, which is within a factor of 2 from our upper bound, but a factor 100 greater than our lower bound. Since our goal is to estimate the answer within a factor of 10, we'll probably be way off.

Instead, we take the geometric mean — the square root of the product of our upper and lower bounds. But square roots often require a calculator, so instead we'll take the approximate geometric mean (AGM). To do that, we average the coefficients and exponents of our upper and lower bounds.

So what is the AGM of 2 and 400? Well, 2 is 2×10⁰, and 400 is 4×10². The average of the coefficients (2 and 4) is 3; the average of the exponents (0 and 2) is 1. So, the AGM of 2 and 400 is 3×10¹, or 30. The precise geometric mean of 2 and 400 turns out to be 28.28. Not bad.

What if the sum of the exponents is an odd number? Then we round the resulting exponent down, and multiply the final answer by three. So suppose my lower and upper bounds for how much TV the average 15-year-old watches had been 20 mins and 400 mins. Now we calculate the AGM like this: 20 is 2×10¹, and 400 is still 4×10². The average of the coefficients (2 and 4) is 3; the average of the exponents (1 and 2) is 1.5. So we round the exponent down to 1, and we multiple the final result by three: 3(3×10¹) = 90 mins. The precise geometric mean of 20 and 400 is 89.44. Again, not bad.

Sanity-check your answer. You should always sanity-check your final estimate by comparing it to some reasonable analogue. You'll see examples of this below.

Use Google as needed. You can often quickly find the exact quantity you're trying to estimate on Google, or at least some piece of the problem. In those cases, it's probably not worth trying to estimate it without Google.

In the early days of the Center for Applied Rationality, Anna Salamon and I had a disagreement about whether we were ready to run our first applied rationality workshops in six weeks. My inside view said "No way"; Her inside view said "Should be fine"; My outside view noted that Anna had more relevant experience than I did, and therefore cowed my inside view into grudgingly shutting up.

It turned out well. Granted, the first couple of workshops were a bit chaotic (hey, sleeping in a dogpile on the living room builds character, amiright May minicampers?). But it's clear in retrospect that we got a lot more value out of diving in than we would have from the extra time spent planning.

The "try stuff fast" habit is responsible for a lot of the techniques in our curriculum; we test out classes on each other and on volunteers, observe "Oh hey, this helps other people too" or "Oh hey, no one else thinks this is useful, turns out I'm just weird," and tweak our curriculum accordingly.

And because we cannot help going recursively meta, we've built a lot of material into our curriculum to make people better at trying things that could make them better at pursuing their goals. Quick, off-the-cuff value of information (VOI) calculations help you decide when it's worth it to spend the time, or money or risk, to try something new. Againstness helps you notice and alleviate the stress responses that can keep you from trying something, once you've noticed that you should. Comfort zone expansion is basically a "try a bunch of new things" drill.

For more details on our curriculum, check out a sample schedule. I also made a simplified map of some of our classes, so you can see how I think of them fitting into the bigger picture of rationality (click to enlarge):

To the extent that I've improved my own rationality skills over the last year, I give a lot of credit to "try stuff fast." Like many Less Wrongers I have historically been more of a "thinking about things" person than a "trying stuff fast" person; given the choice of an afternoon spent debating ignorance priors or one spent figuring out how to improve my public speaking skills, I'd pick the former every time, even though the latter would be more useful to me.

I'm partially reformed now, thanks in part to the influence of Anna, whom you'll frequently overhear saying things like "I think I'll try teaching the class as if I were Val" or "We should try a different meeting format today, it's high VOI." So now I'm much more likely to notice, "Hey, in this situation I always do X (e.g., ask for feedback later, by email), so this time let me try X-prime (e.g., ask for feedback in person on the spot) -- the cost is low and it's plausible I'll learn that I like it better than my default."

In that spirit, I recommend coming to one of our upcoming workshops in April, May or July, where you will not only be introduced to all the stuff that we've tried and found promising so far, but will also be plugged into a growing network of several hundred other thoughtful and creative people who have developed their own habits you can borrow and try (we certainly do – past participants have been the origin of some of our best material). And being surrounded by other people with similar aspirations, during the workshop and in the alumni network afterwards, is the best way I know of to keep your motivation and your discipline strong.

At $3900, it's an investment, but a low-risk one, since we have a money-back guarantee. If you don't feel like what you got out of it was worth it, we'll refund your money without hesitation or complaint.

Here are the basics:

You can apply here for any of our next three applied rationality workshops:

• Friday, April 26 - Monday, April 29
• Friday, May 17 - Monday, May 20
• Saturday, July 20 - Tuesday, July 23

Each workshop will consist of an immersive four days at a retreat near San Francisco, training you in the art of actually using rationality. That means figuring out what your goals are, and what you can be doing to pursue them more effectively; noticing when you're acting out of habit or impulse; cultivating curiosity about the world and how it works; and learning to use both your intuitive (System 1) and analytical (System 2) thinking systems to their fullest.

We're soliciting applications not just from Less Wrongers, but from other entrepreneurs, students, teachers, scientists, engineers, activists -- anyone who is analytical, friendly, and motivated to make their own careers, personal lives, and/or societies better.  

For more information on our content, check out our workshop webpage, our checklist of rationality habits, or a detailed sample schedule.

We're constantly tinkering with our curriculum (as mentioned earlier), and collecting follow-up data on what works well. So while you should be aware that our material hasn't yet been subjected to rigorous long-term studies, our alumni do tend to report that they've gotten a lot of value out of their experience. Here are a few write-ups from Less Wrongers about their CFAR workshop experience and any changes they've made as a result: toner, palladias, Qiaochu_Yuan, thejash, BrandonReinhart, ciphergoth, and a bunch of other people.

The total cost is $3900, and that includes:

• Three days of classes -- Six hours of class a day, with small class sizes (4-6 people) so you get a lot of personal attention from the instructors. We rearrange those small groups several times throughout the workshop to give you a chance to get to know everyone.
• One day of practice – Optional but recommended, so instructors can help you make and troubleshoot a plan to use the material going forward. (If you choose to skip this day, the total cost is $3400.)
• Six weeks of personal follow-ups – Talk to our staff in one-on-one follow-ups to help you get the most value out of what you've learned.
• Staying on site – We rent out lovely retreat centers (lodging and food included in the cost of the workshop) so you can get to know the instructors and other participants in the evenings, during meals, and on breaks. Evenings include everything from unconferences, to parties, to impromptu Rubix-cube lessons.
• An alumni network -- You'll be included in all future CFAR alumni events, parties, online forums, and so on. We'll make every effort to connect you to alumni from other workshops with whom we think you'll hit it off or have opportunities for collaboration. 

Scholarships and financial aid are available -- including for many who thought they wouldn't qualify.  So if you're interested in attending, definitely apply, and mention you'd like to be considered for this. We'll set up a call to discuss.

And please don't hesitate to email me (Julia at appliedrationality dot org). CFAR staff will also be in this comment thread to field questions, and some of the alumni who frequent Less Wrong may be there as well. 

Apply here (the form takes less than 10 minutes, so you should do it now rather than planning on getting to it later!).

Another monthly installment of the rationality quotes thread. The usual rules apply: