Kim Suozzi was a neuroscience student with brain cancer who wanted to be cryonically preserved but lacked the funds. She appealed to reddit and a foundation was set up, called the Society for Venturism.  Enough money was raised, and when she died on the January 17th, she was preserved by Alcor.  

I wasn't sure if I should post about this, but I was glad to see that enough money was raised and it was discussed on LessWrong here, here, and here.

 

Source

 

Edit:  It looks like Alcor actually worked with her to lower the costs, and waived some of the fees.

Edit 2:  The Society for Venturism has been around for a while, and wasn't set up just for her.

From pg812-1020 of Chapter 8 “Sufficiency, Ancillarity, And All That” of Probability Theory: The Logic of Science by E.T. Jaynes:

The classical example showing the error of this kind of reasoning is the fable about the height of the Emperor of China. Supposing that each person in China surely knows the height of the Emperor to an accuracy of at least ±1 meter, if there are N=1,000,000,000 inhabitants, then it seems that we could determine his height to an accuracy at least as good as

(8-49)

merely by asking each person’s opinion and averaging the results.

The absurdity of the conclusion tells us rather forcefully that the rule is not always valid, even when the separate data values are causally independent; it requires them to be logically independent. In this case, we know that the vast majority of the inhabitants of China have never seen the Emperor; yet they have been discussing the Emperor among themselves and some kind of mental image of him has evolved as folklore. Then knowledge of the answer given by one does tell us something about the answer likely to be given by another, so they are not logically independent. Indeed, folklore has almost surely generated a systematic error, which survives the averaging; thus the above estimate would tell us something about the folklore, but almost nothing about the Emperor.

We could put it roughly as follows:

error in estimate = (8-50)

where S is the common systematic error in each datum, R is the RMS ‘random’ error in the individual data values. Uninformed opinions, even though they may agree well among themselves, are nearly worthless as evidence. Therefore sound scientific inference demands that, when this is a possibility, we use a form of probability theory (i.e. a probabilistic model) which is sophisticated enough to detect this situation and make allowances for it.

As a start on this, equation (8-50) gives us a crude but useful rule of thumb; it shows that, unless we know that the systematic error is less than about of the random error, we cannot be sure that the average of a million data values is any more accurate or reliable than the average of ten¹. As Henri Poincare put it: “The physicist is persuaded that one good measurement is worth many bad ones.” This has been well recognized by experimental physicists for generations; but warnings about it are conspicuously missing in the “soft” sciences whose practitioners are educated from those textbooks.

Or pg1019-1020 Chapter 10 “Physics of ‘Random Experiments’”:

…Nevertheless, the existence of such a strong connection is clearly only an ideal limiting case unlikely to be realized in any real application. For this reason, the law of large numbers and limit theorems of probability theory can be grossly misleading to a scientist or engineer who naively supposes them to be experimental facts, and tries to interpret them literally in his problems. Here are two simple examples:

1. Suppose there is some random experiment in which you assign a probability p for some particular outcome A. It is important to estimate accurately the fraction f of times A will be true in the next million trials. If you try to use the laws of large numbers, it will tell you various things about f; for example, that it is quite likely to differ from p by less than a tenth of one percent, and enormously unlikely to differ from p by more than one percent. But now, imagine that in the first hundred trials, the observed frequency of A turned out to be entirely different from p. Would this lead you to suspect that something was wrong, and revise your probability assignment for the 101’st trial? If it would, then your state of knowledge is different from that required for the validity of the law of large numbers. You are not sure of the independence of different trials, and/or you are not sure of the correctness of the numerical value of p. Your prediction of f for a million trials is probably no more reliable than for a hundred.
2. The common sense of a good experimental scientist tells him the same thing without any probability theory. Suppose someone is measuring the velocity of light. After making allowances for the known systematic errors, he could calculate a probability distribution for the various other errors, based on the noise level in his electronics, vibration amplitudes, etc. At this point, a naive application of the law of large numbers might lead him to think that he can add three significant figures to his measurement merely by repeating it a million times and averaging the results. But, of course, what he would actually do is to repeat some unknown systematic error a million times. It is idle to repeat a physical measurement an enormous number of times in the hope that “good statistics” will average out your errors, because we cannot know the full systematic error. This is the old “Emperor of China” fallacy…

Indeed, unless we know that all sources of systematic error - recognized or unrecognized - contribute less than about one-third the total error, we cannot be sure that the average of a million measurements is any more reliable than the average of ten. Our time is much better spent in designing a new experiment which will give a lower probable error per trial. As Poincare put it, “The physicist is persuaded that one good measurement is worth many bad ones.”² In other words, the common sense of a scientist tells him that the probabilities he assigns to various errors do not have a strong connection with frequencies, and that methods of inference which presuppose such a connection could be disastrously misleading in his problems.

I excerpted & typed up these quotes for use in my DNB FAQ appendix on systematic problems; the applicability of Jaynes’s observations to things like publication bias is obvious. See also http://lesswrong.com/lw/g13/against_nhst/

Related to: Generalizing from One Example, Connecting Your Beliefs (a call for help), Beware the Unsurprised

The idea of this article is something I've talked about a couple of times in comments. It seems to require more attention.

As a general rule, what is obvious to some people may not be obvious to others. Is this obvious to you? Maybe it was. Maybe it wasn't, and you thought it was because of hindsight bias.

Imagine a substantive Less Wrong comment. It's insightful, polite, easy to understand, and otherwise good. Ideally, you upvote this comment. Now imagine the same comment, only with "obviously" in front. This shouldn't change much, but it does. This word seems to change the comment in multifarious bad ways that I'd rather not try to list.

Uncharitably, I might reduce this whole phenomenon to an example of a mind projection fallacy. The implicit deduction goes like this: "I found <concept> obvious. Thus, <concept> is inherently obvious." The problem is that obviousness, like probability, is in the mind.

The stigma of "obvious" ideas has another problem in preventing things from being said at all. I don't know how common this is, but I've actually been afraid of saying things that I thought were obvious, even though ignoring this fear and just posting has yet to result in a poorly-received comment. (That is, in fact, why I'm writing this.)

Even tautologies, which are always obvious in retrospect, can be hard to spot. How many of us would have explicitly realized the weak anthropic principle without Nick Bostrom's help?

And what about implications of beliefs you already hold? These should be obvious, and sometimes are, but our brains are notoriously bad at putting two and two together. Luke's example was not realizing that an intelligence explosion was imminent until he read the I.J. Good paragraph. I'm glad he provided that example, as it has saved me the trouble of making one.

This is not (to paraphrase Eliezer) a thunderbolt of insight. I bring it up because I propose a few community norms based on the idea:

• Don't be afraid of saying something because it's "obvious". It's like how your teachers always said there are no stupid questions.
• Don't burden your awesome ideas with "obvious but it needs to be said".
• Don't vote down a comment because it says something "obvious" unless you've thought about it for a while. Also, don't shun "obvious" ideas.
• Don't call an idea obvious as though obviousness were an inherent property of the idea. Framing it as a personally obvious thing can be a more accurate way of saying what you're trying to say, but it's hard to do this without looking arrogant. (I suspect this is actually one of the reasons we implicitly treat obviousness as impersonal.)

I'm not sure if these are good ideas, but I think implementing them would decrease the volume of thoughts we cannot think and things we can't say.

So apparently Richard Feynman once said:

If, in some cataclysm, all scientific knowledge were to be destroyed, and only one sentence passed on to the next generation of creatures, what statement would contain the most information in the fewest words? I believe it is the atomic hypothesis (or atomic fact, or whatever you wish to call it) that all things are made of atoms — little particles that move around in perpetual motion, attracting each other when they are a little distance apart, but repelling upon being squeezed into one another. In that one sentence you will see an enormous amount of information about the world, if just a little imagination and thinking are applied.

I could be missing something, but this strikes me as a terrible answer.

When was the atomic hypothesis confirmed?  If I recall correctly, it was only when chemists started noticing that the outputs of chemical reactions tended to factorize a certain way, which is to say that it took millennia after Democritus to get the point where the atomic hypothesis started making clearly relevant experimental predictions.

How about, "Stop trying to sound wise and come up with theories that make precise predictions about things you can measure in numbers."

I noticed this on Marginal Revolution, so I shall also state my candidate for the one most important sentence about macroeconomics:  "You can't eat gold, so figure out how the heck money is relevant to making countries actually produce more or less food."  This is a pretty large advance on how kings used to think before economics.  I mean, Scott Sumner is usually pretty savvy (so is Richard Feynman btw) but his instruction to try to understand money is likely to fall on deaf ears, if it's just that one sentence.  Think about money?  Everyone wants more money!  Yay, money!  Let's build more gold mines!  And "In the short run, governments are not households"?  Really, Prof. Cowen, that's what you'd pass on to the next generation as they climb up from the radioactive soil?

*Cough.*  Okay, I'm done.  Does anyone want to take their own shot at doing better than Feynman did for their own discipline?

New proposed censorship policy:

Any post or comment which advocates or 'asks about' violence against sufficiently identifiable real people or groups (as opposed to aliens or hypothetical people on trolley tracks) may be deleted, along with replies that also contain the info necessary to visualize violence against real people.

Reason: Talking about such violence makes that violence more probable, and makes LW look bad; and numerous message boards across the Earth censor discussion of various subtypes of proposed criminal activity without anything bad happening to them.

More generally: Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people (e.g., kidnapping, not anti-marijuana laws) may at the admins' option be censored on the grounds that it makes LW look bad and that anyone talking about a proposed crime on the Internet fails forever as a criminal (i.e., even if a proposed conspiratorial crime were in fact good, there would still be net negative expected utility from talking about it on the Internet; if it's a bad idea, promoting it conceptually by discussing it is also a bad idea; therefore and in full generality this is a low-value form of discussion).  

This is not a poll, but I am asking in advance if anyone has non-obvious consequences they want to point out or policy considerations they would like to raise. In other words, the form of this discussion is not 'Do you like this?' - you probably have a different cost function from people who are held responsible for how LW looks as a whole - but rather, 'Are there any predictable consequences we didn't think of that you would like to point out, and possibly bet on with us if there's a good way to settle the bet?'

Yes, a post of this type was just recently made.  I will not link to it, since this censorship policy implies that it will shortly be deleted, and reproducing the info necessary to say who was hypothetically targeted and why would be against the policy.

This post is prompted by the multitude of posts and comments here using quantum this and that in an argument (quantum dice, quantum immortality, quantum many worlds...). But how does one know if they understand the concept they use? In school a student would have to write a test and get graded. It strikes me as a reasonable thing to do here, as well: let people test their understanding of the material so that they can calibrate their estimate of their knowledge of the topic. This is an attempt to do just that.

Let's look at one of the very first experiments demonstrating that in the microscopic world things are usually quantized: the Stern-Gerlach experiment, in which measured angular momentum is shown to take discrete values. The gist of the experiment is that in a varying magnetic field the tidal force on a magnet is not perfectly balanced and so the magnet moves toward or away from the denser field, depending on the orientation of its poles. This is intuitively clear to anyone who ever played with magnets: the degree of attraction or repulsion depends on the relative orientation of the magnets (North pole repels North pole etc.). It is less obvious that this effect is due to the spatially varying magnetic field density, but it is nonetheless the case.

In the experiment, one magnet is large (the S-G apparatus itself) and one is small (a silver atom injected into the magnetic field of the large magnet). The experiment shows that an unoriented atom suddenly becomes aligned either along or against the field, but not in any other direction. It's like a compass needle that would only be able to point North and South (and potentially in a few other directions) but not anywhere in between.

If necessary, please read through the more detailed description of the experiment on Wikipedia or in any other source before attempting the following questions (usually called meditations in the idiosyncratic language used on this forum). 

Meditation 1. When exactly does the atom align itself? As soon as it enters the field? At some random moment as it travels through the field? The instance it hits the screen behind the field? In other words, in the MWI picture, when does the world split into two, one with the atom aligned and one with the atom anti-aligned? In the Copenhagen picture, does the magnetic field measure the atom spin, and if so, when, or does the screen do it?

Hint. Consider whether/how you would tell these cases apart experimentally.

Meditation 2. Suppose you make two holes in the screen where the atoms used to hit it, then merge the atoms into a single stream again by applying a reverse field. Are the atoms now unaligned again, or 50/50 aligned/anti-aligned or something else?

Hint. What's the difference between these cases?

Meditation 3. Suppose that instead of the reversing field in the above experiment you keep the first screen with two holes in it, and put a second screen (without any holes) somewhere behind the first one. What would you expect to see on the second screen and why? Some possible answers: two equally bright blobs corresponding to aligned and anti-aligned atoms respectively; the interference pattern from each atom passing through both holes at once, like in the double-slit experiment; a narrow single blob in the center of the second screen, as if the atoms did not go through the first part of the apparatus at all; a spread-out blob with a maximum at the center, like you would expect from the classical atoms.

Hint. Consider/reconsider your answer to the first two questions.

Meditation 4. Suppose you want to answer M1 experimentally and use an extremely sensitive accelerometer to see which way each atom is deflecting before it hits the screen by measuring the recoil of the apparatus. What would you expect to observe? 

Hint. Consider a similar setup for the double-slit experiment.

 

This test is open-book and there is no time limit. You can consult any sources you like, including textbooks, research papers, your teachers, professional experimental or theoretical physicists, your fellow LWers, or the immortal soul of Niels Bohr through your local medium. If you have access to the Stern-Gerlach apparatus in your physics lab, feel free to perform any experiments you may find helpful. As they say, if you are not cheating, you are not trying hard enough.

By the way, if anyone wants to supply the pictures to make the setup for each question clearer, I'd be more than happy to include them in this post. If anyone wants to turn the meditations into polls, please do so in the comments.

Footnote: not posting this in Main, because I'm not sure how much interest there is here for QM questions like this.

In a previous post, I argued that nihilism is often short changed around here. However I'm far from certain that it is correct, and in the mean time I think we should be careful not to discard our values one at a time by engaging in "selective nihilism" when faced with an ontological crisis, without even realizing that's what's happening. Karl recently reminded me of the post Timeless Identity by Eliezer Yudkowsky, which I noticed seems to be an instance of this.

As I mentioned in the previous post, our values seem to be defined in terms of a world model where people exist as ontologically primitive entities ruled heuristically by (mostly intuitive understandings of) physics and psychology. In this kind of decision system, both identity-as-physical-continuity and identity-as-psychological-continuity make perfect sense as possible values, and it seems humans do "natively" have both values. A typical human being is both reluctant to step into a teleporter that works by destructive scanning, and unwilling to let their physical structure be continuously modified into a psychologically very different being. 

If faced with the knowledge that physical continuity doesn't exist in the real world at the level of fundamental physics, one might conclude that it's crazy to continue to value it, and this is what Eliezer's post argued. But if we apply this reasoning in a non-selective fashion, wouldn't we also conclude that we should stop valuing things like "pain" and "happiness" which also do not seem to exist at the level of fundamental physics?

In our current environment, there is widespread agreement among humans as to which macroscopic objects at time t+1 are physical continuations of which macroscopic objects existing at time t. We may not fully understand what exactly it is we're doing when judging such physical continuity, and the agreement tends to break down when we start talking about more exotic situations, and if/when we do fully understand our criteria for judging physical continuity it's unlikely to have a simple definition in terms of fundamental physics, but all of this is true for "pain" and "happiness" as well.

I suggest we keep all of our (potential/apparent) values intact until we have a better handle on how we're supposed to deal with ontological crises in general. If we convince ourselves that we should discard some value, and that turns out to be wrong, the error may be unrecoverable once we've lived with it long enough.

Consider the following statement: two heads are better than one.

It seems obvious to me that several rationalists working together can, effectively, bring more precious brainpower to bear on a problem, than one rationalist working alone (otherwise, what would be the point of having a Less Wrong forum community? You might as well just leave it as a curated community blog and excise the discussion forums.). Further, due to various efforts (HPMOR especially) it appears that LW is inevitably growing. This makes it not only desirable to find ways to effectively get groups of rationalists to think together, but also increasingly necessary.

It is also desirable that methods of getting groups to think should be feasibly doable over the Internet. (I am aware that real-life meetups and stuff exist, but please be reminded that some people in the world do have to live in shitty little third-world countries and might not at all find it economically feasible to go to first-world countries with atrociously high costs-of-living)

So first, let us start with the current "best methods" of getting groups of Traditional Rationalists to coordinate and think, while avoiding groupthink effects that diminish our aggregate rationality. Hopefully, we can then use it as the basis of part of the art of rationalist group thinking. So I'll discuss:

• Disputation Arenas - procedures with centrifugal and centripetal phases
  1. Delphi Method - members secretly answer questions, non-member summarizer anonymizes answers, members read anonymized summary, repeat
  2. Prediction Market - members place stock bets on market, market settles on price, members react to price signal, repeat
  3. Nominal Group Technique - members write down ideas privately, non-member facilitator guides members in sharing ideas, non-member facilitator guides members in paring down and cleaning up ideas, members vote
• Conclusion - strengths and weaknesses of the various disputation arenas shown here

Disputation Arenas

One of my favorite SF authors, David Brin, talks a bit about what he calls "disputation arenas". I won't discuss his ideas here, since his concept of "disputation arena" is actually a relatively "new", raw, and relatively untested procedure - what I intend to discuss for now are things that have at least been studied more rigorously than just a bunch of blog posts (or personal website pages, whatever).

However, I do want to co-opt the term "Disputation Arena" for any process that tries to achieve the following:

• Avoid groupthink - actively search for information without settling too early on an option considered desirable by certain influential members
• Achieve consensus - designate some choice as the best, given current information known by the group members

We want our group rationality process to avoid groupthink (possibly at some expense of efficiency) because actual, real-world rationalists are not perfect Bayesian reasoners - two words: Robert Aumann. Because rationalists are not perfect, we do not expect a clear consensus to form after the end of the process (i.e. Aumann's does not necessarily apply), so the process we use must force some consensus to become visible.

One thing that David Brin discusses is the general division of the procedures into two "phases":

• Centrifugal phase - members of the group generate ideas separately.
• Centripetal phase - ideas are judged by the group together.

This seems to me to be a good way of labeling parts of any group-coordination process that attempts to avoid groupthink and achieve consensus.

In my personal research, I've found three things that attempt to achieve those two goals (avoid groupthink, achieve consensus) and which might (perhaps with a stretch) be considered as approximately having two phases (centrifugal and centripetal).

Delphi Method

Wikipedia:Delphi method

The Delphi Method was originally developed by RAND Corporation (Project RAND at that time, and no relation to Ayn Rand) in order to get better predictions on future military hardware. It is currently used to get better utilization of current human wetware. (^.^)v

Delphi Method: How To Do It!

Pen and paper version:

1. A panel of experts is chosen, and a questionnaire is prepared.
2. Experts answer the questionnaire, giving the answers and also justifications and reasons for those answers.
3. Summarizer provides anonymous summaries of the expert's answers and justifications.
4. Experts read the summary, and may revise their answers/justifications. The process is repeated (with the same experts and questionnaire) for a set number of rounds, or until everyone gets bored, or the military bunker everyone is in gets nuked.

Internet version (Wikipedia:Real-time Delphi):

1. Username/passwords are chosen and emailed, and a questionnaire is prepared. The questionnaire in the online version is somewhat restricted, however. Here are some ideas:
   • Use a "poll" question. Experts select one of the choices given.
   • Use "multiple-choice" questions. Experts then select from a range of (e.g.) 1 for (strongly disagree) to 10 (strongly agree) for each possible choice.
   • Use "multiple-choice" questions, and also give different aspects such as "feasibility", "desirability", "good side-effects", etc. Experts answer 1 to 10 for each combination of choice and aspect.
2. Experts answer the online questionnaire. Aside from the numerical or selection (quantitative) answer, experts should also supply a short sentence or two justifying each answer (qualitative).
3. After the expert submits his or her answers, he or she is shown the current averages (for scoring-type questionnaires) or current poll results - this is the quantitative group answer. Expert is also shown (or provided links to) randomly-sorted (and randomly-chosen, if groups are very large and the number of answers may overwhelm a typical human) qualitative answers for each poll item / choice / choice+aspect - the qualitative group answers - for each score or aspect.
   • IMPORTANT: individual qualitative answers should not show the username of the expert who gave them!
   • In effect, the average (or poll results) plus the randomized sample of qualitative answers are a simple, anonymous, machine-generated summary.
4. Experts may change their own quantitative and/or qualitative answers at any time, and see the current quantitative group answers and qualitative group answers at any time after they have submitted their own answers.
5. The questionnaire is kept open until some specified time, or somebody hacks the server to put LOLcats instead.

Delphi Method: Analysis

Delphi methods avoid groupthink largely by anonymity: this avoids the bandwagon effect, the halo effect, and the mind-killer. Anonymity and constant feedback also encourage people to revise their positions in light of new information from their peers (by reducing consistency pressure): in non-anonymous face-to-face meetings, people tend to stick to their previously stated opinions, and to conform to the meeting leader(s) or their own bosses in the meeting. A lot of those effects is reduced by anonymity. Pen-and-paper form makes anonymity much easier, since the summary gets the tone and language patterns of the summarizer; some amount of anonymity is lost in the online version (since language patterns might theoretically be analyzed) but hopefully the small sample size (just a short sentence or two) can make language pattern analysis difficult. Note that randomizing the order of the comments in the online version is important, as this reduces the effects of anchoring; sorting by time or karma may increase groupthink due to anchoring on earlier comments, but if each expert sees different "first comments", then this bias gets randomized (hopefully into irrelevancy).

Delphi methods achieve consensus by the summary (which often serves as the "final output" when the process is finished). Arguably, the pen-and-paper version is better at achieving consensus due to the "turn-based" arrival of the summary, which makes the expert pay more attention to the summary, compared to the real-time online system.

The Delphi method's centrifugal phase is the expert's private answering of the questionnaire: each expert makes this decision, and provides the justification, without other's knowledge or help.

The Delphi method's centripetal phase is the act of summarizing, and having the experts read the summary.

Delphi Method: Other Thoughts

I think that forum polls, in general, can be easily adapted into online real-time Delphis by adding the following:

• Members will be required to give a short sentence or two (probably limited to say 200 chars or so) justifying their poll choice.
• Members should be allowed to change their poll choice and their justification at any time until the poll closes or the forum is hacked by LOLcat vandals.
• Members should be able to click on a poll choice on the poll results page to get a random anonymous sampling of the justifications that the other members have made in choosing that poll choice.

The procedure says "experts" but I think that in something more democratic than the military you're supposed to read that as "anyone who bothers to participate".

Prediction Market

Wikipedia:Prediction market

Prediction markets are speculation markets built around bets on what things will happen in the future. They are also the core of Robin Hanson's Futarchy, and which you can see somewhere in the background of LW's favorite tentacle alien porn novella (O.o);;.

Prediction Market: How To Do It!

Pen and paper version:

1. Convince a trusted monetary institution to sell you "X is gonna happen" stock and "X is not gonna happen" stock for $1 a pair (i.e. $1 for a pair of contracts, one that says "If X happens, Monetary Institution pays you $1" and another that says "If X doesn't happen, Monetary Institution pays you $1", so the pair costs $1 total since X can't both happen and not happen). You may need to pay some sort of additional fee or something if the monetary institution is for-profit.
2. Sell the stock (i.e. the contract) you think is false for as high as you can get on the open market. Buy more stock of what you think is true from others who are willing to sell to you.
3. Just buy and sell stocks depending on what you think is the best prediction, based on what you hear on the news, gossip you hear from neighbors, and predictions from the tea leaves. Keep doing this until X definitely happens or X definitely does not happen (in which case you cash in your stock contracts, if you bet correctly on what happened), or a market crash results because someone discovers that the weak nuclear force actually allows you to make nuclear bombs out of orange juice, and Einstein and the gang were lying about it and distracting you by talking about dice-playing gods.

(what I described above is the simplest and most basic form I found; refer to the Wikipedia article for better elaborations)

Internet version:

1. Hack Intrade so that the topic you want to bet on is in their list of markets. Or better yet just hack Intrade and put one million dollars into your account.

Prediction Market: Analysis

Prediction markets avoid groupthink by utilizing the invisible hand. Someone selling you a stock might be an idiot who can't read the tea leaves properly. Or the seller might have knowledge you do not possess, so maybe buying the stock wasn't such a good idea after all? Remember: if you can't find who the sucker on the table is, that sucker is you!! You can't simply assume that what your neighbor says is true and you should sell as many stock of X as possible: maybe he or she is trying to take advantage of you to get your hard-earned cash. Groupthink in such a mistrusting environment gets hard to sustain. Prediction markets work better with very large groups of people, so that you get practical anonymity (although not perfect, in theory you or anyone else can keep track of who's selling to who; online versions are also likely to hide user identities). Anonymity in the prediction market also has the advantages previously described under Delphi Method above.

Prediction markets achieve consensus by utilizing the invisible hand. The price point of any sale serves as an approximate judgment of the epistemic probability of X occurring (or not occurring, depending on the contract that got sold). This gives a real-time signal on what the group of traders as a whole think the probability of X occurring is.

The prediction market's centrifugal phase is each individual trader's thought process as he or she considers whether to buy or sell stock, and at what price.

The prediction market's centripetal phase is any actual sale at any actual price point.

Prediction Market: Other Thoughts

Prediction markets are well-represented online; Intrade is just one of the more famous online prediction markets. Prediction markets appear to be the most popular and widely-known of the disputation arenas I've researched. These all tend to suggest that prediction markets are one of the better disputation arenas - but then remember that the Internet itself has no protection against groupthink.

Nominal Group Technique

Wikipedia: Nominal group technique

Nominal group technique is a group decision-making process, appropriate for groups of many different sizes. This procedure's pen-and-paper form is faster than the pen-and-paper forms of the other disputation arenas discussed here.

Nominal Group Technique: How To Do It!

Pen and paper version:

1. The facilitator informs the group of the issue to be discussed.
2. Silent idea generation: group members are provided pen and paper, and are told to write down all ideas they can think of about the issue on the paper. They are given a fixed amount of time to do this (usually 10 to 15 minutes).
   • IMPORTANT: members are not allowed to discuss, show, or otherwise share their ideas with others during this stage. There's a reason it's called "silent".
3. Idea sharing: the facilitator asks group members, one at a time, to discuss their own ideas, until all members have shared their ideas. The facilitator writes the shared ideas into a whiteboard, or a similar location visible to all members.
   • IMPORTANT: debate is not allowed at this stage; only one member at a time can speak at this stage.
   • Group members may also add additional ideas and notes to their written-down ideas while waiting for their turn.
4. Group discussion: members may ask for clarification or further details about particular ideas shared by other group members. The group may agree to split ideas, or merge ideas, or group ideas into categories.
   • IMPORTANT: the facilitator must ensure that (1) all members participate, and (2) no single idea gets too much attention (i.e. all ideas must be discussed).
   • The discussion should be as neutral as possible, avoiding judgment or criticism.
   • The final result of this stage should be a set of options to be chosen among.
5. Ranking: members secretly rank the options from 1 (best) to N (worst), where N is the number of options generated in the previous stage. The facilitator then tallies the (anonymous) rankings (by adding the rankings for each option) and declares the option with the lowest total as the group consensus.

Unlike the previous procedures, which have been extrapolated into Internet versions, there is currently no Internet version of nominal group technique.

Nominal Group Technique: Analysis

Nominal group techniques avoid groupthink by the two "secret" steps: silent idea generation, and the secret ranking. Having members write down their ideas in the silent idea generation step helps them precommit to those ideas in the idea sharing step, even though more influential group members may present opposite or incompatible ideas. Although the group discussion step disallows explicit criticism of ideas, those criticisms are implicitly expressed during the secret ranking step (i.e. if you have a criticism of an idea, then you should rank it lower).

Nominal group techniques achieve consensus by the idea sharing, group discussion, and ranking steps. In particular, tallying of option rankings is the final consensus-achieving step.

The nominal group technique's centrifugal phase is largely the silent idea generation step, and is the most explicit centrifugal phase among the disputation arenas discussed here.

The nominal group technique's centripetal phase is largely the rest of the procedure.

Nominal Group Technique: Other Thoughts

A modified form of nominal group technique eats up a quarter of my recently-finished novel, Judge on a Boat, which I talked about on LessWrong here, and whose latest raw text source you can read online. Yes, this entire article is just a self-serving advertisement to garner interest in my novel o(^.^o)(o^.^)o.

Compared to the other procedures here, nominal group technique is more complicated and much more dependent on the centralized facilitator; the extreme dependency on the facilitator makes it difficult to create an automated online version. On the other hand, a small group of say 5 to 10 people can finish the nominal group technique in 1 to 2 hours; the other procedures tend to work better when done over several days, and are largely impossible (in pen-and-paper form) to do in a similar time frame. Even the real-time online versions of the other procedures are difficult to do within 2 hours. Prediction markets in particular tend to fail badly if too thin (i.e. not enough participants); for small groups with tight schedules, nominal group technique tends to be the best.

Conclusion

For small groups that need to make a decision within one or two hours, use nominal group technique. It's relatively unwieldy compared to the other disputation arenas, and is less ideal (it has fewer protections against groupthink, in particular), but is fast compared to the others. Also, one might consider parallelizing nominal group technique: split a large group into smaller, randomly-selected sub-groups, have each perform the procedure independently, and then have them send a representative that performs the idea sharing, group discussion, and ranking steps with other representatives (i.e. each sub-group's nominal group technique serves as the silent idea generation of the super-group of representatives). This tends to bias the super-group towards the agendas of the chosen representatives, but if speed is absolutely necessary for a large group, this may be the best you can do.

As mentioned above, prediction markets tend to fail badly if there are too few participants in the speculation market; use it only for extremely large groups that are impossible to coordinate otherwise. In addition, using prediction markets for policy decisions is effectively futarchy; you may want to see the (defunct?) futarchy_discuss Yahoo! group's message archives. In particular the earlier messages in the archive tend to discuss the general principles of prediction markets. Prediction markets are the most famous of the disputation arenas here, but remember that the Internet is not a decent disputation arena.

The Delphi methods seem to be a "dark horse" of sorts. I don't see much discussion online about Delphi methods; I'm not sure whether it's because it's been tried and rejected, or if it simply isn't well known enough to actually be tried by most people. I tend to suspect the latter, since if the universe were in the former case I would at least see some "Delphi Methods suck!!" blog posts.

Both prediction markets and Delphi methods are continuously repeated methods. At any time, the procedure may be stopped or repeated in order to make decisions. However, unlike the nominal group techniques, both are targeted more towards generating advice for decision-makers, rather than making actual decisions themselves.

It may be possible to organize a large, hierarchical group (say a company) with a prediction market for the rank-and-file, some key experts (who should be aware of the prediction market's results) running a Delphi method, and the key decision-making individuals (who read the Delphi method's report) at the top who form a decision using nominal group technique. For more democratic processes, a "poll-style" real-time online Delphi method by itself may work.

Suppose we have two different human beings, Connor and Diane, who agree to interpret their subjective anticipations as probabilities, thereby commonly earning them the title "Bayesian".  On a particular project or venture, they might disagree on Trick A or Trick B to decide the next step in the project.  It might be that Trick A is commonly labelled a "Frequentist inference method" and B is a "Bayesian inference method".  Why might they disagree?

As far as I can see, there are 3 disagreements that get labelled "Bayesian vs Frequentist" debates, and conflating them is a problem:

(1) Whether to interpret all subjective anticipations as probabilities.

(2) Whether to interpret all probabilities as subjective anticipations.

(3) Whether, on a particular project, to use Statistical Trick B instead of Statistical Trick A to infer the best course of action, when B is commonly labelled a "Bayesian method" and A is a "Frequentist method".

(Regarding 3, UC Berkeley professor Michael Jordan offers a good heuristic for how statistical tricks get labelled as Bayesisn or Frequentist, in terms of which terms in a loss function one treats as fixed or variable.  I recommend watching the first twenty minutes of his video lecture on this if you're not familiar.)

The question "is Connor a Bayesian or a Frequentist?" is commonly posed as though Connor's position on 1, 2, and 3 must be either "yes, yes, yes" or "no, no, no".  I don't believe this is so often the case. For example, my position is:

(1) - Yes.  Insofar as we have subjective anticipations, I agree normatively that they should behave and update as probabilities.

(2) - Don't care much. Expressions like P(X|Y) and P(X and Y) are useful for denoting both subjective anticipations and proportions of a whole, and in particular, proportions of real future events.  Whether to use the word "probability" is a terminological question.  Personally I try to reserve the word "probability" for when they mean subjective anticipations, and say "proportion" when they mean proportions of real future, but this is word choice.  Unfortunately this word choice is strongly associated and confused with positions on (1) and (3).

(3) - It depends.  In statistical inference, we commonly consider data sets x, world models M, and parameters θ that specify the model M more precisely.  I consider the separation of belief into M and θ to be purely formal. When guessing the next data set y, one considers expressions of the form P(x|M,θ) in some way.  If I'm already very confident in a specific world model M, and expect θ to actually vary from situation to situation, I'll probably try to estimate the parameters θ from x in a way that has the best expected success rate across all possible data sets M would generate. You might say here that I "trust the model more than the data" (though what I really don't trust are the changing model parameters), and this is a trick commonly referred to as "Frequentist".  If I'm not confident in the model M, or expect the parameters θ to the be the same in many future situations, I'll probably try to estimate M,θ from x in a way that has the best expected success rate assuming x.  You might say here that I "trust the data more than the model", and label this a "Bayesian" trick.  

Throughout (3), since my position in (1) is not changing, a member of the Bayes Tribe will say I'm "really a Bayesian all along", but I don't want to continue with this conflation of position names.  It's true that if I use the "Frequentist trick", it will be because I've updated in favor of it, i.e. my subjective confidence levels in the various theory elements are appropriate for it. 

... But from now on, when term "Bayesian" or "Frequentist" arises in a debate, my plan is to taboo the terms immediately, and proceed to either dissolve the issue into (1), (2), and (3) above, or change the conversation if people don't have the energy or interest for that length of conversation.

Do people agree with this breakdown?  I think I could be persuaded otherwise and would of course appreciate it if I were :)

 

---

 

ETA: I think the wisdom to treat beliefs as anticipation controllers and update our confidences based on evidence might be too precious to alienate people from it with the label "Bayesian", especially if the label is as ambiguous as my breakdown has found it to be.