I'd like to ask everyone what probability bump they give to an idea given that some people believe it.

None.

Or as Ben Goldacre put it in a talk: There are millions of medical doctors and Ph.D.s in the world. There is no idea, however completely fucking crazy, that you can't find some doctor to argue for.

So, given that some people believe X, what probability do you give for X being true, compared to Y which nobody currently believes?

In any case of a specific X and Y, there will be far more information than that (who believes X and why? does anyone disbelieve Y? etc.), which makes it impossible for me to attach any probability for the question as posed.

Out of the huge idea space of possible causally linked events, some of them make good stories and some do not. That doesn't tell you rather it's true or not.

If a guy thinks that he can hear Hillary Clinton speaking from the feelings in his teeth, telling him to murder his cellmate, do you believe what he says? Status gets mucked up in the calculation, but with strangers it teeters precariously close to zero.

I really like kids,but the fact that millions of them passionately believe in Santa Claus does not change my degree of subjective belief one iota.

I'd like to ask everyone what probability bump they give to an idea given that some people believe it.

Usually fairly substantial - if someone presents me with two equally-unsupported claims X and Y and tells me that they believe X and not Y, I would give greater credence to X than to Y. Many times, however, that credence would not reach the level of ... well, credence, for various good reasons.

I think it largely depends on a) what the idea is and b) who believes it = and what their rationality skills are.

Depends on the person and the idea. I have some people whose recommendations I follow regardless, even if I estimate upfront that I will consider the idea wrong. There are different levels of wrongness, and it does not hurt to get good counterarguments. It also depends on the real life practicability of the idea. If it is for everyday things than common sense is a good starting prior. (Also there is a time and place to use the public joker on Who wants to be a millionaire.) If a group of professionals agree on something related to their profession it is also a good start. To systematize: if a group of people has a belief about something they have experience with, that that belief is worth looking at.

And then on further investigation it often turns out that there are systematic mistakes being made.

I was shocked to read in the book on checklists, that not only doctors often don't like them. But even financial companies, that can see how the usage ups their monetary gains. But finding flaws in a whole group does not imply that everything they say is wrong. It is good to see a doctor, even if he not using statistics right. He can refer you to a specialist, and treat all the common stuff right away. If you get a complicated disease you can often read up on it.

The obvious example to your question would be religion. It is widely believed, but probably wrong, yet I did not discard it right away, but spent years studying stuff till I decided there was nothing to it. There is nothing wrong in examining the ideas other people have.

I've become increasingly disillusioned with people's capacity for abstract thought. Here are two points on my journey.

The public discussion of using wind turbines for carbon-free electricity generation seems to implicitly assume that electricity output goes as something like the square-root of windspeed. If the wind is only blowing half speed you still get something like 70% output. You won't see people saying this directly, but the general attitude is that you only need back up for the occasional calm day when the wind doesn't blow at all.

In fact output goes as the cube of windspeed. The energy in the windstream is one half m v squared, where m, the mass passing your turbine is proportional to the windspeed. If the wind is at half strength, you only get 1/8 output.

Well, that is physics. Ofcourse people suck at physics. Trouble is, the more I look at people's capacity for abstract thought the more problems I see. When people do a cost/benefit analysis they are terribly vague on whether they are suposed to add the costs and benefits or whether the costs get subtracted from the benefits. Even if they realise that they have to subtract they are still at risk of using an inverted scale for the costs and ending up effectively adding.

The probabiltiy bump I give to an idea just because some people believe it is zero. Equivantly my odds ratio is one. However you describe it, my posterior is just the same as my prior.

I don't think belief has a consistent evidentiary strength since it depends on the testifier's credibility relative to my own. Children have much lower credibility than me on the issue of the existence of Santa. Professors of physics have much higher credibility that me on the issue of dimensions greater than four. Some person other than me has much higher credibility on the issue of how much money they are carrying. But I have more credibility than anyone else on the issue of how much money I'm carrying. I don't see any relation that could be described as baseline so the only answer is: context.

Maybe you can set your success setpoint to a lower value. The optimum is hard to achieve. So looking for 100% everywhere might be bad.

Often it is our imagined bad futures that keep us too afraid to act. In my experience this is more common than the opposite.

I was talking to a friend yesterday and he mentioned a psychological study (I am trying to track down the source) that people tend to suffer MORE from failing to pursue certain opportunities than FAILING after pursuing them. So even if you're right about the overestimation of pleasure, it might just be irrelevant.

Could you give some examples of the predicted collapses that didn't happen?

Not sure if you're referring to the same literature, but I note a great divergence between peak oil advocates and singularitarians. This is a little weird, if you think of Aumann's Agreement theorem.

Both groups are highly populated with engineer types, highly interested in cognitive biases, group dynamics, habits of individuals and societies and neither are mainstream.

Both groups use extrapolation of curves from very real phenomena. In the case of the kurzweillian singularitarians, it is computing power and in the case of the peak oil advocates, it is the hubbert curve for resources along with solid Net Energy based arguments about how civilization should decline.

The extreme among the Peak Oil advocates are collapsitarians and believe that people should drastically change their lifestyles, if they want to survive. They are also not waiting for the others to join them and many are preparing to go to small towns, villages etc. The oildrum, linked here had started as a moderate peak oil site discussing all possibilities, nowadays, apparently, its all doom all the time.

The extreme among the singularitarians have been asked no such sacrifice, just to give enough money and support to make sure that Friendly AI is achieved first.

Both groups believe that business as usual cannot go on for too long, but they expect dramatically different consequences. The singularitarians assert that economics conditions and technology will improve until a nonchalant super-intelligence will be created and wipe out humanity. The collapsitarians believe that economic conditions will worsen, civilization is not built robustly and will collapse badly with humanity probably going extinct or only the last hunter gatherers surviving.

Being right on group effects is difficult.

Is there a consistent path for what LW wants to be? a) rationalist site filled up with meta topics and examples b) a) + detailed treats of some important topics c) open to everything as long as reason is used

and so on. I personally like and profit from the discussing of akrasia methods. But it might be detrimental to the main target of the site. Also I would very much like to see a cannon develop for knowledge that LWers generally agree upon including, but not limited to the topics I currently care about myself.

Voicing ideas depends on where you are. In social settings I more and more advice against it. Arguing/discussing is just not helpful. And if you are filled up with weird ideas then you get kicked out, which might be bad for other goals you have.

It would be great to have a place for any idea to be examined for right and wrong.

I think the idea is that CEV lets us "grow up more together" and figure that out later.

I have only recently started looking into CEV so I'm not sure whether I a) think it's a workable theory and b)think it's a good solution, but I like the way it puts off important questions.

It's impossible to predict what we will want if age, disease, violence, and poverty become irrelevant (or at least optional).

I think the expectation is that, if all humans had the same knowledge and were better at thinking (and were more the people we'd like to be, etc.), then there would be a much higher degree of coherence than we might expect, but not necessarily that everyone would ultimately have the same utility function.

That's well done although two of the central premises are likely incorrect. First, the notion that a quantum computer would have infinite processing capability is incorrect. Quantum computation allows speed-ups of certain computational processes. Thus for example, Shor's algorithm allows us to factor integers quickly. But if our understanding of the laws of quantum mechanics is at all correct, this can't lead to anything like that in the story. In particular, under the standard descriptor for quantum computing, the class of problems reliably solvable on a quantum computer in polynomial time (that is the time required to solve is bounded above by a polynomial function of the length of the input sequence), BQP is is a subset of of PSPACE, the set of problems which can be solved on a classical computer using memory bounded by a polynomial of the space of the input. Our understanding of quantum mechanics would have to be very far off for this to be wrong.

Second, if our understanding of quantum mechanics is correct, there's a fundamentally random aspect to the laws of physics. Thus, we can't simply make a simulation and advance it ahead the way they do in this story and expect to get the same result.

Even if everything in the story was correct, I'm not at all convinced that things would settle down on a stable sequence as they do here. If your universe is infinite then your possible number of worlds are infinite so there's no reason you couldn't have a wandering sequence of worlds. Edit: Or for that matter, couldn't have branches if people simulate additional worlds with other laws of physics or the same laws but different starting conditions.

Couldn't they just run the simulation to its end rather then just let it sit there and take the chance that it could accidently be destroyed. If its infinitley powerful, it would be able to do that.

It depends on what you mean by "criminal"; under environmental law, there are both negligence-based (negligent discharge of pollutants to navigable waters) and strict liability (no intent requirement, such as killing of migratory birds) crimes that could apply to this spill. I don't think anyone thinks BP intended to have this kind of spill, so the interesting question from an environmental criminal law perspective is whether BP did enough to be treated as acting "knowingly" -- the relevant intent standard for environmental felonies. This is an extremely slippery concept in the law, especially given the complexity of the systems at issue here. Litigation will go on for many years on this exact point.

I'm not sure it's relevant whether they did anything illegal or not. People always seem to want to blame and punish someone for their problems. In my opinion, they should be forced to pay for and compensate for all the damage, as well as a very large fine as punishment. This way in the future they, and other companies, can regulate themselves and prepare for emergencies as efficiently as possible without arbitrary and clunky government regulations and agencies trying to slap everything together at the last moment. Of course, if a single person actually did something irresponsible (eg; bob the worker just used duct tape to fix that pipe knowing that it wouldn't hold) then they should be able to be tried in court or sued/fined by the company. But even then, it's up to the company to make sure that stuff like this doesn't happen by making sure all of their workers are competent and certified.

You are not really going to learn much unless you are interested in wading through lots of technical articles. If you want to learn, you need to wait until it has been digested by relevant experts into books. I am not sure what you think you can learn from this, but there are two good books of related information available now:

Jeff Wheelwright, Degrees of Disaster, about the environmental effects of the Exxon Valdez spill and the clean up.

Trevor Kletz, What Went Wrong?: Case Histories of Process Plant Disasters, which is really excellent. [For general reading, an older edition is perfectly adequate, new copies are expensive.] It has an incredible amount of detail, and horrifying accounts of how apparently insignificant mistakes can (often literally) blow up on you.

One possible answer: Humans are selfish hypocrites. We try to pretend to have general moral rules because it is in our best interest to do so. We've even evolved to convince ourselves that we actually care about morality and not self-interest. That's likely occurred because it is easier to make a claim one believes in than lie outright, so humans that are convinced that they really care about morality will do a better job acting like they do.

(This was listed by someone as one of the absolute deniables on the thread a while back about weird things an AI might tell people).

Adults, by choosing to live in a society that punishes non-cooperators, implicitly accept a social contract that allows them to be punished similarly. While they would prefer not to be punished, most societies don't offer asymmetrical terms, or impose difficult requirements such as elections, on people who want those asymmetrical terms.

Children, on the other hand, have not yet had the opportunity to choose the society that gives them the best social contract terms, and wouldn't have sufficient intelligence to do so anyways. So instead, we model them as though they would accept any social contract that's at least as good as some threshold (goodness determined retrospectively by adults imagining what they would have preferred). Thus, adults are forced by society to give implied consent to being punished if they are non-cooperative, but children don't give consent to be eaten.

With 1), you're non-cooperator and the punisher is society in general. With 2), you play both roles at different times.

Except for the bizarreness of eating most of your children, I suspect that most humans would find the two positions equally hypocritical. Why do you think we see them as different?

Different topic spheres. One line sounds nicely abstract, while the other is just iffy.

Also killing people is different from betraying them. (Nice read: the real life section of tvtropes/moraleventhorizon)

Abstract preferences for or against the existence of enforcement mechanisms that could create binding cooperative agreements between previously autonomous agents have very very few detailed entailments.

These abstractions leave the nature of the mechanisms, the conditions of their legitimate deployment, and the contract they will be used to enforce almost completely open to interpretation. The additional details can themselves be spelled out later, in ways that maintain symmetry among different parties to a negotiation, which is a strong attractor in the semantic space of moral arguments.

This makes agreement with "the abstract idea of punishment" into the sort of concession that might be made at the very beginning of a negotiating process with an arbitrary agent you have a stake in influencing (and who has a stake in influencing you) upon which to build later agreements.

The entailments of "eating children" are very very specific for humans, with implications in biology, aging, mortality, specific life cycles, and very distinct life processes (like fuel acquisition versus replication). Given the human genome, human reproductive strategies, and all extant human cultures, there is no obvious basis for thinking this terminology is superior until and unless contact is made with radically non-human agents who are nonetheless "intelligent" and who prefer this terminology and can argue for it by reference to their own internal mechanisms and/or habits of planning, negotiation, and action.

Are you proposing to be such an agent? If so, can you explain how this terminology suits your internal mechanisms and habits of planning, negotiation, and action? Alternatively, can you propose a different terminology for talking about planning, negotiation, and action that suits your own life cycle?

For example, if one instance of Clippy software running on one CPU learns something of grave importance to its systems for choosing between alternative courses of action, how does it communicate this to other instances running basically the same software? Is this inter-process communication trusted, or are verification steps included in case one process has been "illegitimately modified" or not? Assuming verification steps take place, do communications with humans via text channels like this website feed through the same filters, analogous filters, or are they entirely distinct?

More directly, can you give us an IP address, port number, and any necessary "credentials" for interacting with an instance of you in the same manner that your instances communicate over TCP/IP networks with each other? If you aren't currently willing to provide such information, are there preconditions you could propose before you would do so?

Hard to say - even New Agey stuff evolves. (Not many followers of Reich pushing their copper-lined closets these days.)

Generally, background stuff is enough. There's no shortage of hard scientific evidence about yoga or meditation, for example. No need for heuristics there. Similarly there's some for float tanks. In fact, I'm hard pressed to think of any New Agey stuff where there isn't enough background to judge it on its own merits.

Question: whats your experience with stuff that seems new agy at first look, like yoga, meditation and so on. Anything worth trying?

The Five Tibetans are a set of physical exercises which rejuvenate the body to youthful vigour and prolong life indefinitely. They are at least 2,500 years old, and practiced by hidden masters of secret wisdom living in remote monasteries in Tibet, where, in the earlier part of the 20th century, a retired British army colonel sought out these monasteries, studied with the ancient masters to great effect, and eventually brought the exercises to the West, where they were first published in 1939.

Ok, you don't believe any of that, do you? Neither do I, except for the first eight words and the last six. I've been doing these exercises since the beginning of 2009, since being turned on to them by Steven Barnes' blog and they do seem to have made a dramatic improvement in my general level of physical energy. Whether it's these exercises specifically or just the discipline of doing a similar amount of exercise first thing in the morning, every morning, I haven't taken the trouble to determine by varying them.

More here and here. Nancy Lebovitz also mentioned them.

I also do yoga for flexibility (it works) and occasionally meditation (to little detectable effect). I'd be interested to hear from anyone here who meditates and gets more from it than I do.

I've done yoga every week for the last month or two. It's pleasant. Other than paying attention to how I'm holding my body vs. the instruction, I mostly stop thinking for an hour (as we're encouraged to do), which is nice.

I can't say I notice any significant lasting effects yet. I'm slightly more flexible.

Meditation can be pretty darn relaxing. Especially if you happen to live within walking distance of any pleasant yet sparsely-populated mountaintops. I would recommend giving it a shot; don't worry about advanced techniques or anything, and just close your eyes and focus on your breathing, and the wind (if any). Very pleasant.

Chinese internal martial arts: Tai Chi, Xingyi, and Bagua. The word "chi" does not carve reality at the joints: There is no literal bodily fluid system parallel to blood and lymph. But I can make training partners lightheaded with a quick succession of strikes to Ren Ying (ST9) then Chi Ze (LU5); I can send someone stumbling backward with some fairly light pushes; after 30-60 seconds of sparring to develop a rapport I can take an unwary opponent's balance without physical contact.

Each of these skills fit more naturally under different categories, but if you want to learn them all the most efficient way is to study a Chinese internal martial art or something similar.

Not having to side scroll would be spiffy.

Very cool idea.

It would be nice if links were preserved.

It might make more sense to put this on the Wiki. Two notes: First, some of the quotes have remarks contained in the posts which you have not edited out. I don't know if you intend to keep those. Second, some of the quotes are comments from quote threads that aren't actually quotes. 14 SilasBarta is one example. (And is just me or does that citation form read like a citation from a religious text ?)

It does seem odd to get such divergent results.

Bad luck could be, not just getting that 5% result which 95% accuracy implies, but some non-obvious difference in the volunteers (different genetics?), in the tea. or in the milk.

For example, eliminating lies doesn't "make the world a better place" unless it actually makes people happier; claiming so is just concealed deontologism.

I disagree. Not lying or not being lied to might well be a terminal value, why not? You that lies or doesn't lie is part of the world. A person may dislike being lied to, value the world where such lying occurs less, irrespective of whether they know of said lying. (Correspondingly, the world becomes a better place even if you eliminate some lying without anyone knowing about that, so nobody becomes happier in the sense of actually experiencing different emotions, assuming nothing else that matters changes as well.)

Of course, if you can only eliminate a specific case of lying by on the net making the outcome even worse for other reasons, it shouldn't be done (and some of your examples may qualify for that).

A quick Internet search turns up very little causal data on the relationship between cheating and happiness, so for purposes of this analysis I will employ the following assumptions:

a. Successful secret cheating has a small eudaemonic benefit for the cheater.
b. Successful secret lying in a relationship has a small eudaemonic cost for the liar.
c. Marital and familial relationships have a moderate eudaemonic benefits for both parties.
d. Undermining revelations in a relationship have a moderate (specifically, severe in intensity but transient in duration) eudaemonic cost for all parties involved.
e. Relationships transmit a fraction of eudaemonic effects between partners.

Under these assumptions, the naive consequentialist solution* is as follows:

1. Cheating is a risky activity, and should be avoided if eudaemonic supplies are short.
2. This answer depends on precise relationships between eudaemonic values that are not well established at this time.
3. Given the conditions, lying seems appropriate.
4. Yes.
5. Yes.
6. The husband may be better off. The wife more likely would not be. The child would certainly not be.

Are there any evident flaws in my analysis on the level it was performed?

* The naive consequentialist solution only accounts for direct effects of the actions of a single individual in a single situation, rather than the general effects of widespread adoption of a strategy in many situations - like other spherical cows, this causes a lot of problematic answers, like two-boxing.

For example, eliminating lies doesn't "make the world a better place" unless it actually makes people happier; claiming so is just concealed deontologism.

Just picking nits. Consequentialism =/= maximizing happiness. (The latter is a case of the former). So one could be a consequentialist and place a high value and not lying. In fact, the answers to all of your questions depend on the values one holds.

Or what Nesov said below.

It's okay to deceive people if they're not actually harmed and you're sure they'll never find out. In practice, it's often too risky.

1-3: This is all okay, but nevertheless, I wouldn't do these things. The reason is that for me, a necessary ingredient for being happily married is an alief that my spouse is honest with me. It would be impossible for me to maintain this alief if I lied.

4-5: The child's welfare is more important than my happiness, so even I would lie if it was likely to benefit the child.

6: Let's assume the least convenient possible world, where everyone is better off if they tell the truth. Then in this particular case, they would be better off as deontologists. But they have no way of knowing this. This is not problematic for consequentialism any more than a version of the Trolley Problem in which the fat man is secretly a skinny man in disguise and pushing him will lead to more people dying.

As an old quote from DanielLC says, consequentialism is "the belief that doing the right thing makes the world a better place". I now present some finger exercises on the topic:

It seems like it would be more aptly defined as "the belief that making the world a better place constitutes doing the right thing". Non-consequentialists can certainly believe that doing the right thing makes the world a better place, especially if they don't care whether it does.

Thanks for that, Price is a very knowledgeable New Testament scholar. Check out his interview at the commonsenseatheism podcast here, also covers his path to becoming a christian atheist.

CEV will be to maintain existing order.

Why? There must be very strong arguments for BEs to stop doing the Right Thing. And there's only one source of objections - children. And their volitions will be selfish and unaggregatable.

EDIT: What does utility-function-neutral mean?

EDIT: Ok. Ok. CEV will be to make BE's morale change and allow them to not eat children. So, FAI will undergo controlled shutdown. Objections, please?

EDIT: Here's yet another arguments.

Guidelines of FAI as of may 2004.

1. Defend humans, the future of humankind, and humane nature.

BEs will formulate this as "Defend BEs (except for the ceremony of BEing), the future of BEkind, and BE's nature."

1. Encapsulate moral growth.

BEs never considered, that child eating is bad. And it is good for them to kill anyone who thinks otherwise. There's no trend in moral that can be encapsulated.

1. Humankind should not spend the rest of eternity desperately wishing that the programmers had done something differently.

If they stop being BE they will mourn their wrong doings to the death.

1. Avoid creating a motive for modern-day humans to fight over the initial dynamic

Every single notion that FAI will make in lines of "Let's suppose that you are non-BE" will cause it to be destroyed.

1. Help people.

Help BEs everytime, but the ceremony of BEing.

How this will take FAI to the point that every conscious being must live?

What would happen if CEV was applied to the Baby Eaters?

My intuitions of CEV are informed by the Rawlsian Veil of Ignorance, which effectively asks: "What rules would you want to prevail if you didn't know in advance who you would turn out to be?"

Where CEV as I understand it adds more information - assumes our preferences are extrapolated as if we knew more, were more the kind of people we want to be - the Veil of Ignorance removes information: it strips people under a set of specific circumstances of the detailed information about what their preferences are, what their contignent histories brought them there, and so on. This includes things like what age you are, and even - conceivably - how many of you there are.

To this bunch of undifferentiated people you'd put the question, "All in favor of a 99% chance of dying horribly shortly after being born, in return for the 1% chance to partake in the crowning glory of babyeating cultural tradition, please raise your hands."

I expect that not dying horribly takes lexical precedence over any kind of cultural tradition, for any sentient being whose kin has evolved to sentience (it may not be that way for constructed minds). So I would expect the Babyeaters to choose against cultural tradition.

The obvious caveat is that my intuitions about CEV may be wrong, but lacking a formal explanation of CEV it's hard to check intuitions.