Eliezer, do you have a rough plan for when you will start programming an AI?

The "probability" of an event is how much anticipation you have for that event occurring. For example if you assign a "probability" of 50% to a tossed coin landing heads then you are half anticipating the coin to land heads.

Silas: My post wasn't meant to be "shockingly unintuitive", it was meant to illustrate Eliezer's point that probability is in the mind and not out there in reality in a ridiculously obvious way.

Am I somehow talking about something entirely different than what Eliezer was talking about? Or should I complexificationafize my vocabulary to seem more academic? English isn't my first language after all.

Here is another example me, my dad and my brother came up with when we were discussing probability.

Suppose there are 4 card, an ace and 3 kings. They are shuffled and placed face side down. I didn't look at the cards, my dad looked at the first card, my brother looked at the first and second cards. What is the probability of the ace being one of the last 2 cards. For me: 1/2 For my dad: If he saw the ace it is 0, otherwise 2/3. For my brother: If he saw the ace it is 0, otherwise 1.

How can there be different probabilities of the same event? It is because probability is something in the mind calculated because of imperfect knowledge. It is not a property of reality. Reality will take only a single path. We just don't know what that path is. It is pointless to ask for "the real likelihood" of an event. The likelihood depends on how much information you have. If you had all the information, the likelihood of the event would be 100% or 0%.

"Hard AI Future Salon" lecture, good talk. Most of the audience's questions however were very poor.

One more comment about the mind projection fallacy. Eliezer, you also have to keep in mind that the goal of a sci-fi writer is to make a compelling story which he can sell. Realism is only important in so far as it helps him achieve this goal. Agreed on the point that it's a fallacy, but don't expect it to change unless the audience demands/expects realism. http://tvtropes.org/ if full of tropes that illustrate stuff like that.

Good post. I have a feeling I've read this very same example before from you Eliezer. I can't remember where.

OK, time to play:

Q: Why am I confused by the question "Do you have free will?"? A: Because I don't know what "free will" really means. Q: Why don't I know what "free will" means? A: Because there is no clear explanation of it using words. It's an intuitive concept. It's a feeling. When I try to think of the details of it, it is like I'm trying to grab slime which slides through my fingers. Q: What is the feeling of "free will"? A: When people talk of "free will" they usually put it thusly. If one has "free will", he is in control of his own actions. If one doesn't have "free will" then it means outside forces like the laws of physics control his actions. Having "free will" feels good because being in control feels better then being controlled. On the other hand, those who have an appreciation for the absolute power of the laws of physics feel the need to bow down to them and acknowledge their status as the ones truly in control. The whole thing is very tribal really. Q: Who is in control, me or the laws of physics? A: Since currently saying [I] is equivalent to saying [a specific PK shaped collection of atoms operating on the laws of physics], then saying "I am in control" is equivalent to saying "a specific PK shaped collection of atoms operating on the laws of physics is in control". The laws of physics are not an outside force apart from me, they are inside me too. Q: Why do people have a tendency to believe their minds are somehow separate from the rest of the universe? A: Ugghhh... I don't know the details well enough to answer that.

Ughh more homework. Overcoming bias should have a sister blog called Overcoming laziness.

This reminds me of an item from a list of "horrible job interview questions" we once devised for SIAI:

Would you kill babies if it was intrinsically the right thing to do? Yes/No

If you circled "no", explain under what circumstances you would not do the right thing to do: ___________________

If you circled "yes", how right would it have to be, for how many babies? ___________________

What a horrible horrible question. My answer is ... what do you mean when you say "intrinsically the right thing to do"? The "right thing" according to whom? If it was the right thing according to an authority figure but I disagreed, I probably would not do it. If the circumstances were so extreme that I truly believed it's the right thing(eg: not killing a baby results in the baby's death anyway + 1 million babies) then I would kill babies(assuming I could overcome my aversion to killing).

Actually I don't really know how I would react. This is how I wish I would act. Calmly theorising in front of the computer never having experienced circumstances remotely as awful is not the same as being in those circumstances when the fear and dread overtakes you. There would probably be a significant shift from what I consider and feel is "me" right now to the "me" I would become in that hypothetical situation.

Lately I've been thinking about "mind killing politics". I have come to the conclusion that this phenomenon is pretty much present to some degree in any kind of human communication where being wrong means you or your side lose status.

It is incorrect to assume that this bias can only occurs when the topic involves government, religion, liberalism/conservatism or any other "political" topics. Communicating with someone who has a different opinion than you is sufficient for the "mind killing politics" bias to start creeping in.

The pressure to commit "mind killing politics" type biases is proportional to how much status one or one's side has to lose for being wrong in any given disagreement. This doesn't mean the bias can't be mixed or combined with other biases.

I've also noticed six factors that can increase or decrease the pressure to be biased.

1)If you are talking to your friends or people close to you that you trust then the pressure to be right will be reduced because they are less likely to subtract status from you for being wrong. Talking to strangers will increase it.

2)Having an audience will increase the pressure to be right. That's because the loss of status for being wrong is multiplied by the number of people that see you lose(each weighted for how important it is for them to consider you as having a high status).

3)If someone is considered an 'expert', the pressure to be right will be enormous. Thats because experts have special status for being knowledgeable about a topic and getting answers about it right. Every mistake is seen as reducing that expertise and proportionatly reducing the status of the expert. Being wrong to someone considered a non expert is even more painful then being wrong to an expert.

4)It is very hard psychologically to disagree with authority figures or the group consensus. Therefore "mind killing politics" biases will be replaced by other biases when there is disagreement with authority figure or the group consensus but will be amplified against those considered outside the social group.

5)People will easily spot "mind killing politics" biases in the enemy side but will deny, not notice or rationalize the same biases in themselves.

6)And finally, "mind killing politics" biases can lead to agitation(ei. triggering of the fight or flight response) which will amplify biased thinking.