Hmmm... and I thought you were going to suggest that, if you succeded in making an AI that you must be god. I would've loved to be there to offer him that option instead. LOL!

Of course, even if what you said is what we really mean, I'm not sure which one is more effective at getting people to think, but your story shows that it's usually good (and at least entertaining) to try being more direct, every once in a while. I just find it easier to break through the social convention of politeness with humor.

Nice story.

Nice job, but the mention of Aumann's theorem looks a bit like a sleight of hand: did the poor fellow ever learn that the theorem requires the assumption of common priors?

This story is related to the phenomena whereby the most intelligent and educated religious folks are very careful to define their beliefs so that there can be no conflict with observations, while ordinary people are more prone to allow their religion to have implications, which are then subject to challenges like Eliezer's. It is fun to pick holes in the less educated views, but to challenge religion overall it seems more honest to challenge the most educated views. But I usually have trouble figuring out just what it is that the most educated religious folks think exactly.

I've mentioned before that my attempt to salvage a belief in God ultimately resulted in something like H. P. Lovecraft's Azathoth, which might not be too surprising as it was that ardent materialist's parody of the God of the Old Testament.

That was cruel. Fun, but cruel.

A woman who had stood nearby, listening to the conversation, said to me gravely, "That was beautiful."

And people wonder why men argue so often.

I only torture them to make them stronger.

Good catch, Pseudonymous. Robin, my guess is that they're crypto-skeptics, performing for their self-perceived comparative economic/social advantage. Eliezer, please don't make something that will kill us all.

"Funny, I've always thought that debates are one of the most entertaining forms of social interaction available."

We may not have rationality dojos, but in-person debating is as good an irrationality dojo as you're going to get. In debating, you're rewarded for 'winning', regardless of whether what you said was true; this encourages people to develop rhetorical techniques and arguments which are fully general across all possible situations, as this makes them easier to use. And while it may be hard to give public demonstrations of rationality, demonstrations of irrationality are easy: simply talk about impressive-sounding nonsense in a confident, commanding voice, and people will be impressed (look at how well Hitler did).

I think the idea of argument is to explore an issue, not "win" or "lose". If you enter an argument with the mentality that you must be right, you've rather missed the point. There wasn't an argument here, just a one-sided discussion. It was a bludgeoning by someone with training and practice in logical reasoning on someone without. It was both disgusting and pathetic, no different than a high-school yard bully pushing some kid's face in the dirt because he's got bigger biceps. Did the outcome of this "argument" stroke your ego?

All-in-all, I'm not sure this is a story you should want to share. To put this in uncomplicated terms, it makes you sound like a real a$$hole.

Where do people get the impression that we all have the right not to be challenged in our beliefs? Tolerance is not about letting every person's ideas go unchallenged; it's about refraining from other measures (enforced conformity, violence) when faced with intractable personal differences.

As for politeness, it is an overrated virtue. We cannot have free and open discussions, if we are chained to the notion that we should not challenge those that cannot countenance dissent, or that we should be free from the dissent of others. Some people should be challenged often and publicly. Of course, the civility of these exchanges matters, but, as presented by Eliezer, no serious conversational fouls or fallacies were committed in this case (contemptuous tone, ad hominems, tu quoque or other Latinate no-nos, etc.).

Mark D,

How do you know what the putative AI "believes" about what is advantageous or logical? How do you know that other humans are feeling compassion? In other words, how you feel about the Turing test, and how, other than their behavior, would you be able to know about what people or AIs believe and feel?

We may not have rationality dojos, but in-person debating is as good an irrationality dojo as you're going to get. In debating, you're rewarded for 'winning', regardless of whether what you said was true

Only if you choose to approach it that way.

Mark M.,

"His beliefs have great personal value to him, and it costs us nothing to let him keep them (as long as he doesn’t initiate theological debates). Why not respect that?"

Values may be misplaced, and they have consequences. This particular issue doesn't have much riding on it (on the face of it, anyway), but many do. Moreover, how we think is in many ways as important as what we think. The fellows ad hoc moves are problematic. Ad hoc adjustments to our theories/beliefs to avoid disconfirmation are like confirmation bias and other fallacie... (read more)

His rationality was my business.

JL, I’ve programmed in several languages, but you have me correctly pegged as someone who is more familiar with databases. And since I’ve never designed anything on the scale we’re discussing I’m happy to defer to your experience. It sounds like an enormously fun exercise though.

My original point remains unanswered however. We’re demanding a level of intellectual rigour from our monotheistic party goer. Fair enough. But nothing I’ve seen here leads me to believe that we’re as open minded as we’re asking him to be. Would you put aside your convictions and a... (read more)

Mark D,

"JL, I’ve programmed in several languages, but you have me correctly pegged as someone who is more familiar with databases. And since I’ve never designed anything on the scale we’re discussing I’m happy to defer to your experience. It sounds like an enormously fun exercise though."

There are programs (good ol' chatter bots) that use methods like you supposed, but they are far from promising. No need to defer to me-- I am familiar with machine learning methods, some notable programs and the philosophical debate, but I am far from an expert ... (read more)

A less personal response to the second bit I quoted from Mark D: Yes, changing our beliefs in the face of good evidence and argument is desirable, and to the extent that we are able to do this we can be called critical thinkers.

Would you put aside your convictions and adopt religion if a skilful debater put forward an argument more compelling than yours?

To the extent the answer is "No" my atheism would be meaningless. I hope the answer is "Yes", but I have not been so tested (and do not expect to be; strong arguments for false theses should not exist).

First: The argument wasn't the author being an a$$hole. He was stating the nature of his business, which is a very normal thing to do at a social gathering. (We are, to a disturbing extend, defined by our income.) Godboy dismissed his profession as quixotic, leading the author to the notion that if he created a working AI, that it would disprove God, in the mind of his coparticipant in discussion. This was a logical inferrence, based on the statement that inspired it.

Second: The only winner in a conversation is the person who learns something. I believe, t... (read more)

Wouldn't it be easier to say, an AI is not a soul? In what sense do these two words have the same meaning? An AI is a non-existant entity which, due to the unflagging faith of some, is being explored. A soul is an eternal being granted life (human only?) by god (should that be capitalized?) Comparing them is what leads to the problem.

Before using Aumann one should ask, "What does this guy know that I don't?"

Douglas, (1) what makes you think that anyone was suggesting that "AI" and "soul" have the same meaning?, (2) in what way would "an AI is not a soul" be a useful substitute for anything else said in this discussion?, and (3) why should comparing the two notions lead to any problems, and in particular to whatever you're calling "the problem" here?

I don't think it's any more obvious that there are no AIs than that there are no souls. That is: perhaps, despite appearances, my computer is really intelligent, and thinks i... (read more)

g- the man said, "I don't believe AI is possible because only God can make a soul." "...If I can make an AI it proves your religion false?" Somebody in this exchange has equated the making of an AI with the making of a soul. That's why I would suggest that the words have been confused. An AI is not a soul would be useful in this discussion because it would clarify that the making of one would not invalidate the existence of the other or the statement that "only God can make a soul". Comparing the two notions would not be a pr... (read more)

Douglas: OK, I hadn't realised you were talking about him; my bad. And, sure, another approach Eliezer could have taken is to say "an AI and a soul aren't the same thing". But I don't see why that would be any improvement on what he actually did do.

Also: "soul" is used vaguely enough that I don't think Eliezer could justifiably claim that an AI wouldn't have to be a soul. If his interlocutor believed, e.g., that a soul is what it takes in order to have real beliefs, feelings, will, etc., then saying "oh no, I'm not talking about so... (read more)

g- you ask good questions. My point about AI and religion is that rather than pretending that one is related to the other, AI would benefit from clearing up this confusion. (So would the religious) Perhaps the way Elizer went about it was OK I would define "soul" as a non-corporeal being that exists separable from the body and that survives body death. (I want to say something about the soul being the true source of consciousness and ability-- OK, I said it)

cool :)

Seems more like Aikido. I sense a broken spirit more than a redirection of thought or processes of it and his belief system. Simply put, honey has always gotten me more flies than vinegar.

I have to heartily disagree with those that seem to think it impolite to disagree with the religious. Remember this same person is going to go out and make life and death decisions for himself and others. Notice also that it was the theist who started the debate.

I attended a lecture by noted theologian Alvin Plantinga, about whether miracles are incompatible with science. Most of it was "science doesn't say it's impossible, so there's still a chance, right?"-type arguments. However, later on, his main explanation for why it wasn't impossible that God could intervene from outside a closed system and still not violate our laws of physics was that maybe God works through wavefunction collapse. Maybe God creates miracles by causing the right wavefunction collapses, resulting in, say, Jesus walking on water, rising from the dead, unscrambling eggs, etc.

Recalling this article, I wrote down and asked this question when the time came:

"The Many-Worlds Interpretation is currently [I said "currently" because he was complaining earlier about other philosophers misrepresenting modern science] one of the leading interpretations of quantum mechanics. The universe splits off at quantum events, but is still deterministic, and only appears probabilistic from the perspective of any given branch. Every one of the other branches still exists, including ones where Jesus doesn't come back. If true, how does this affect your argument?&q

There's a theorem of rationality called Aumann's Agreement Theorem which shows that no two rationalists can agree to disagree. If two people disagree with each other, at least one of them must be doing something wrong.

This seems like one of those things that can be detrimental if taught in isolation.

It may be a good idea to emphasize that only one person in a disagreement doing something wrong is far less likely than both sides in a disagreement doing something wrong.

I can easily imagine someone casually encountering that statement, and taking it to ins... (read more)

I would have loved to watch that.

That was beautiful!

I wrote a long post saying what several people had already said years ago, then shortened it. Still, because this post has made me mad for years:

1) Of COURSE people can agree to disagree! If not, EY is telling this guy that no two rationalists currently disagree about anything. If THAT were true, it's so fascinating that it should have derailed the whole conversation!

(Leaving aside, for the moment, the question of whether Aumann's theory "requires" a rationalist to agree with a random party goer. If it really did, then the party goer could convince EY by ... (read more)

While I understand the absolute primal urge to stomp on religious texts used to propagate compulsory heterosexuality, I do think this exchange ended up a bit of a poor game, when it seems like he'd be mostly interested in discussing how the emotions of programmed thought might differ from ours (and that's a fun well to splash around in, for a while)(though deposing of cult-friendly rhetoric is valuable too, even if you have to get nasty).

I'm mildly concerned about the Reign Of Terror Precept, but I also understand it. It's just disappointing to know that t... (read more)

I wonder if this post would have gotten a better reception if the stooge had been a Scientologist or a conspiracy theorist or something, instead of just a hapless normie.

This is a great example of an ad hoc hypothesis.