LESSWRONG
LW

Comment Permalink

This seems like a very fun game, but I'm not sure how much merit it has as a method of determining which side in a debate is the most informed.

Much like the actual Turing Test, it also seems vulnerable to hard-ball inquisitorial tactics. There are probably quite a few questions you could ask which lie outside the ostensible scope of the debate, but which would correlate strongly with a user's true beliefs, or require faculties the respondent simply doesn't have in order to answer satisfactorily. I'm pretty sure if I asked a fake-atheist to name ten atheist-related figures that inspired them, they'd come up with a predictable, odd, or obviously Googled list. Equally, if I was posing as a Christian I'd struggle to name ten inspiring Bible passages that weren't either predictable, odd, or obviously pulled off Google.

Kaj_Sotala14y20

You could just forbid tactics violating the spirit of the test, such as these ones.

4Paul Crowley14y

Yes, I think this is the biggest drawback with the proposal. Lots of us understand pretty well the arguments that Christians make for their position, but we wouldn't be able to produce the shibboleths that Christians would look for.

11Desrtopa14y

I've been an atheist for most of my life, and I'm pretty sure I don't have that many atheism-related figures who've inspired me. I'd have a hard time coming up with a list even if I interpreted the question as broadly as possible.

See in context

46 The Ideological Turing Test

by Paul Crowley

25th Jun 2011

1 min read

46

Bryan Caplan:

Mill states it well: "He who knows only his own side of the case knows little of that." If someone can correctly explain a position but continue to disagree with it, that position is less likely to be correct. And if ability to correctly explain a position leads almost automatically to agreement with it, that position is more likely to be correct.

[...] Put me and five random liberal social science Ph.D.s in a chat room. Let liberal readers ask questions for an hour, then vote on who isn't really a liberal. Then put Krugman and five random libertarian social science Ph.D.s in a chat room. Let libertarian readers ask questions for an hour, then vote on who isn't really a libertarian. Simple as that.

My challenge: Nail down the logistics, and I'll happily bet money that I fool more voters than Krugman.

Leah at Unequally Yoked:

Just like Caplan, I'd like to put my money where my mouth is and play in an ideological Turing Test against a Christian blogger.

UPDATE: Two Christians have contacted me to tell me they're interested. Please suggest format ideas for us to talk over and let me know if you'd like to join in!

update to clarify: When the panel is made up of mostly genuine Blues and one or a few Greens pretending to be Blue, then the judges are all Blue.

Ideological Turing Tests

Frontpage

46

Mentioned in

25Training Regime Day 17: Deflinching and Lines of Retreat

15*Another* Double Crux Framework

2How do you know you are right when debating? Calculate your AmIRight score.

New Comment

30 comments, sorted by

top scoring

Click to highlight new comments since: Today at 7:34 AM

[-]CronoDAS14y170

A while back, David Brin wrote up a proposal for "disputation arenas".

An interesting part of the competition/argument process:

Phase Four: The Paraphrasing Challenge

Here is a unique, but crucial phase. After both manifestos are declared logically usable, a distinct period -- say a month -- will be given to each side so they may paraphrase the other side's position.

This step aims to ensure that each party has actually read and understood where the other one stands, so they aren't simply shouting past each other at chimeric caricatures. Paraphrasing is hard to do when you've spent years demonizing the opposition, calling them venal or stupid and dismissing their concerns. Success at paraphrasing will be seen as a way of winning credibility. It means "I do understand my opponents... so my disagreement with them is well-informed."

Each side may lace their formal paraphrasing with asterisks and footnotes asserting that the statements they are describing are idiotic. That's fine. But if they fail to depict their opponents' point of view in a manner that the jury finds at least generally accurate, they will be disqualified.

[-]sixes_and_sevens14y130

This seems like a very fun game, but I'm not sure how much merit it has as a method of determining which side in a debate is the most informed.

[-]Desrtopa14y110

I'm pretty sure if I asked a fake-atheist to name ten atheist-related figures that inspired them, they'd come up with a predictable, odd, or obviously Googled list.

I've been an atheist for most of my life, and I'm pretty sure I don't have that many atheism-related figures who've inspired me. I'd have a hard time coming up with a list even if I interpreted the question as broadly as possible.

[-]Emile14y20

Same here. So if I had to pretend to be a Christian, I'd waffle around a bit just like I do if you ask me that about atheists.

[-]sixes_and_sevens14y20

What if it was just "name ten atheists who you find inspiring"?

[-]Desrtopa14y120

I'd probably have to do some googling to make sure the list really is composed solely of atheists.

[-]Paul Crowley14y40

[-]Kaj_Sotala14y20

You could just forbid tactics violating the spirit of the test, such as these ones.

[-][anonymous]14y120

Indeed, I'll happily bet that any libertarian with a Ph.D. from a top-10 social science program can fool more voters than Krugman. We learn his worldview as part of the curriculum. He learns ours in his spare time - if he chooses to spare it.

If we take this as a test, It should be noted that it seems to be biased against popular/conventional/establishment/dominant schools of thought. Historically speaking religious minorities often know a lot about the dominant religion, though the slant may be biased.

However depending on which other tests we use in our investigation this might be a feature not a bug.

[-]Paul Crowley14y90

I think there are several big problems with this that make it a lot less useful than you might hope as a way of telling which side better understands the other side's arguments - leaving aside the big problem with picking a "side" in an argument. Nonetheless, I think it's likely to be informative and worth a go!

As stated, it sounds like everyone is expected to be on the same IRC channel. That doesn't seem like such a good idea - the imitators can for the most part simply listen to the genuines to work out what to say. What you really want is a way to ask the panel a question and have their answers revealed only once they have all been submitted. However, I can't immediately think of an existing software platform that makes this convenient.

I don't know how best to minimise the shibboleth problem. At the very least, we could remind panellists and judges that shibboleths may be more easily imitated than real winning arguments. Bear in mind that the judges are effectively all Blues pretending to be Greens, in that they want to test how well the panel understand the arguments against Green positions.

[-]CronoDAS14y90

"Things that try to look like things often look more like things than things. Well known fact."

—Granny Weatherwax, Wyrd Sisters (by Terry Pratchett)

In other words, you win this game not by trying to imitate actual people who hold a position, but by imitating the stereotype of people that hold that position.

[-]Paul Crowley14y130

If the judges were random disinterested people, maybe, but the judges are people sympathetic to the stated position of the panel; they're less likely to be fooled by someone parroting a stereotype of themselves. I think that to cryonicists, someone trying to fulfill a stereotype of a cryonicists among a panel of genuine cryonicists would stick out like a sore thumb.

[-]mutterc14y50

Depends on the position. Remember Poe's Law - there are some positions where it's impossible to distinguish a parody from the real thing.

[-]satt14y80

There are people who do this for free. They're usually known as Internet trolls.

[-]MixedNuts14y50

No, they can't fool supporters of their fake position (e.g. trolls on Alternet pretending to be atheists are quite different from trolls who are sincere about atheism and trolling about another topic).

[-]satt14y30

Depends on the troll.

[-]Eugine_Nier14y80

I suspect the results would depend heavily on the jury. Specifically a blue pretending to be a green, could probably fool fellow blues much more easily then he could fool actual greens.

[-]Paul Crowley14y150

In this proposal, whenever a Blue is pretending to be a Green he is trying to fool Green judges.

[-]Manfred14y70

Troublesome because you have an uncontrolled variable - the investigative ability of the people you're trying to fool.

[-]RHollerith14y00

That's why you make it so that there are a lot of them.

[-]Manfred14y50

Not noise - I really mean a variable. For example, say you want to tax cigarettes, and someone else wants to ban cigarettes. To find out who knows the other person's position better you trade places and try to fool some large group of their peers. The taxer chats with some banners for a while, and the banner chats with some taxers for a while. Then the two groups write down how knowledgeable they thought people were.

But what if taxers tend to be introverts, and banners tend to be police officers who are masters at interrogation? Or, more plausibly, what if there are group-specific things that will impact peoples' judgement of your knowledge other than the substance of the issue? There are all sorts of things that could cause groups to grade transplants differently, and these factors are left uncontrolled if you just do the swap as suggested.

[-]RHollerith14y10

Ah, OK then. (Selection bias.)

[-]CharlieSheen14y30

If its carreid out it should be entertaining. Some comparisons among other ideological constructs would be interesitng as well. I wonder how say someone like Paul Gottfried or John Derbyshire would do immitating left or Neoconservative positions.

[-]Bo10201014y30

I asked Leah about participating in her test. I think I will find it challenging the emulate a Christian's viewpoint, instead of mocking it.

That is, normally when I've examined Christian thought, I've approached it from the pretense "let me see where this fails."

I'm interested to see if I can construct an argument that I myself as an atheist would find at least superficially compelling - that is, I want to avoid doing what pretty much every popular Christianity book does.

[-]Paul Crowley14y80

If you try to present radically better arguments for Christianity than you think most Christians present, you will sound out of step with the rest of the panel to an audience of Christian judges.

[-]jfm14y10

I think this is correct. If you want to successfully pose as a Christian, you might be well advised to read a bunch of C. S. Lewis, and then imitate his arguments and style. I say this because I think his books constitute the most accessible body of reasonable-but-still-wrong arguments in favour of Christian orthodoxy. If you can quote him, all the better, because being able to quote C. S. Lewis is a high-status marker among people who have both a self-identity as Christians, and a self-identity as intellectuals.

[-]Bo10201014y20

Voting for the Christian / "Christian" half is open - see Leah's site.

I had fun figuring these out. Once it's done I'd be interested to see what criteria LW users used to determine someone's real beliefs...

[-]Zack_M_Davis14y10

First round (Christians pretending to be atheists, atheists being sincere) voting is open.

[-]Emile14y00

Sounds like a neat idea! I've been looking for these kinds of "meta" approaches to political discussion, and this looks like a pretty good one I hadn't thought of!

An improvement might be to restrict the discussion to a topic narrower than "libertarians vs. liberals" or "christians versus atheists" - for example "labour laws", "immigration", "the documentary hypothesis", "morality of homsexuality", etc.

This looks like it'd have a much better likelihood-I-change-my-mind / invested-time ratio than an "ordinary" poltical debate.

[-][anonymous]14y00

I asked Leah about participating in her test. I think I will find it challenging the emulate a Christian's viewpoint, instead of mocking it.

That is, normally when I've examined Christian thought, I've approached it from the pretense "let me see where this fails."

[This comment is no longer endorsed by its author]Reply

Moderation Log