pianoforte611 comments on Open thread, Oct. 27 - Nov. 2, 2014 - Less Wrong

5 Post author: MrMind 27 October 2014 08:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (400)

You are viewing a single comment's thread. Show more comments above.

Comment author: pianoforte611 29 October 2014 07:43:46PM 0 points [-]

Are you directing this at LW? Ie. is there a sociopath that you think is bad for our community?

Comment author: Viliam_Bur 30 October 2014 09:03:12AM *  0 points [-]

Well, I suspect Eugine Nier may have been one, to show the most obvious example. (Of course there is no way to prove it, there are always alternative explanations, et cetera, et cetera, I know.)

Now that was an online behavior. Imagine the same kind of person in real life. I believe it's just a question of time. Using the limited experience to make predictions, such person would be rather popular, at least at the beginning, because they would keep using the right words that are tested to evoke a positive response from many lesswrongers.

Comment author: IlyaShpitser 30 October 2014 09:59:43AM *  3 points [-]

A "sociopath" is not an alternative label for [someone I don't like.] I am not sure what a concise explanation for the sociopath symptom cluster is, but it might be someone who has trouble modeling other agents as "player characters", for whatever reason. A monster, basically. I think it's a bad habit to go around calling people monsters.

Comment author: Viliam_Bur 30 October 2014 01:46:19PM *  6 points [-]

I know; I know; I know. This is exactly what makes this topic so frustratingly difficult to explain, and so convenient to ignore.

The thing I am trying to say is that if a real monster would come to this community, sufficiently intelligent and saying the right keywords, we would spend all our energy inventing alternative explanations. That although in far mode we admit that the prior probability of a monster is nonzero (I think the base rate is somewhere around 1-4%), in near mode we would always treat it like zero, and any evidence would be explained away. We would congratulate ourselves for being nice, but in reality we are just scared to risk being wrong when we don't have convincingly sounding verbal arguments on our side. (See Geek Social Fallacy #1, but instead of "unpleasant" imagine "hurting people, but only as much as is safe in given situation".) The only way to notice the existence of the monster is probably if the monster decides to bite you personally in the foot. Then you will realize with horror that now all other people are going to invent alternative explanations why that probably didn't happen, because they don't want to risk being wrong in a way that would feel morally wrong to them.

I don't have a good solution here. I am not saying that vigilantism is a good solution, because the only thing the monster needs to draw attention away is to accuse someone else of being a monster, and it is quite likely that the monster will sound more convincing. (Reversed stupidity is not intelligence.) Actually, I believe this happens rather frequently. Whenever there is some kind of a "league against monsters", it is probably a safe bet that there is a monster somewhere at the top. (I am sure there is a TV Tropes page or two about this.)

So, we have a real danger here, but we have no good solution for it. Humans typically cope with such situations by pretending that the danger doesn't exist. I wish we had a better solution.

Comment author: NancyLebovitz 30 October 2014 08:37:13PM 2 points [-]

I can believe that 1% - 4% of people have little or no empathy and possibly some malice in addition. However, I expect that the vast majority of them don't have the intelligence/social skills/energy to become the sort of highly destructive person you describe below.

Comment author: Viliam_Bur 30 October 2014 10:36:49PM *  3 points [-]

That's right. The kind of person I described seems like combination of sociopathy + high intelligence + maybe something else. So it is much less than 1% of population.

(However, their potential ratio in rationalist community is probably greater than in general population, because our community already selects for high intelligence. So, if high intelligence would be the only additional factor -- which I don't know whether it's true or not -- it could again be 1-4% among the wannabe rationalists.)

Comment author: Lumifer 31 October 2014 01:05:53AM 2 points [-]

The kind of person I described seems like combination of sociopathy + high intelligence + maybe something else.

I would describe that person as a charismatic manipulator. I don't think it requires being a sociopath, though being one helps.

Comment author: NancyLebovitz 30 October 2014 10:59:15PM 1 point [-]

The kind of person you described has extraordinary social skills as well as being highly (?) intelligent, so I think we're relatively safe. :-)

I can hope that a people in a rationalist community would be better than average at eventually noticing they're in a mind-warping confusion and charisma field, but I'm really hoping we don't get tested on that one.

Comment author: Viliam_Bur 31 October 2014 09:39:38AM 2 points [-]

I think we're relatively safe

Returning to the original question ("Where are you right, while most others are wrong? Including people on LW!"), this is exactly the point where my opinion differs from the LW consensus.

I can hope that a people in a rationalist community would be better than average at eventually noticing they're in a mind-warping confusion and charisma field

For a sufficiently high value of "eventually", I agree. I am worried about what would happen until then.

I'm really hoping we don't get tested on that one.

I'm hoping that this is not the best answer we have. :-(

Comment author: NancyLebovitz 31 October 2014 02:01:33PM 3 points [-]

To what extent is that sort of sociopath dependent on in-person contact?

Thinking about the problem for probably less than five minutes, it seems to me that the challenge is having enough people in the group who are resistant to charisma. Does CFAR or anyone else teach resistance to charisma?

Would noticing when one is confused and writing the details down help?

Comment author: Viliam_Bur 31 October 2014 09:24:16PM *  4 points [-]

In addition to what I wrote in the other comment, a critical skill is to imagine the possibility that someone close to you may be manipulating you.

I am not saying that you must suspect all people all the time. But when strange things happen and you notice that you are confused, you should assign a nonzero value to this hypothesis. You should alieve that this is possible.

If I may use the fictional evidence here, the important thing for Rational!Harry is to realize that someone close to him may be Voldemort. Then it becomes a question of paying attention, good bookkeeping, gathering information, and perhaps making a clever experiment.

As long as Harry alieves that Voldemort is far away, he is likely to see all people around him as either NPCs or his party members. He doesn't expect strategic activity from the NPCs, and he believes that his party members share the same values even if they have a few wrong beliefs which make cooperation difficult. (For example, he is frustrated that Minerva doesn't trust him more, or that Dumbledore is okay with the idea of death, but he wouldn't expect either of them trying to hurt him. And the list of nice people includes also Quirrell, which is the most awesome of them all.) He alieves that he lives in a relatively safe bubble, that Voldemort is somewhere outside of the bubble, and that if Voldemort tried to enter the bubble, it would be an obviously extraordinary event that he would notice. (Note: This is no longer true in the recent chapters.)

Comment author: Viliam_Bur 31 October 2014 03:51:17PM *  4 points [-]

Perhaps we should taboo "resistance to charisma" first. What specifically are we trying to resist?

Looking at an awesome person and thinking "this is an awesome person" is not harmful per se. Not even if the person uses some tricks to appear even more awesome than they are. Yeah, it would be nice to measure someone's awesomeness properly, but that's not the point. A sociopath may have some truly awesome traits, for example genuinely high intelligence.

So maybe the thing we are trying to resist is the halo effect. An awesome person tells me X, and I accept it as true because it would be emotionally painful to imagine that an awesome person would lie to me. The correct response is not to deny the awesomeness, but to realize that I still don't have any evidence for X other than one person saying it is so. And that awesomeness alone is not expertise.

But I think there is more to a sociopath than mere charisma. Specifically, the ability to lie and harm people without providing any nonverbal cues that would probably betray a neurotypical person trying to do the same thing. (I suspect this is what makes the typical heuristics fail.)

Would noticing when one is confused and writing the details down help?

Yes, I believe so. If you already have a suspicion that something is wrong, you should start writing a diary. And a very important part would be, for every information you have, write down who said that to you. Don't report your conclusions; report the raw data you have received. This will make it easier to see your notes later from a different angle, e.g. when you start suspecting someone you find perfectly credible today. Don't write "X", write "Joe said: X", even if you perfectly believe him at the moment. If Joe says "A" and Jane says "B", write "Joe said A. Jane said B" regardless of which one of them makes sense and which one doesn't. If Joe says that Jane said X, write "Joe said that Jane said X", not "Jane said X".

Also, don't edit the past. If you wrote "X" yesterday, but today Joe corrected you that he actually said "Y" yesterday but you have misunderstood it, don't erase the "X", but simply write today "Joe said he actually said Y yesterday". Even if you are certain that you really made a mistake yesterday. When Joe gives you a promise, write it down. When there is a perfectly acceptable explanation later why the promise couldn't be fulfilled, accept the explanation, but still record that for perfectly acceptable reasons the promise was not fulfilled. Too much misinformation is a red flag, even if there is always a perfect explanation for each case. (Either you are living in a very unlikely Everett branch, or your model is wrong.) Even if you accept an excuse, make a note of the fact that something had to be excused.

Generally, don't let the words blind you from facts. Words are also a kind of facts (facts about human speech), but don't mistake "X" for X.

I think gossip is generally a good thing, but only if you can follow these rules. When you learn about X, don't write "X", but write "my gossiping friend told me X". It would be even better to gossip with friends who follow similar rules; who can make a distinction between "I have personally seen X" and "a completely trustworthy person said X and I was totally convinced". But even when your friends don't use this rule, you can still use it when speaking with them.

The problem is that this kind of journaling has a cost. It takes time; you have to protect the journal (the information it contains could harm not only you but also other people mentioned there); and you have to keep things in memory until you get to the journal. Maybe you could have some small device with you all day long where you would enter new data; and at home you would transfer the daily data to your computer and erase the device.

But maybe I'm overcomplicating things and the real skill is the ability to think about anyone you know and ask yourself a question "what if everything this person ever said to me (and to others) was a lie; what if the only thing they care about is more power or success, and they are merely using me as a tool for this purpose?" and check whether the alternative model explains the observed data better. Especially with the people you love, admire, of depend on. This is probably useful not only against literally sociopaths, but other kinds of manipulators, too.

Comment author: Lumifer 31 October 2014 04:16:23PM 3 points [-]

the challenge is having enough people in the group who are resistant to charisma.

Since we are talking about LW, let me point out that charisma in meatspace is much MUCH more effective than charisma on the 'net, especially in almost-purely-text forums.

Comment author: Azathoth123 02 November 2014 04:12:50AM 2 points [-]

Does CFAR or anyone else teach resistance to charisma?

Well, consider who started CFAR (and LW for that matter) and how he managed to accomplish most of what he has.

Comment author: IlyaShpitser 02 November 2014 03:47:05PM *  -1 points [-]

resistance to charisma?

Ex-cult members seem to have fairly general antibodies vs "charisma." Perhaps studying cults without being directly involved might help a little as well, it would be a shame if there was no substitute for a "school of hard knocks" that actual cult membership would be.

Incidentally, cults are a bit of a hobby of mine :).

Comment author: arromdee 30 October 2014 06:47:49PM 1 point [-]

Whenever there is some kind of a "league against monsters", it is probably a safe bet that there is a monster somewhere at the top. (I am sure there is a TV Tropes page or two about this.)

https://allthetropes.orain.org/wiki/Hired_to_Hunt_Yourself

Comment author: Lumifer 30 October 2014 03:31:59PM 3 points [-]

Well, I suspect Eugine Nier may have been one, to show the most obvious example.

Why do you suspect so? Gaming ill-defined social rules of an internet forum doesn't look like a symptom of sociopathy to me.

You seem to be stretching the definition too far.

Comment author: Viliam_Bur 30 October 2014 05:07:03PM 3 points [-]

Abusing rules to hurt people is at least a weak evidence. Doing it persistently for years, even more so.