CronoDAS comments on Harry Potter and the Methods of Rationality discussion thread, part 14, chapter 82 - Less Wrong

7 Post author: FAWS 04 April 2012 02:53AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (790)

You are viewing a single comment's thread.

Comment author: CronoDAS 04 April 2012 03:03:06AM 13 points [-]

Honestly, Harry is placing far too little weight on the hypothesis that Hermione actually did do exactly what she confessed to under Veritaserum.

Story-logic would indicate that she is indeed innocent, and we as readers have evidence that someone has indeed been messing with her mind, but Harry doesn't know what we as readers know. And, to be honest, in a similar situation in the real world, I'd also conclude that the 11 year old probably did indeed do exactly what she is accused of doing.

Comment author: ArisKatsaris 04 April 2012 03:16:44AM *  10 points [-]

Story-logic would indicate that she is indeed innocent, and we as readers have evidence that someone has indeed been messing with her mind, but Harry doesn't know what we as readers know.

Harry's had 7 months to know that Hermione isn't a sociopath or a psychopath, that she's a very kind and moral and ethical person instead.

What's the prior probability he should therefore assign to this person, out of all of Hogwarts, to be the one to commit a cold-blooded murder on another 11-year-old kid? I think he's giving the hypothesis of her actual guilt pretty much all the weight that it deserves - effectively zero.

Comment author: Eugine_Nier 04 April 2012 05:38:51AM 12 points [-]

Outside view: when someone in a similar situations does do something horrible, all of his friends and family insist that they "have no idea how he could have done something like this".

Comment author: bogdanb 04 April 2012 06:22:08AM 8 points [-]

I wonder how much of that is a “don’t speak bad of the dead” reflex, or “nobody could have seen it, so it’s not my fault I didn’t”, or even just “I’m such a good & loving friend/relative I didn’t see anything wrong with him”.

I’m sure there are cases that really came out of the blue, but I also have a nagging feeling that if you could interview the same people before the something horrible, and do it from an insider point of view (i.e., a question asked by another friend of the interviewee rather than by a reporter), a lot of answers would be of the “he’s kind of a weirdo” type.

Comment author: NancyLebovitz 04 April 2012 01:23:54PM 2 points [-]

In some cases, people who commit major violence have a history of minor violence.

However, another possibility is that even people who commit major violence have people they like and/or want to please, and behave better in some contexts than in others.

Comment author: DanArmak 04 April 2012 03:52:35PM *  7 points [-]

Now update on the amount of people who call somebody "a weirdo" who does not end up murdering anyone. And add the negative halo effect, and fundamental attribution fallacy, from knowing in hindsight that the person you're talking about has recently murdered someone.

Comment author: bogdanb 04 April 2012 11:33:10PM *  0 points [-]

As I said, I don’t really have any real evidence, and I believe it’d be very hard to collect. That said:

Now update on the amount of people who call somebody "a weirdo" who does not end up murdering anyone.

I’m not quite sure I understand what you mean by this. Let H=(did something horrible), S=(really suspicious), W=(just a bit creepy, weird, etc.).

I suspect that (H & S) > (H & W & !S) > (H & !W & !S) and that 1 > H/S >> H/(W & !S) >> H/(!W & !S). All fractions are low, but I’m not sure what you mean to say by that.

And add the negative halo effect, and fundamental attribution fallacy, from knowing in hindsight that the person you're talking about has recently murdered someone.

I’m pretty sure such effects are not linearly additive. Especially when there’s a conflict (friend/non-hated-family, did something bad), I don’t think you can determine the result just by logic, you have to see what people actually do.

Notice how media narratives tend to become either “I always knew he was up to no good” or “I’d never have thought he would do something like that”, but you almost never hear something in the middle. I’m even having trouble finding a concise wording for a middle case other than “meh”.

I’m sure the media has a lot to do with that, showing just the witnesses with the most “interesting” story, but I’m almost sure people also do this more-or-less automatically in their heads.

Comment author: DanArmak 05 April 2012 09:17:44AM 1 point [-]

I’m not quite sure I understand what you mean by this.

You said,

if you could interview the same people before the something horrible [...] a lot of answers would be of the “he’s kind of a weirdo” type.

What I meant was: you should also consider the amount of cases where people said the same "he's kind of a weirdo", but that person did not go on to do something horrible. And also the amount of cases where people did not say it, and yet the person did something horrible. All three are necessary to calculating the strength of the evidence "people say he's kind of a weirdo" in favor of the hypothesis "he will do something horrible".

There's a common fallacy, which you may not have committed, but which your comment as written seemed to me to evoke. Logically it's equivalent to base rate neglect. In conversation, it's often triggered like this: a nontrivial value for P(A|B) is given, but the probability P(A|~B) is not mentioned. The listener doesn't have a good estimation of P(A) or P(B), and he doesn't think to ask; instead the high value of P(A|B) makes him think B is a good predictor of A, which is a fallacy. (Here, A=be called a weirdo, B=commit horrible deed.)

I’m pretty sure such effects are not linearly additive.

I'm not saying they're linear or otherwise well-behaved, but I'm pretty sure they are all generally additive in the sense that in any given combination, if you increase any of the factors, the total also increases.

Notice how media narratives tend to become either “I always knew he was up to no good” or “I’d never have thought he would do something like that”, but you almost never hear something in the middle. I’m even having trouble finding a concise wording for a middle case other than “meh”.

It may also be that "I'd never have thought he'd do that" is the middle, the default way in which people think of anyone they have no reason to specially suspect. After all, I don't expect a random stranger on the street to suddenly commit a horrible deed; why should I expect it more of an acquaintance unless there are concrete warning signs, which would make me say "I always knew he was up to no good".

The true opposite of "I always knew..." would on this view be like Harry's reaction to Hermione confessing attempted murder: "I don't believe she did it, it's a priori so improbable there must be another explanation or very special circumstances". However, when the media has concluded someone has committed a horrible deed and is morally culpable, of course you won't hear many people saying this to the media, even if they think so privately.

Comment author: bogdanb 05 April 2012 07:15:49PM 0 points [-]

you should also consider the amount of cases where people said the same "he's kind of a weirdo", but that person did not go on to do something horrible. And also the amount of cases where people did not say it, and yet the person did something horrible. All three are necessary to calculating the strength of the evidence "people say he's kind of a weirdo" in favor of the hypothesis "he will do something horrible"

Well, yeah, I agree, but I wasn’t trying to do that. At least I don’t think I was, and if it’s an implied assumption in what I said I don’t see it.

My original comment just said that I suspect many of the “I had no suspicion” after-crime statements are false (consciously or not), and was based mostly on how I suspect people’s brains might react, not on the rates of horrible acts.

My second comment I think said the same thing your quote above does, except adding that I also suspect a certain ordering of rates. But as I said in my first comment, I don’t have the actual rates and I believe they’re hard to obtain, so it’s just a suspicion.

After all, I don't expect a random stranger on the street to suddenly commit a horrible deed; why should I expect it more of an acquaintance unless there are concrete warning signs, which would make me say "I always knew he was up to no good".

The true opposite of "I always knew..." would on this view be like Harry's reaction to Hermione confessing attempted murder: "I don't believe she did it, it's a priori so improbable there must be another explanation or very special circumstances".

That’s true. I guess there are just very few people with this kind of reasoning (à la the Wizengamot); once they heard it happened, most probably take it for granted it was so, and they have only the “knew it all the time”/“didn’t see it coming” alternatives.

After all, I don't expect a random stranger on the street to suddenly commit a horrible deed; why should I expect it more of an acquaintance unless there are concrete warning signs, which would make me say "I always knew he was up to no good".

For a random stranger you have only the base rate to go on, you’ve got no other evidence. (Though for specific strangers you might have stuff like “he looks like a mobster” or “I’m in a dangerous neighborhood”, or maybe “he’s black and wears a hoodie”, which are a bit different as signs go.)

My claim is not quite that “weird people murder more often”. Instead, I suspect that “of the people who murder, a big majority were not stable/calm enough before and did give signs before”, and many if not most of the cases of people claiming there were no signs are because they just forget or ignore those signs.

(The two sentences are different if it so happens that there are very few people that give no signs, enough so that the fraction of them who do horrible things is less than the fraction of those who did give signs. Which I believe unlikely but not quite impossible.)

Comment author: Grognor 04 April 2012 02:38:02PM 7 points [-]

This could easily be face-saving. You can't well publicly say, "You know, I thought he might have been a dangerous criminal, but I didn't bother trying to prevent any crimes."

And you're ignoring the many more cases where people expected a person to be a murderer and he wasn't.

Comment author: Eliezer_Yudkowsky 04 April 2012 06:19:48AM 7 points [-]

See also: Amanda Knox.

Comment author: khafra 04 April 2012 12:32:02PM 4 points [-]

I was pretty sure that "prior probability of a normal girl just hauling off and murdering someone in cold blood" was a Knox allusion. I wonder if Ms. Knox herself has read it.

Comment author: Alsadius 05 April 2012 05:06:48AM 2 points [-]

Yeah, this set of chapters started making a lot more sense when I realized it was a gigantic Amanda Knox allegory.

Comment author: Eliezer_Yudkowsky 05 April 2012 05:50:03AM 10 points [-]

Well, sure, but it's also an allegory for everyone sent to prison for using marijuana by politicians who somehow manage to care more about other things than about smashing the life of some nice person who never hurt anyone; and an allegory for the public response to 9/11/2001. Et cetera. If story events only allegorized one insanity at a time, the story would have to be three times as long to make the same set of points.

Comment author: Luke_A_Somers 05 April 2012 06:02:04AM 4 points [-]

Who, it may be noted, was eventually found innocent.

Comment author: Eugine_Nier 04 April 2012 06:35:29AM -1 points [-]

Well, then condition on the fact that Querril caught her and she has memories of doing it.

Comment author: Percent_Carbon 04 April 2012 06:44:12AM 4 points [-]

Quirrell didn't say he caught her. He did not claim to observe Hermione at or departing the duel. Time-travelling Dumbledore did not claim to have observed Hermione at or departing the duel.

We are meant to learn from Rita's Folly that memories are not worth trust.

Just what condition is your condition in?

Comment author: maia 04 April 2012 05:00:22AM 6 points [-]

Actually, based on the Legilimens finding all the fantasies about Draco and Snape conspiring to hurt Harry and her, I've adjusted my probability estimate that Hermione actually did it significantly upward.

Hermione has been having paranoid fantasies about her friends being harmed by Draco -> Draco attacks her, she weakens Draco -> Hermione is suddenly in a position of power over someone she views as a threat to her friends -> Hermione temporarily goes crazy and tries to eliminate Draco.

However, my probability that she did this without mind control being the deciding factor is still virtually zero.

Comment author: Alejandro1 04 April 2012 07:47:05AM *  12 points [-]

My belief is and has always been that she did it, and was not given false memories, nor Imperiused or controlled in any way beyond the Groundhog Day attack. Eliezer believes that humans are hackable (cf. AI box experiments) and this Hermione storyline is showcasing it. Hat-and-Cloak had to find the right hack by proof and error, but once he found it, it was just ordinary words and no magic which influenced Hermione to "freely" decide to murder Draco (just like the AI gatekeepers who "freely" let the AI escape).

ETA: by "proof and error" I meant "trial and error". I guess the reason for the mental typo is the Spanish equivalent "prueba y error".

Comment author: NancyLebovitz 04 April 2012 01:26:35PM 0 points [-]

What about the possibility that Draco attacked Hermione with sufficient force that the blood-cooling spell was plausibly self-defense?

Comment author: Random832 04 April 2012 01:42:09PM 6 points [-]

A "blood-cooling charm" doesn't sound like it would have had enough stopping power to be effective in self-defense.

Comment author: DanArmak 04 April 2012 03:49:31PM 3 points [-]

In that case: she knew when she awakened next morning that it should have killed him. If she had known that the night before, then after disabling Draco with the charm, she should alerted a teacher, or maybe woken him up and stunned him the usual way. If she didn't do any of that, she was knowingly leaving him to die.

Comment author: ChrisHallquist 04 April 2012 12:51:34PM 1 point [-]

This is a really interesting point.

We know for a fact that Hermione has been manipulated because we've seen the scene with Hat and Cloak. That may bias us in favor of thinking the evidence is clearer than it really is. Yet Harry knows that in this universe memories can be tampered with. The prior probability of a morally upstanding little girl trying to murder someone is much less than the prior probability of someone else trying the memory-charm plot, especially given that Hermione is friends with Harry and Harry's status as the Boy Who Lived makes him a target.

Now, I might put a probability of something like 0.2 on the possibility of Hermione casting the charm after someone messed with her mind. But if you're the plotter, it's so much easier to do a false memory charm to make her remember casting the charm, than to do all the delicate manipulation to get her to actually cast it. So more likely than not, Hermione didn't cast the charm.

Harry is making a subtle mistake here, though: he's over-confident about his ideas about the details of what was done to Hermione. For example, he thinks a false-memory charm was used to cause Hermione to start obsessing over Draco, when in fact that was based on true memories of a conversation (albeit one involving lies and some kind of shape-shifting/illusion magic). Feminist bank tellers and all that.

Comment author: qjmw 04 April 2012 09:02:34PM 2 points [-]

Anybody believing Hermione was meddled with ought to be asking who will be next. The mentioned mistake has Harry assuming Hermione didn't go undetected for six months, so he is far less alarmed than he should be.

Comment author: tadrinth 05 April 2012 01:49:17AM 1 point [-]

Neville is the most likely next target for an attack intended to separate Harry from his allies. Voldemort is probably too clever to try the same trick twice, though.

Comment author: tadrinth 05 April 2012 01:47:30AM 0 points [-]

On the last point: Hermione almost certainly was false memory charmed twice; H&C would have removed the memory of their final conversation and replaced with with something innocuous at the same time as he implanted the false memory of casting the blood-cooling charm, so as to not leave a suspicious gap. He might also have implanted false memories immediately after the groundhog day attack, either to cover up the time gap or to not have Hermione wandering around with an extremely suspicious memory in her head (If Dumbledore or Snape had seen the memory of H&C, or his less-creepy disguise, they probably would have been fairly suspicious).

Comment author: Brickman 06 April 2012 03:02:37AM 2 points [-]

Personally my problem with Harry wasn't so much that he immediately assumed there was a trick (shouldn't get a probability of 1.0, no, but certainly a basket worth piling some eggs in) but that he assumed the truth would get her off. He never once stopped and asked Dumbledore and Snape "If it was proven that she had been tricked into doing this with false memories, but still cast the spell willingly and with her own hand, would the Wizengamot still convict her?" I don't even know the answer to that question, but I'd certainly ask before I assumed it was "no".

Especially considering how draconian the law is and how one of the two most important members of the judge/jury is not only the victim's father but someone already predisposed to dislike her (for what amounts to unapologetic and on-record racial discrimination). In advance I wouldn't have been surprised to see a show trial that blatantly ignored the evidence to get a conviction, though Dumbledore's faction was a bit too vocal for me to expect that with my current knowledge.

Comment author: Eugine_Nier 06 April 2012 04:39:23AM *  4 points [-]

He never once stopped and asked Dumbledore and Snape "If it was proven that she had been tricked into doing this with false memories, but still cast the spell willingly and with her own hand, would the Wizengamot still convict her?"

Harry doesn't know about the GHD attack and so his working hypothesis is that her memory of attempting to kill Draco is false.

Comment author: Brickman 06 April 2012 02:28:05PM 3 points [-]

"Ah!" Harry said suddenly. "I get it now. The first False Memory Charm was cast on Hermione after Professor Snape yelled at her, and showed, say, Draco and Professor Snape plotting to kill her. Then last night that False Memory was removed by Obliviation, leaving behind the memories of her obsessing about Draco for no apparent reason, at the same time she and Draco were given false memories of the duel."

Since that was the last theory Harry proposed before he switched from theories to lines of attack, and nobody fully shot it down (there was an objection, but the objection was just that it'd be difficult), I have to assume that was his working theory when he left the room. And I would not automatically assume that this scenario, which we know to be very close to the correct one, would count as "innocent" in front of a wizengamot led by the angry, racist father of the victim.