thomblake comments on Harry Potter and the Methods of Rationality discussion thread, part 5 - Less Wrong

6 Post author: NihilCredo 02 November 2010 06:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (648)

You are viewing a single comment's thread. Show more comments above.

Comment author: thomblake 08 November 2010 01:23:57PM 4 points [-]

I mean, obviously he avoids the Vast majority of unFriendly design space, simply by virtue of being human. He isn't going to tile the galaxy with paperclips or anything like that.

We don't know much about how stable human values are under recursive self-modification. It's entirely possible (albeit seemingly unlikely) that humans even tend towards tiling the galaxy with paperclips in particular.

Comment author: DanArmak 08 November 2010 02:38:46PM 6 points [-]

It's entirely possible (albeit seemingly unlikely) that humans even tend towards tiling the galaxy with paperclips in particular.

Compared to the Vast space of minds in general, they certainly do. Few minds in that Vast space have heard of the concept of a paperclip, after all.

Comment author: shokwave 08 November 2010 01:48:44PM 1 point [-]

Indeed, it seems likely. Many humans have the concept that 'locked' values are better than 'wishy-washy' ones; few have the concept of local maximums and even fewer the understanding of complex, changing human value systems. Thus a priori we should expect there is some bias or leaning in that direction, which would presumably have a chance of affecting one human in particular. This chance is greater than that of an AI's, who chooses at random.

Harry is aware of these ideas, but he often catches himself in errors. When it comes to self-modification there are no opportunities to catch your errors; you are stuck with them and will never even realise there are any.

I wonder if Quirrell realises Harry desires to be an actual god, and not just Supreme Emperor of the magical world.

Comment author: MartinB 08 November 2010 02:36:51PM 1 point [-]

Hopefully Harry is bright enough not to test invasive intelligence improvement on himself.

Comment author: shokwave 08 November 2010 02:39:57PM 7 points [-]

Ugh, he's exactly bright enough to do just that, complete with justification that he can't trust anyone else to both be safe (Quirell, Dumbledore, Draco all too dangerous) and effective (Hermione wouldn't exploit enough).

Comment author: AdShea 08 November 2010 10:48:09PM 4 points [-]

He could time-turn himself to allow for self-monitoring of the experiment.