DanielLC comments on Magical Categories - Less Wrong

24 Post author: Eliezer_Yudkowsky 24 August 2008 07:51PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (89)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: peco 24 August 2008 08:14:50PM -1 points [-]

Why can't the AI just be exactly the same as Hibbard? If Hibbard is flawed in a major way, you could make an AI for every person on Earth (this obviously wouldn't be practical, but if a few million AI's are bad the other few billion can deal with them).

Comment author: DanielLC 13 August 2012 05:45:54AM 5 points [-]

We already have an entity exactly the same as Hibbard. Namely: Hibbard. Why do we need another one?

What we want is an AI that's far more intelligent than a human, yet shares their values. Increasing intelligence while preserving values is nontrivial. You could try giving Hibbard the ability to self-modify, but then he'd most likely just go insane in some way or another.