Eliezer_Yudkowsky comments on What I Think, If Not Why - Less Wrong

25 Post author: Eliezer_Yudkowsky 11 December 2008 05:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (100)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 13 December 2008 09:11:00AM 1 point [-]

Wei, the criterion "intelligent" compresses down to a very simple notions of effective implementation abstracted away from the choice of goal. After that, the only question is how to get "intelligence", which is something you can, in principle, learn by observation (if you start with learning ability). Flaws in a notion of "intelligence" can be self-corrected if not too great; you observe that what you're doing isn't working (for your goal criterion).

Morality does not compress; it's not something you can learn just by looking at the (nonhuman) environment or by doing logic; if you want to get all the details correct, you have to look at human brains.

That's the difference.

Samantha, were you around for the metaethics sequence?