Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

nshepperd comments on Failed Utopia #4-2 - Less Wrong

52 Post author: Eliezer_Yudkowsky 21 January 2009 11:04AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (248)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: nshepperd 30 May 2013 02:08:31PM 3 points [-]

If your genie is using your vocal emissions as information toward the deduction of your extrapolated volition, then I'd say your situation is good.

Your problems start if it works more by attempting to extract a predicate from your sentence by matching vocal signals against known syntax and dictionaries, and output an action that maximises the probability of that predicate being true with respect to reality.

To put it simply, I think that "understanding what we mean" is really a complicated notion that involves knowing what constitutes true desires (as opposed to, say, akrasia), and of course having a goal system that actually attempts to realize those desires.