Kawoomba comments on Failed Utopia #4-2 - Less Wrong

52 Post author: Eliezer_Yudkowsky 21 January 2009 11:04AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (247)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 30 May 2013 08:42:21AM 4 points [-]

If a genie cares enough about your request to interpret and respond to its naive denotation, it also cares enough to interpret your request's obvious connotations.

That doesn't follow. There just isn't any reason that the former implies the latter. Either kind of caring is possible but they are not the same thing (and the second is likely more complex than the first).

Your proposed interpretation only makes sense if the genie is a rules-lawyer

This much is true. (Or at least it must be something that follows rules.)

with at-least-instrumentally-oppositional interests/incentives

This isn't required. It need no oppositional interests/incentives at all beyond, after they are given a request, the desire to honour it. This isn't a genie trying to thwart someone in order to achieve some other goal. It is just the genie trying to the intent in order to for some other purpose. It is a genie only caring about the request and some jackass asking for something they don't want. (Rather than 'oppositional' it could be called 'obedient', where it turns out that isn't what is desired.)

in which case one wonders where those oppositional interests/incentives came from.

Presumably it got it's wish granting motives from whoever created it or otherwise constructed the notion of the wish granter genie.

Comment author: Kawoomba 30 May 2013 08:48:31AM 0 points [-]

Presumably it got it's wish granting motives from whoever created it or otherwise constructed the notion of the wish granter genie.

Why would there be some creating agency involved any more than we need a "whoever" to explain where human characteristics come from?