Lumifer comments on What should a friendly AI do, in this situation? - Less Wrong

8 Post author: Douglas_Reay 08 August 2014 10:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (66)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lumifer 13 August 2014 02:47:20PM 0 points [-]

If the results are communicated with perfect clarity, but the recipient is insufficiently moved by the evidence ... then the AI must manipulate us

That's an interesting "must".

Comment author: VAuroch 13 August 2014 09:04:43PM -1 points [-]

You're misquoting me.

then the AI must manipulate us to bring the emotional justification in line with the logical one.

Comment author: Lumifer 13 August 2014 09:14:32PM -1 points [-]

then the AI must manipulate us to bring the emotional justification in line with the logical one

<shrug>

That's an interesting "must".

Comment author: VAuroch 13 August 2014 09:37:09PM -1 points [-]

This is a commonly-used grammatical structure in which 'must' acts as a conditional. What's your problem?

Comment author: Lumifer 14 August 2014 12:44:34AM 0 points [-]

Conditional?

Your sentence structure is: if {condition} then {subject} MUST {verb} in order to {purpose}. Here "must" carries the meaning of necessity and lack of choice.

Comment author: VAuroch 14 August 2014 06:31:19AM -1 points [-]

No, 'must' here is acting as a logical conditional; it could be rephrased as 'if {condition} and {subject} does not {verb}, then {purpose} will not occur' without changing the denotation or even connotation. This isn't a rare structure, and is the usual interpretation of 'must' in sentences of this kind. Leaving off the {purpose} would change the dominant parsing to the imperative sense of must.

Comment author: Lumifer 14 August 2014 03:25:02PM 0 points [-]

It's curious that we parse your sentence differently. To me your original sentence unambiguously contains "the imperative sense of must" and your rephrasing is very different connotationally.

Let's try it:

"If the results are communicated with perfect clarity, but the recipient is insufficiently moved by the evidence ... and the AI does not manipulate us then the emotional justification will not be in line with the logical one."

Yep, sounds completely different to my ear and conveys a different meaning.