John_Maxwell_IV comments on [draft] Concepts are Difficult, and Unfriendliness is the Default: A Scary Idea Summary - Less Wrong

7 Post author: Kaj_Sotala 31 March 2012 10:07AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (39)

You are viewing a single comment's thread. Show more comments above.

Comment author: JamesAndrix 01 April 2012 05:02:32PM 0 points [-]

That would make (human[s] + predictor) in to an optimization process that was powerful beyond the human[s]'s ability to steer. You might see a nice looking prediction, but you won't understand the value of the details, or the value of the means used to achieve it. (Which would be called trade-offs in a goal directed mind, but nothing weighs them here.)

It also won't be reliable to look for models in which you are predicted to not hit the Emergency Regret Button As that may just find models in which your regret evaluator is modified.

Comment author: John_Maxwell_IV 03 April 2012 12:06:29AM 0 points [-]

Is a human equipped with Google an optimization process powerful beyond the human's ability to steer?

Comment author: JamesAndrix 04 April 2012 03:06:27AM 0 points [-]

Tell me from China.