Multiheaded comments on Muehlhauser-Wang Dialogue - Less Wrong

24 Post author: lukeprog 22 April 2012 10:40PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (284)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 23 April 2012 06:05:46AM 13 points [-]

If I understand correctly, his theses are that the normal research path will produce safe AI because it won't blow up out of our control or generally behave like a Yudkowsky/Bostrom-style AI. Also that trying to prove friendlyness in advance is futile (and yet AI is still a good idea) because it will have to have "adaptive" goals, which for some reason has to extend to terminal goals.

It is my belief that an AGI will necessarily be adaptive, which implies that the goals it actively pursues constantly change as a function of its experience, and are not fully restricted by its initial (given) goals.

He needs to taboo "adapive", read and understand Bostroms AI-behaviour stuff, and comprehend the Superpowerful-Optimizer view, and then explain exactly why it is that an AI cannot have a fixed goal architecture.

If AI's can't have a fixed goal architecture, Wang needs to show that AI's with unpredictable goals are somehow safe, or start speaking out against AI.


So what sort of inconvienient word would it take for Wang's major conclusions to be correct?

I don't know, I'm not good enough at this steel-man thing, and my wife is sending me to bed.

Comment author: Multiheaded 23 April 2012 08:03:12AM *  2 points [-]

He needs to taboo "adapive", read and understand Bostroms AI-behaviour stuff, and comprehend the Superpowerful-Optimizer view, and then explain exactly why it is that an AI cannot have a fixed goal architecture.

If AI's can't have a fixed goal architecture, Wang needs to show that AI's with unpredictable goals are somehow safe, or start speaking out against AI.

Damn right! These were my first thoughts as well. I know next to nothing about AI, but seriously, this is ordinary logic.