jsteinhardt comments on Muehlhauser-Wang Dialogue - Less Wrong

24 Post author: lukeprog 22 April 2012 10:40PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (284)

You are viewing a single comment's thread. Show more comments above.

Comment author: jsteinhardt 24 April 2012 12:13:45AM 3 points [-]

If such a poorly designed system as a human has the ability to change its goals in response to stimuli, and we find this to be a desirable property, then surely a carefully designed AI will have the same property, unless we have an even better property to replace it with? The argument, "humans are bad, AIs are good, therefore AIs will do something bad" seems unlikely at face value.

(Note that I would like something that more reliably acquires desirable goals than humans, so still think FAI research is worthwhile, but I would prefer that only the strongest arguments be presented for it, especially given the base rate of objection to FAI-style arguments.)

Comment author: John_Maxwell_IV 24 April 2012 04:36:43AM 0 points [-]

Why is changing one's goals in response to stimuli a valuable property? A priori, it doesn't seem valuable or harmful.

This wasn't meant to be an argument either way for FAI research, just a thought on something Pei said.