jsalvatier comments on What are you working on? - Less Wrong

9 Post author: jsalvatier 06 October 2011 04:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (97)

You are viewing a single comment's thread. Show more comments above.

Comment author: jsalvatier 06 October 2011 05:37:51PM 10 points [-]

I suspect some people here will have a negative reaction to your comment. Your approach comes off as not very serious, your last paragraph sounds like reasoning from conclusion to argument, and your mention of friendliness seems like an afterthought.

Comment author: anotheruser 07 October 2011 09:42:04AM 0 points [-]

I assure you that I have thought a lot about freindliness in AI. I just don't think that it is reasonable or indeed possible to make the AI have a moral system from the very start. You can't define morality well if the AI doesn't have a good understanding of the world already. Of course it shouldn't be taught too late under any circumstances but I actually think that the risk will be higher if you try to hardcode friendliness into the AI at the very beginning, which will necessarily be flawed because you have so little to use in your definition, and then work under the assumption that the AI is friendly already and will stay so, than if you only implement friendliness later once it actually understands the concepts involved. The difference would be like between the moral understandings of a child and an adult philosopher.