Vladimir_Nesov comments on The Domain of Your Utility Function - Less Wrong

24 Post author: Peter_de_Blanc 23 June 2009 04:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (94)

You are viewing a single comment's thread. Show more comments above.

Comment author: pjeby 24 June 2009 01:07:30AM 1 point [-]

It looks like you're suggesting that AIs that mirror human values must be implemented in the way humans really work

I'm saying that a system that's based on utility maximizing is likely too alien of a creature to be able to be safely understood and utilized by humans.

That's more or less the premise of FAI, is it not? Any strictly-maximizing agent is bloody dangerous to anything that isn't maximizing the same thing. What's more, humans are ill-equipped to even grok this danger, let alone handle it safely.

Comment author: Vladimir_Nesov 24 June 2009 02:55:19AM 2 points [-]

The best bridges are not humans either.