Vladimir_Nesov comments on The Domain of Your Utility Function - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (94)
I'm saying that a system that's based on utility maximizing is likely too alien of a creature to be able to be safely understood and utilized by humans.
That's more or less the premise of FAI, is it not? Any strictly-maximizing agent is bloody dangerous to anything that isn't maximizing the same thing. What's more, humans are ill-equipped to even grok this danger, let alone handle it safely.
The best bridges are not humans either.