= 27d567bc2d48d9f40e9ccc20ea370b15)
eli_sennesh comments on UFAI cannot be the Great Filter - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (90)
A UFAI that doesn't go around eating stars to make paper-clips is probably already someone's attempted FAI. Bringing arbitrarily large sums of mass-energy and negentropy under one's control is a Basic AI Drive, so you have to program the utility function to actually penalize it.
Only if the AI has goals that both require additional energy, and don't have a small, bounded success condition.
For example, if an UFAI for humans has a goal that requires humans to be there, but is not allowed to create/lead to the creation of more, then if all humans are already dead it won't do anything.