= 27d567bc2d48d9f40e9ccc20ea370b15)
Morendil comments on Open Thread June 2010, Part 2 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (534)
Well, we would say "no" to that, if we had the means to abolish old age. We'd want to have our cake and eat it too.
The text stipulates that it is within the BE's technological means to abolish the suffering of the babies, so I expect that they would choose to do so, behind the Veil.
Yes, but a surprisingly large number of humans seem to react in horror when you talk about getting rid of aging.
Who will ask them? FAI have no idea, that a) baby eating is bad, b) it should generalize moral values past BE to all conscious beings.
Even if FAI will ask that question and it turns out that majority of population don't want to do inherently good thing (it is for them), then FAI must undergo controlled shutdown.
EDIT: To disambiguate. I am talking about FAI, which is implemented by BEs.
As we should not allow FAI to generalize morals past conscious beings, just to be sure, that it will not take CEV of all bacterium, so BEs should not allow their FAI to generalize past BEs.
As we should built in automatic off switch into our FAI, to stop it if its goals is inherently wrong, so should BEs.