Will_Pearson comments on Ethics Notes - Less Wrong

12 Post author: Eliezer_Yudkowsky 21 October 2008 09:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Will_Pearson 22 October 2008 01:35:45PM 0 points [-]

I shall try one last time to get my point across.

Forgetting has all the bad points of consciously lying. People may no longer trust you to perform tasks. A senile catholic priest that occasionally forgot that he wasn't supposed to tell anyone what happens during confession, wouldn't be trusted. In this case the system has to make a trade off between forgetting how to get eat and forgetting the oath of secrecy

Approximating something can cause you to get bitten by black swans. Because while you think it might not hurt to approximate a Real number with a floating point number in this situation, it might be end up being crucial.

Not being trusted and being bitten by black swans are the reasons given not to lie to yourself, and to build an AI that shuts down if it starts lying to itself.

Should a system shut itself down if it thinks it is a good idea to forget or approximate something?