FAWS comments on Open Thread: April 2010, Part 2 - Less Wrong

3 Post author: Unnamed 08 April 2010 03:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (194)

You are viewing a single comment's thread. Show more comments above.

Comment author: FAWS 08 April 2010 02:18:32PM *  0 points [-]

I don't think the idea of a conditional terminal value is very useful. If information is a terminal value for me I'd want to know what the unfriendly superintelligence would say, but unless it's the only terminal value and I don't think the result would have any influence on other information gathering there would be other considerations speaking against learning that particular piece of information and probably outweighing it. No need to make any exceptions for mind damaging truths because to the extent mind damage is a bad thing according to my terminal values they will already be accounted for anyway.