taw comments on When is further research needed? - Less Wrong

0 Post author: RichardKennaway 17 June 2011 03:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (80)

You are viewing a single comment's thread. Show more comments above.

Comment author: Perplexed 17 June 2011 07:34:13PM *  8 points [-]

As certain wise Paperclip Optimizer once said, information that someone is blackmailing you is bad.

Actually, no it isn't. What is bad for you is for the blackmailer to learn that you are aware of the blackmail.

Acquiring information is never bad, in and of itself. Allowing others to gain information can be bad for you. Speaking as an egoist, that is.

ETA: I now notice that gjm already made this point.

Comment author: taw 02 July 2011 09:27:24AM 0 points [-]

This seems incorrect. It doesn't really matter for blackmailer if you're aware of the blackmail or not, what matters is his estimate of the chance than you know.

Blackmailing is profitable if gain from successful blackmail * chance you'll know about it * chance you'll give in > cost of blackmail.

Unless you can guarantee 100% solid precommitment to not giving in to blackmail (and let's face it - friendly AI is easier than that), the more you increase the chance of knowing about it, the more blackmailing you'll face.