Eliezer_Yudkowsky comments on Cryptographic Boxes for Unfriendly AI - Less Wrong

24 Post author: paulfchristiano 18 December 2010 08:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (155)
Tags: ai

Comments (155)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 18 December 2010 03:16:38PM 6 points [-]

The probability I assign to achieving a capability state where it is (1) possible to prove a mind Friendly even if it has been constructed by a hostile superintelligence, (2) possible to build a hostile superintelligence, and (3) not possible to build a Friendly AI directly, is very low.

A general theory of quarantines would nevertheless be useful.

Comment author: Eliezer_Yudkowsky 18 December 2010 04:11:09PM 2 points [-]

For what?

Comment author: PeterS 18 December 2010 08:48:34PM 4 points [-]

The OP framed the scenario in terms of directing the AI to design a FAI, but the technique is more general. It's possibly safe for all problems with a verifiable solution.

Comment author: wedrifid 20 December 2010 05:59:32AM *  2 points [-]

For what?

People I don't trust but don't want to kill (or modify to cripple). A non-compliant transhuman with self modification ability may not be able to out-compete an FAI but if it is not quarantined it could force the FAI to burn resources to maintain dominance.

But it is something we can let the FAI build for us.

Comment deleted 20 December 2010 10:22:57AM [-]
Comment author: wedrifid 20 December 2010 11:14:24AM 0 points [-]

Shrug. For the purposes here they could be called froogles for all I care. The quarantine could occur in either stage depending on the preferences being implemented.

Powered by Reddit Powered by Reddit