Tynam comments on Sorting Pebbles Into Correct Heaps - Less Wrong

75 Post author: Eliezer_Yudkowsky 10 August 2008 01:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (99)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Tynam 08 March 2013 10:00:50AM 8 points [-]

Only until you build a self-modifying super-intelligent bull. Because the first thing it will do is become smart enough to persuade you to give the ring to someone else, who it's calculated it can con into taking the ring off.

Human minds are really badly adapted for defence against con artists operating on the same level; how on earth would we defend ourselves against an exponentially smarter one?