You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Viliam_Bur comments on Is a paperclipper better than nothing? - Less Wrong Discussion

6 Post author: DataPacRat 24 May 2013 07:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (116)

You are viewing a single comment's thread. Show more comments above.

Comment author: Viliam_Bur 26 May 2013 01:05:25PM 0 points [-]

Well, the paperclip maximizer may be imperfect in some aspect.

Maybe it didn't research cryptography, because at given time making more paperclips seemed like a better choice than researching cryptography. (All intelligent agents may at some moment face a choice between developing an abstract theory with uncertain possible future gains vs pursuing their goals more directly; and they may make a wrong choice.)

Comment author: gwern 26 May 2013 05:13:30PM 3 points [-]

The crypto here is a bit of a red herring; you want that in adversarial contexts, but a paperclipper may not necessarily optimize much for adversaries (the universe looks very empty). However, a lot of agents are going to research error-checking and correction because you simply can't build very advanced computing hardware without ECC somewhere in it - a good chunk of every hard drive is devoted to ECC for each sector and discs like DVD/BDs have a lot of ECC built in as well. And historically, ECC either predates the most primitive general-purpose digital computers (scribal textual checks) or closely accompanies them (eg. Shannon's theorem), and of course we have a lot of natural examples (the redundancy in how DNA codons code for amino acids turns out to be highly optimized in an ECC sense).

So, it seems pretty probable that ECC is a convergent instrumental technique.

Comment author: lukstafi 30 May 2013 06:10:33PM *  2 points [-]