MichaelVassar comments on Hacking the CEV for Fun and Profit - Less Wrong

52 Post author: Wei_Dai 03 June 2010 08:30PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (194)

You are viewing a single comment's thread. Show more comments above.

Comment author: Alexandros 04 June 2010 01:01:36PM *  10 points [-]

Thinking about this a bit more, and assuming CEV operates on the humans that exist at the time of its application: Why would CEV operate on humans that do exist, and not on humans that could exist? It seems this is what Dr. Evil is taking advantage of, by densely populating identity-space around him and crowding out the rest of humanity. But this could occur for many other reasons: Certain cultures encouraging high birth rates, certain technologies or memes being popular at the time of CEV-activation that affect the wiring of the human brain, or certain historical turns that shape the direction of mankind. A more imaginative scenario: what if another scientist, who knows nothing about FAI and CEV, finds it useful to address a problem by copying himself into trillions of branches, each examining a certain hypothesis, and all the branches are (merged/discarded) when the answer is found. Let's further say that CEV t-zero occurs when the scientist is deep in a problem-solving cycle. Would the FAI take each branch as a separate human/vote? This scenario involves no intent to defraud the system. It also is not manipulation of a proxy, as there is a real definitional problem here whose answer is not easily apparent to a human. Applying CEV to all potential humans that could have existed in identity-space would deal with this, but pushes CEV further and further into uncomputable territory.

Comment author: MichaelVassar 08 June 2010 05:13:43AM 1 point [-]

Yep. This is also, arguably, why cryonics doesn't work.

Comment author: JoshuaZ 08 June 2010 05:17:30AM 3 points [-]

I don't follow your logic. What is the connection to cryonics?

Comment author: Vladimir_Nesov 08 June 2010 08:19:04AM *  3 points [-]

If it's just to use volitions of "people in general" as opposed to "people alive now", it might also turn out to be correct to ignore "people present now" altogether, including cryonauts, in constructing the future. So, this is not just an argument for cryonics not working, but also for people alive at the time in no sense contributing/surviving (in other words, everyone survives to an equal degree, irrespective of whether they opted in on cryonics).

Comment author: MichaelVassar 09 June 2010 04:20:13PM 0 points [-]

Conditional on the development of FAI. The fact that humanity arises and produces FAI capable of timeless trade at a certain rate would cause the human-volition regarding behaviors of all trading super-intelligences and of all human friendly AIs in the multiverse.

Comment author: Vladimir_Nesov 09 June 2010 07:25:09PM 2 points [-]

The fact that humanity arises and produces FAI capable of timeless trade at a certain rate would cause the human-volition regarding behaviors of all trading super-intelligences and of all human friendly AIs in the multiverse.

Can't parse this statement. The fact that a FAI could appear causes "human-volition" about all AGIs? What's human-volition, what does it mean to cause human-volition, how is the fact that FAI could appear relevant?

Comment author: Alexandros 08 June 2010 10:14:01AM 2 points [-]

I'm stumped. Can you elaborate?

Comment author: NancyLebovitz 08 June 2010 09:26:09AM 1 point [-]

Do you mean that cryonics doesn't work, or that cryonics isn't worth doing?