Draft of Muehlhauser & Helm, 'The Singularity and Machine Ethics'

lukeprog

Louie and I are sharing a draft of our chapter submission to The Singularity Hypothesis for feedback:

Thanks in advance.

Also, thanks to Kevin for suggesting in February that I submit an abstract to the editors. Seems like a lifetime ago, now.

Edit: As of 3/31/2012, the link above now points to a preprint.

Louie and I are sharing a draft of our chapter submission to The Singularity Hypothesis for feedback:

The Singularity and Machine Ethics

Thanks in advance.

Also, thanks to Kevin for suggesting in February that I submit an abstract to the editors. Seems like a lifetime ago, now.

Edit: As of 3/31/2012, the link above now points to a preprint.

If a machine superoptimizer’s goal system is programmed to maximize pleasure, it might not tile the local universe with tiny digital minds running continuous loops of a single, maximally pleasurable experience, but we think it would do something undesirable like that.

Step 1 - for many minds without too short a horzion - is to conquer the galaxy to make sure there are no aliens arount that might threaten their entire value system. Tiling the local universe with tiny happy digital minds could easily turn out to be a recipe for long-term disaster - resulting in a universe with happiness levels being dictated by others.

11

Draft of Muehlhauser & Helm, 'The Singularity and Machine Ethics'

11

11

11

Draft of Muehlhauser & Helm, 'The Singularity and Machine Ethics'

11

11