Cyan comments on Consciousness of simulations & uploads: a reductio - Less Wrong

1 Post author: simplicio 21 August 2010 08:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (139)
Tags: uploading consciousness zombies simulation

Comments (139)

You are viewing a single comment's thread. Show more comments above.

Comment author: Perplexed 26 August 2010 04:30:45PM 3 points [-]

perhaps the first rule should be "Do not simulate without sufficient resources to maintain that simulation indefinitely."

There have been some opinions expressed on another thread that disagree with that.

The key question is whether terminating a simulation actually does harm to the simulated entity. Some thought experiments may improve our moral intuitions here.

  • Does slowing down a simulation do harm?
  • Does halting, saving, and then restarting a simulation do harm?
  • Is harm done when we stop a simulation, restore an earlier save file, and then restart?
  • If we halt and save a simulation, then never get around to restarting it, the save disk physically deteriorates and is eventually placed in a landfill, exactly at which stage of this tragedy did the harm take place? Did the harm take place at some point in our timeline, or at a point in simulated time, or both?

I tend to agree with your invocation of xenia, but I'm not sure it applies to simulations. At what point do simulated entities become my guests? When I buy the shrink-wrap software? When I install the package? When I hit start?

I really remain unconvinced that the metaphor applies.

Comment author: Cyan 27 August 2010 03:03:38AM *  0 points [-]

Applying the notion of information-theoretic death to simulated beings results in the following answers:

  • Does slowing down a simulation do harm? If/when time for computation becomes exhausted, those beings who lost the opportunity to be simulated are harmed, relative to the counterfactual world in which the simulation was not slowed.
  • Does halting, saving, and then restarting a simulation do harm? No.
  • Is harm done when we stop a simulation, restore an earlier save file, and then restart? If the restore made the stopped simulation unrecoverable, yes.
  • If we halt and save a simulation, then never get around to restarting it, the save disk physically deteriorates and is eventually placed in a landfill, exactly at which stage of this tragedy did the harm take place? When the information became unrecoverable. Did the harm take place at some point in our timeline, or at a point in simulated time, or both? Both.
Comment author: NancyLebovitz 27 August 2010 02:03:40PM 1 point [-]

Does slowing down a simulation do harm? If/when time for computation becomes exhausted, those beings who lost the opportunity to be simulated are harmed, relative to the counterfactual world in which the simulation was not slowed.

Slowing down a simulation also does harm if there are interactions which the simulation would prefer to maintain which are made more difficult or impossible.

The same would apply to halting a simulation.

Comment author: Perplexed 27 August 2010 03:25:40AM *  1 point [-]

Request for clarification:

Is harm done when we stop a simulation, restore an earlier save file, and then restart? If the restore made the stopped simulation unrecoverable, yes.

Do I understand this properly to say that if the stopped simulation had been derived from the save file state using non-deterministic or control-console inputs, inputs that are not duplicated in the restarted simulation, then harm is done?

Hmmm. I am imagining a programmer busy typing messages to his simulated "creations":

Thou shalt commit adultery. <return>

Looks at what was entered ...

<Undo>

Thinks about what just happened. ... "Aw Sh.t!"

Comment author: Cyan 27 August 2010 03:36:01AM *  0 points [-]

Do I understand this properly to say that if the stopped simulation had been derived from the save file state using non-deterministic or control-console inputs, inputs that are not duplicated in the restarted simulation, then harm is done?

As I understand it, yes. But the harm might not be as bad as what we currently think of as death, depending on how far back the restore went. Backing one's self up is a relatively common trope in a certain brand of Singularity fic (e.g. Glasshouse).

(I needed three parentheses in a row just now: the first one, escaped, for the Wikipedia article title, the second one to close the link, and the third one to appear as text.)

Powered by Reddit Powered by Reddit