= 27d567bc2d48d9f40e9ccc20ea370b15)
wedrifid comments on Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (244)
My argument isn't about the machine not sharing goals with the humans - it's about whether the humans can shut the machine down if they want to.
I argue that it is not rocket science to build a machine with a stop button - or one that shuts down at a specified time.
Such a machine would not want to fool the research team - in order to avoid shutting itself down on request. Rather, it would do everything in its power to make sure that the shut-down happened on schedule.
Many of the fears here about machine intelligence run amok are about a runaway machine that disobeys its creators. However, the creators built it. They are in an excellent position to install large red stop buttons and other kill switches to prevent such outcomes.
Given 30 seconds thought I can come up with ways to ensure that the universe is altered in the direction of my goals in the long term even if I happen to cease existing at a known time in the future. I expect an intelligence that is more advanced than I to be able to work out a way to substantially modify the future despite a 'red button' deadline. The task of making the AI respect the 'true spirit of a planned shutdown' shares many difficulties of the FAI problem itself.
You might say it's an FAI-complete problem, in the same way "building a transhuman AI you can interact with and keep boxed" is.
You think building a machine that can be stopped is the same level of difficulty as building a machine that reflects the desires of one or more humans while it is left on?
I beg to differ - stopping on schedule or on demand is one of the simplest possible problems for a machine - while doing what humans want you to do while you are switched on is much trickier.
Only the former problem needs to be solved to eliminate the spectre of a runaway superintelligence that fills the universe with its idea of utility against the wishes of its creator.
Beware simple seeming wishes.
Exactly, I like the terminology.
Well, I think I went into most of this already in my "stopping superintelligence" essay.
Stopping is one of the simplest possible desires - and you have a better chance of being able to program that in than practically anything else.
I gave several proposals to deal with the possible issues associated with stopping at an unknown point resulting in plans beyond that point still being executed by minions or sub-contractors - including scheduling shutdowns in advance, ensuring a period of quiescence before the shutdown - and not running for extended periods of time.
It does seem to be a safety precaution that could reduce the consequences of some possible flaws in an AI design.