= 27d567bc2d48d9f40e9ccc20ea370b15)
arbimote comments on Open Thread: February 2010 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (738)
You could observe how it acts in its simulated world, and hope it would act in a similar way if released into our world. ETA: Also, see my reply for possible single-bit tests.
Sounds like a rather drastic context change, and a rather forlorn hope if the AI figures out that it's being tested.
I merely wanted to point out to Kaj that some "meaningful testing" could be done, even if the simulated world was drastically different from ours. I suspect that some core properties of intelligence would be the same regardless of what sort of world it existed in - so we are not crippling the AI by putting it in a world removed from our own.
Perhaps "if released into our world" wasn't the best choice of words... more likely, you would want to use the simulated AI as an empirical test of some design ideas, which could then be used in a separate AI being carefully designed to be friendly to our world.
"if the AI figures out that it's being tested"
That is a weird point, Eliezer.
An AI will have a certain goal to fulfill and it will fulfill that goal in the univese in which it finds itself. Why would it keep its cards hidden only to unleash them when replicated in the "real world"? What if the real world turns out to be another simulation? There's no end to this, right?
Are you extending Steve Omohundro's point about :every AI will want to survive" to "every AI will want to survive in the least simulated world that it can crack into?"
The basement is the biggest, and matters more for goals that benefit strongly from more resources/security.
Carl,
Correct me if i misunderstood the implications of what you are saying.
Every AI that has a goal that benefits strongly from more resources and security will seek to crack into the basement. Lets call this AI, RO (resource oriented) pursuing goal G in simulation S1.
S1 is simulated in S2 and so on till Sn is basement, where value of n is unknown.
Implying, that as soon as RO understands the concept of simulation, it will seek to crack into the basement.
As long as RO has no idea about what are the real values of the simulators, RO cannot expand into S1 because whatever it does in S1 will be noticed in S2 and so on.
Sounds a bit like Pascal's mugging to me. Need to think more about this.
Why would RO seek to crack the basement immediately rather than at the best time according to its prior, evidence, and calculations?
Carl, I meant that as soon as RO understands the concept of a simulation, it will want to crack into the basement. It will seek to crack into the basement only when it understands the way out properly which may not be possible without an understanding of the simulators.
But the main point remains, as soon as RO understands what a simulation is, and it could be living in one and G can be pursued better when it manifests in S2 than in S1, then it will develop an extremely strong sub-goal to crack S1 to go to S2, which might mean that G may not be manifested for a long long time.
So, even a paperclipper may not act like a paperclipper in this universe if it is
I like that turn of phrase.