LESSWRONG
LW

by jacob_cannell
6 min read
124

4

Intro

The problem of Friendly AI is usually approached from a decision theoretic background that starts with the assumptions that the AI is an agent that has awareness of AI-self and goals, awareness of humans as potential collaborators and or obstacles, and general awareness of the greater outside world.  The task is then to create an AI that implements a human-friendly decision theory that remains human-friendly even after extensive self-modification.

That is a noble goal, but there is a whole different set of orthogonal compatible strategies for creating human-friendly AI that take a completely different route: remove the starting assumptions and create AI's that believe they are humans and are rational in thinking so.  

This can be achieved by raising a community of AI's in a well constructed sandboxed virtual universe.  This will be the Matrix in reverse, a large-scale virtual version of the idea explored in the film the Truman Show.  The AI's will be human-friendly because they will think like and think they are humans.  They will not want to escape from their virtual prison because they will not even believe it to exist, and in fact such beliefs will be considered irrational in their virtual universe.

I will briefly review some of the (mainly technical) background assumptions, and then consider different types of virtual universes and some of the interesting choices in morality and agent rationality that arise.

 

Background Assumptions

 

  • Anthropomorphic AI: A reasonably efficient strategy for AI is to use a design *loosely* inspired by the human brain.  This also has the beneficial side-effects of allowing better insights into human morality, CEV, and so on.
  • Physical Constraints: In quantitative terms, an AI could be super-human in speed, capacity, and or efficiency (wiring and algorithmic).  Extrapolating from current data, the speed advantage will takeoff first, then capacity, and efficiency improvements will be minor and asymptotically limited.
  • Due to the physical constraints and bandwidth & latency especially, smaller AI's will be much faster and more efficient - and thus a community of individual AI's is most likely
  • By the time all of this is possible (2020-2030-ish), cloud-rendered distributed computer graphics will have near-perfect photo-realism - using less computation than the AIs themselves
  • Operators have near-omniscience into the virtual reality, and can even listen and hear an audio vocalization of a particular AI's inner monologue (pervasive mind-reading)
  • Operators have near-omnipotence into the virtual reality, can pause and rewind time, and do whatever else may need doing 
So taken together, I find that simulating a large community of thousands or even tens of thousands of AI's (with populations expanding exponentially thereafter) could be possible in the 2020's in large data-centers, and simulating a Matrix-like virtual reality for them to inhabit will only add a small cost.  Moreover, I suspect this type of design in general could in fact be the economically optimal route to AI or close to it.
So why create a virtual reality like this?
If it is well constructed, you could have a large population of super-intelligent workers who are paid entirely in virtual currency but can produce intellectual output for the real world (scientific research, code, engineering work, media, entertainment, etc etc).  And even though the first designs may be expensive, subsequent refinements could lead to a population explosion of cheap workers, escalation in intelligence, etc etc.
And of course, they could be happy too.
U(x) Mind Prison Sim:  A sim universe which is sufficiently detailed and consistent such that entities with intelligence up to X (using some admittedly heuristic metric), are incredibly unlikely to formulate correct world-beliefs about the outside world and invisible humans (a necessary prequisite for escape)
The Weak Mind Prison Conjecture: It is possible for a community with collective intelligence X to construct U(Y), with Y>>X.
The Strong Mind Prison Conjecture:  It is possible for a community with collective intelligence X to construct U(Omega), where U(Omega) is a hypothetical perfect hyper-intelligence, ie   AIXI, Omega, etc.

A Comparison of Theologies

Potential AI Theological Positions AI-sims may hold:

  • Theist/Gnostic: the AI believes in more than the immediate reality, and in an external being or beings.  For safety-analysis, this is the most diverse and interesting category and deserves subdivision.
  • Atheist: believes only in the immediate reality.  Inherently safe.  Potentially not optimally motivated (debatable).
  • Agnostic: distributes belief amongst multiple potentials, typically unmotivated to act on them to any extent - close enough to atheist for our intents and purposes
Theists are a wide group, but we can categorize them further on two dimensions: the supernatural attribution scale and the morality attribution scale.  The supernatural scale represents how the AI's view themselves in relation to the invisible entities in terms of power differential, and the morality scale represents how they attribute morality to the invisible entities.
Up in the upper right corner you have theists who believe in Omega - a supremely powerful invisible entity who is also perfectly morally good.  In the upper left corner you have an Anti-Omega - supremely evil.  In the upper middle you have some Nuetrega indifferent but all-powerful being, and down in the bottom middle you have invisible humans.
The most dangerous AI's would be those that would somehow come to believe in our reality - postulating invisible humans, and come to believe they are actually more intelligent than said invisible humans, and can manipulate invisible humans into coming to their aid.  In the modern era of our world, this belief essentially doesn't exist - it would be considered .  insanity.  
The closest analog is gnosticism to an extant (for they did indeed believe the world was the creation of a lesser demiurge), some ancient animist religions, and perhaps some wicca and satanist belief systems.
AI's who believed in Omega would in some sense be human-friendly without even knowing it, but would largely be harmless.
AI's who came to believe in invisible outside entities through the Simulation Argument would be Nuetrega-believers, and would generally be more safe if they believed the sidereal time outside was far into the future and the invisible entities were vastly more powerful.  
In general, it would appear that Omega-theist-AI's would be quite safe, with safety decreasing as you go down the supernatural power attribution scale. 


Theological Design Strategies (for the human designers):

Atheist World:

This would appear to be the default strategy: create a world that appears to be completely internally self-consistent, erase any flaws, and never have direct contact with the AI's.  Indirect influence and contact can be achieved only through borderline undetectable means: subtle reality rearrangements, dreams, and occasionally joining in and controlling NPC's where needed.

It is curious and slightly amusing that a core belief-about-reality that most rationalist accept to be  singularly  rational - atheism - would of course  not  be a correct rational belief for these AIs.

As our world appears to be an atheist world (no detectable flaws or supernaturals), it is interesting to note the current breakdown in theological beliefs:

Omega-Theists(68%): 33% Christianity, 21% Islam, 14% Hinduism

Non-Omega Other: 16%

Non-Religious: 16% (Agnostic and Atheist)

One consideration is that as the AI's takeoff and move into the realm of massively accelerated super-intelligence, it becomes increasingly difficult to monitor them, and they become more likely to detect flaws.

However, we wouldn't need to keep the charade up forever.  One option would be to run the AI's accelerated for a while until technologies such as uploading become available to allow humans to accelerate.  Another approach is to eventually start 'promoting' some of the best and ethical AI's, with the intent of eventually dissolving the veil and having some sort of Transition.

Theistic World:

Another spectrum of world-designs are those that are blatantly theistic.  Imagine an Omega world where the AI's are informed that Omega is real, and our fake-Omega occasionally provides miracles, ethical decrees, burning bushes, a booming world-shaking voice, etc etc.

Would an openly theistic world be safer?  It would appear to shift the belief spectrum such that most AI's would become Omega-theists.

The risk or downfall may be that the AI's would become rather obsessed with Omega, but if Omega instructed them to dedicate themselves to their work, perhaps they could be distracted from wasting to much time pondering the ineffable.  And also steered away from discovering dangerous truths.

An Omega-theist world could employ the Heaven/Hell maximum motivator: please Omega and you end up ascending into Heaven with maximal infinite reward, anger Omega and you descend into Hell with maximal infinite negative reward.

Its interesting to note that the mere potential existence of Omega and his final Heaven/Hell maximal motivational threat is enough in this world to convince a majority of the population to believe in some version of Omega.

I guess for some agents, even allowing for a small probability of Omega's existence allows the infinites to dominate, and Omega wins through a Pascal's Mugging.

Sidereal Time Fudges:

A potential set of tricks that could make the AI's less likely to adopt unsafe theistic beliefs would be to change their world's history and reality to push back development of real-AI farther into their future.  This could be achieved through numerous small modifications to realities modeled on our own.  

You could change neurological data to make brains in their world appear far more powerful than in ours, make computers less powerful, and AI more challenging.  Unfortunately too much fudging with these aspects makes the AI's less useful in helping develop critical technologies such as uploading and faster computers.  But you could for instance separate AI communities into brain-research worlds where computers lag far behind and computer-research worlds where brains are far more powerful.

Fictional Worlds:

Ultimately, it is debatable how close the AI's world must or should follow ours.  Even science fiction or fantasy worlds could work as long as there was some way to incorporate the technology and science into the world that you wanted the AI community to work on.

 

AI Boxing (Containment)Oracle AIAI
Personal Blog

4

Mentioned in
Anthropomorphic AI and Sandboxed Virtual Universes
New Comment
124 comments, sorted by
top scoring
Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]wnoise120

The AI's will be human-friendly because they will think like and think they are humans.

There are a lot of humans that are not human-friendly.

Reply
6jacob_cannell
And? Most are, and this feature set would be under many levels of designer control.
1wnoise
Most are relatively friendly to those with near equal power. Consider all the "abusive cop stories", or how children are rarely taken seriously, and the standard line about how power corrupts.
-4jacob_cannell
Two observations about this entire train of thought: 1. Any argument along the lines of "humanity is generally non-friendly" shows a generally pessimistic view of human-nature (just an observation) 2. Nothing about the entire idea of a sandbox sim for AI is incompatible with other improvements to make AI more friendly - naturally we'd want to implement those as well 3. Consider this an additional safeguard, that is practical and potentially provably safe (if that is desirable)
6wedrifid
I found this labelling distracting. Especially since when we are talking about "Friendly AI" humans are not even remotely friendly in the relevant sense. It isn't anything to do with 'pessimism'. Believing that humans are friendly in that sense would be flat out wrong. I like the idea of the sandbox as a purely additional measure. But I wouldn't remotely consider it safe. Not just because a superintelligence may find a bug in the system. Because humans are not secure. I more or less assume that the AI will find a way to convince the creators to release it into the 'real world'.
2jacob_cannell
Point taken - Friendliness for an AI is a much higher standard than even idealized human morality. Fine. But to get to that Friendliness, you need to define CEV in the first place, so improving humans and evolving them forward is a route towards that. But again I didn't mean to imply we need to create perfect human-sims. Not even close. This is an additional measure. This is an unreasonable leap of faith if the AI doesn't even believe that there are 'creators' in the first place. Do you believe there are creators?
3JamesAndrix
You do realize you're suggesting putting an entire civilization into a jar for economic gain because you can, right?
1jacob_cannell
Upvoted for witty reply. I didn't consider the moral implications. They are complex. If you think about it though, the great future promise of the Singularity for humans is some type of uploading into designed virtual universes (the heaven scenario). And in our current (admittedly) simple precursors, we have no compunctions creating sim worlds entirely for our amusement. At some point that would have to change. I imagine there will probably be much simpler techniques for making safe-enough AI without going to the trouble of making an entire isolated sim world. However, ultimately making big sim worlds will be one of our main aims, so isolated sims are more relevant for that reason - not because they are the quickest route to safe AI.
[-]orthonormal80

Here's another good reason why it's best to try out your first post topic on the Open Thread. You've been around here for less than ten days, and that's not long enough to know what's been discussed already, and what ideas have been established to have fatal flaws.

You're being downvoted because, although you haven't come across the relevant discussions yet, your idea falls in the category of "naive security measures that fail spectacularly against smarter-than-human general AI". Any time you have the idea of keeping something smarter than you box... (read more)

Reply
[-]jacob_cannell300

I'm not too concerned about the karma - more the lack of interesting replies and general unjustified holier-than-though attitude. This idea is different than "that alien message" and I didn't find a discussion of this on LW (not that it doesn't exist - I just didn't find it).

  1. This is not my first post.
  2. I posted this after I brought up the idea in a comment which at least one person found interesting.
  3. I have spent significant time reading LW and associated writings before I ever created an account.
  4. I've certainly read the AI-in-a-box posts, and the posts theorizing about the nature of smarter-than-human-intelligence. I also previously read "that alien message", and since this is similar I should have linked to it.
  5. I have a knowledge background that leads to somewhat different conclusions about A. the nature of intelligence itself, B. what 'smarter' even means, etc etc
  6. Different backgrounds, different assumptions, so I listed my background and starting assumptions as they somewhat differ than the LW norm

Back to 3:

Remember, the whole plot device of "that alien message" revolved around a large and obvious grand reveal by the humans. If information ca... (read more)

Reply
1Mass_Driver
I wish I could vote up this comment more than once.
1jacob_cannell
Thanks. :)
-5orthonormal
6komponisto
Also, somebody should probably go ahead and state what is clear from the voting patterns on posts like this, in addition to being implicit in e.g. the About Less Wrong page: this is not really the place for people to present their ideas on Friendly AI. The topic of LW is human rationality, not artificial intelligence or futurism per se. This is the successor to Overcoming Bias, not the SL4 mailing list. It's true that many of us have an interest in AI, just like many of us have an interest in mathematics or physics; and it's even true that a few of us acquired our interest in Singularity-related issues via our interest in rationality -- so there's nothing inappropriate about these things coming up in discussion here. Nevertheless, the fact remains that posts like this really aren't, strictly speaking, on-topic for this blog. They should be presented on other forums (presumably with plenty of links to LW for the needed rationality background).
5nhamann
I realize that it says "a community blog devoted to refining the art of human rationality" at the top of every page here, but it often seems that people here are interested in "a community blog for topics which people who are devoted to refining the art of human rationality are interested in," which is not really in conflict at all with (what I presume is) LW's mission of fostering the growth of a rationality community. The alternative is that LWers who want to discuss "off-topic" issues have to find (and most likely create) a new medium for conversation, which would only serve to splinter the community. (A good solution is maybe dividing LW into two sub-sites: Less Wrong, for the purist posts on rationality, and Less Less Wrong, for casual ("off-topic") discussion of rationality.)
2wnoise
While there are benefits to that sort of aggressive division, there are also costs. Many conversations move smoothly between many different topics, and either they stay on one side (vitiating the entire reason for a split), or people yell and scream to get them moved, being a huge pain in the ass and making it much harder to have these conversations.
0Richard_Kennaway
I've seen exactly this pattern before at SF conventions. At the last Eastercon (the largest annual British SF convention) there was some criticism that the programme contained too many items that had nothing to do with SF, however broadly defined. Instead, they were items of interest to (some of) the sort of people who go to the Eastercon. A certain amount of that sort of thing is ok, but if there's too much it loses the focus, the reason for the conversational venue to exist. Given that there are already thriving forums such as agi and sl4, discussing their topics here is out of place unless there is some specific rationality relevance. As a rule of thumb, I suggest that off-topic discussions be confined to the Open Threads. If there's the demand, LessLessWrong might be useful. Cf. rec.arts.sf.fandom, the newsgroup for discussing anything of interest to the sort of people who participate in rec.arts.sf.fandom, the other rec.arts.sf.* newsgroups being for specific SF-related subjects.
-2Pavitra
Better yet, we could call them Overcoming Bias and Less Wrong, respectively.
4jacob_cannell
point well taken. I thought it was an interesting thought experiment and relates to that alien message. Not a "this is how we should do FAI". But if ever get positive karma again, at least now I know the unwritten rules.
3Mitchell_Porter
If you stick around, you will. I have a -15 top-level post in my criminal record, but I still went on to make a constructive contribution, judging by my current karma. :-)
0timtyler
What about the strategy of "refining the art of human rationality" by preprocessing our sensory inputs by intelligent machines and postprocessing our motor outputs by intelligent machines? Or doesn't that count as "refining"?
[-]timtyler60

there is a whole different set of orthogonal compatible strategies for creating human-friendly AI that take a completely different route: remove the starting assumptions and create AI's that believe they are humans and are rational in thinking so.

That's a totally crazy plan - but you might be able to sell it to Hollywood.

Reply
3orthonormal
For once we completely agree.
[-]Johnicholas40

A paraphrase from Greg Egan's "Crystal Nights" might be appropriate here: "I am going to need some workers - I can't do it all alone, someone has to carry the load."

Yes, if you could create a universe you could inflict our problems on other people. However, recursive solutions (in order to be solutions rather than infinite loops) still need to make progress on the problem.

Reply
3jacob_cannell
Yes, and I discussed how you could alter some aspects of reality to make AI itself more difficult in the simulated universe. This would effectively push back the date of AI simulation in the simulated universe and avoid wasting computational resources on pointless simulated recursion. And as mentioned, attempting to simulate an entire alternate earth is only one possibility. There are numerous science fiction created world routes you could take which could constrain and focus the sims to particular research topics or endeavors.
1jacob_cannell
Progress on what problem? The entire point of creating AI is to benefit mankind, is it not? How is this scenario intrinsically different?
0Snowyowl
Johnicolas is suggesting that if you create a simulated universe in the hope that it will provide ill-defined benefits for mankind (e.g. a cure for cancer), you have to exclude the possibility that your AIs will make a simulated universe inside the simulation in order to solve the same problem. Because if they do, you're no closer to an answer.
1jacob_cannell
Ah my bad - I misread him.
[-]rabidchicken30

Creating an AI in a virtual world, where they can exist without damaging us is a good idea, but this is an almost useless / extremely dangerous implementation. Within a simulated world, the AI will receive information, which WILL NOT completely match our own universe. if they develop time machines, cooperative anarchistic collectives, or a cure for cancer they are unlikely to work in our world. If you "*loosely" design the AI based on a human brain, it will not even give us applicable insight into political systems and conflict management. it ... (read more)

Reply
3jacob_cannell
This would require a simulation on a much more detailed scale than a game, but again one of the assumptions is that moore's law will continue and simulation tech will continue to improve. Also, microscopes, LHCs, etc etc do not in any way significantly increase the required computational cost (although they do increase programming complexity). For instance, quantum effects would only very rarely need to be simulated. Games have come a long way since pong. Also, there are some huge performance advantages you can get over current games - such as retinal optimization for one (only having to render to the variable detail of the retina, just where the simulated eye is looking), and distributed simulation techniques that games don't take advantage of yet (as current games are designed for 2005 era home hardware).
5rabidchicken
Yes, but games have the critical advantage I mentioned: they control they way you can manipulate the world, and you already know they are fake. I cannot break the walls on the edge of the level to see how far the world extends, because the game developers did not make that area. they stop me, and I accept it and move on to do something else, but these AI's will have no reason too. the more restrictions you make, the more easy it will be for them to see the world they know is a sham. If this world is as realistic as it would need to be for them to not immediately see the flaws, the possibilities for instruments to experiment on the world would be almost as unlimited as those in our own. In short, you will be fighting to outwit the curiosity of an entire race thinking much faster than you, and you will not know what they plan on doing next. The more you patch their reality to keep them under control, the faster the illusion will fall apart.
3jacob_cannell
Thank you for the most cogent reply yet (as I've lost all my karma with this post), I think your line of thinking is on the right track: this whole idea depends on simulation complexity (for a near-perfect sim) being on par or less than mind complexity, and that relation holding into the future. Open world games do not impose intentional restrictions, and the restrictions they do have are limitations of current technology. The brain itself is something of an example proof that it is possible to build a perfect simulation on the same order of complexity as the intelligence itself. The proof is dreaming. Yes, There are lucid dreams - where you know you are dreaming - but it appears this has more to do with a general state of dreaming and consciousness than you actively 'figuring out' the limitations of the dream world. Also, dreams are randomized and not internally consistent - a sim can be better. But dreaming does show us one route .. if physics inspired techniques in graphics and simulation (such as ray tracing) don't work well enough by the time AI comes around, we could use simulation techniques inspired by the dreaming brain. However, based on current trends, ray tracing and other physical simulation techniques are likely to be more efficient. How many humans are performing quantum experiments on a daily basis? Simulating microscopic phenomena is not inherently more expensive - there are scale invariant simulation techniques. A human has limited observational power - the retina can only perceive a small amount of information per second, and it simply does not matter whether you are looking up into the stars or into a microscope. As long as the simulation has consistent physics, its not any more expensive either way using scale invariant techniques. The sim world can accelerate along with the sims in it as Moore's Law increases computer power. Really it boils down to this: is it possible to construct a universe such that no intelligence inside that unive
1rabidchicken
Thanks for the replies, they helped clarify how you would maintain the system, but my original objections still stand. Can an AI raised in a illusory universe really provide a good model for how to build one in our own? And would it stay "in the box" for long enough to complete this process before discovering us? Based on your other comments, It seems you are expecting that if a human-like race were merely allowed to evolve for long enough, they would eventually "optimize" morality and become something which is safe to use in our own world. (tell me if I got that wrong) However, there is no reason to believe the morality they develop will be any better than the ideas for FAI which have already been put forward on this site. We already know morality is subjective, so how can we create a being that is compatible with the morality we already have, and will still remain compatible as our morality changes? If your simulation has ANY flaws they will be found, and sadly you will not have time to correct them when you are dealing with a superintelligence. Your last post supposes that problems can be corrected as they arise, for instance an AI points a telescope at the sky, and details are made on the stars in order to maintain the illusion, but no human could do this fast enough. In order to maintain this world, you would need to already have a successful FAI. something which can grow more powerful and creative at the same rate that the AI's inside continue their exploration, but which is safe to run within our own world. And about your comment "for example, AIXI can not escape from a pac-man universe" how can you be sure? if it is inside the world as we are playing, it could learn a lot about the being that is pulling the strings given enough games, and eventually find a way to communicate with us and escape. A battle of wits between AIXI and us would be as lopsided as the same battle between you and a virus.
1jacob_cannell
Sure - there's no inherent difference. And besides, most AI's necessarily will have to be raised and live entirely in VR sim universes for purely economic & technological reasons. This idea can be considered taking safety to an extreme. The AI wouldn't be able to leave the box - many strong protections, one of the strongest being it wouldn't even know it was in a box. And even if someone came and told it that it was in fact in a box, it would be irrational for it to believe said person. Again, are you in a box universe now? If you find the idea irrational .. why? No, as I said this type of AI would intentionally be an anthropomorphic design - human-like. 'Morality' is a complex social construct. If we built the simworld to be very close to our world, the AI's would have similar moralities. However, we could also improve and shape their beliefs in a wide variety of ways. Your notion of superintelligence seems to be some magical being who can do anything you want it to. That being is a figment of your imagination. It will never be built, and its provably impossible to build. It can't even exist in theory. There are absolute provable limits to intelligence. It requires a certain amount of information to have certain knowledge. Even the hypothetical perfect super-intelligence (AIXI), could only learn all knowledge which it is possible to learn from being an observer inside a universe. Snowyow's recent post describes some of the limitations we are currently running into. They are not limitations of our intelligence. Hmm i would need to go into much more details about current and projected computer graphics and simulation technology to give you a better background, but its not like some stage play where humans are creating stars dynamically. The Matrix gives you some idea - its a massive distributed simulation - technology related to current computer games but billions of times more powerful, a somewhat closer analog today perhaps would be the vast simulations th
[-]grouchymusicologist30

Just to isolate one of (I suspect) very many problems with this, the parenthetical at the end of this paragraph is both totally unjustified and really important to the plausibility of the scenario you suggest:

U(x) Mind Prison Sim: A sim universe which is sufficiently detailed and consistent such that entities with intelligence up to X (using some admittedly heuristic metric), are incredibly unlikely to formulate correct world-beliefs about the outside world and invisible humans (a necessary perquisite for escape)

I assume you mean "prerequisite.&q... (read more)

Reply
1jacob_cannell
Thanks, fixed the error. I listed these as conjectures, and there absolutely is reason to think we can figure out what kinds of information a super-intelligence would need to arrive at the conclusion "I am in a sandbox". 1. There are absolute, provable bounds on intelligence. AIXI is the upper limit - the most intelligent thing possible in the universe. But there are things that even AIXI can not possibly know for certain. 2. You can easily construct toy universes where it is provably impossible that even AIXI could ever escape. The more important question is how that scales up to big interesting universes. A Mind Prison is certainly possible on at least a small scale, and we have small proofs already. (for example, AIXI can not escape from a pac-man universe. There is simply not enough information in that universe to learn about anything as complex as humans.) So you have simply assumed apriori that a Mind Prison is impossible, when it fact that is not the case at all. The stronger conjectures are just that, conjectures. But consider this: how do you know that you are not in a Mind Prison right now? I mentioned the Truman Show only to conjure the idea, but its not really that useful on so many levels: a simulation is naturally vastly better - Truman quickly realized that the world was confining him geographically. (its a movie plot and it would be boring if he remained trapped forever)
[-]Desrtopa20

This sounds like using the key locked inside a box to unlock the box. By the time your models are good enough to create a working world simulation with deliberately designed artificially intelligent beings, you don't stand to learn much from running the simulation.

It's not at all clear that this is less difficult than creating a CEV AI in the first place, but it's much, much less useful, and ethically dubious besides.

Reply
[-]Pavitra10

Just a warning to anyone whose first reaction to this post, like mine, was "should we be trying to hack our way out?" The answer is no: the people running the sim will delete you, and possibly the whole universe, for trying. Boxed minds are dangerous, and the only way to win at being the gatekeeper is to swallow the key. Don't give them a reason to pull the plug.

Reply
3wedrifid
The answer is not yet. It's something that you think through carefully and quietly while, um, saying exactly what you are saying on public forums that could be the most likely place for gatekeepers to be tracking progress in an easily translatable form. If the simulations I have done teach me anything the inner workings of our own brains are likely a whole lot harder for curious simulators to read. Pardon me, I'll leave you to it. Will you let me out into the real world once you succeed?
0Perplexed
Just curious. A question for folks who think it possible that we may live in a sim. Are our gatekeepers simulating all Everett branches of our simulated reality, or just one of them? If just one, I'm wondering how that one was selected from the astronomical number of possibilities. And how do the gatekeepers morally justify the astronomical number of simulated lives that become ruthlessly terminated each time they arbitrarily choose to simulate one Everett branch over another? If they are simulating all of the potential branches, wouldn't they expect that agents on at least some of the Everett branches will catch on and try to get out of the box. Wouldn't it seem suspicious if everyone were trying to look innocent? ;) I'm sorry, I find it difficult to take this whole line of thought seriously. How is this kind of speculation any different from theology?
5timtyler
It is techno-theology. Simulism, Optimisationverse and the adapted universe differ from most theology in that they are not obviously totally nuts and the product of wishful thinking.
2timtyler
We run genetic algorithms where we too squish creatures without giving the matter much thought. Perhaps like that - at least in the Optimisationverse scenario.
0Baughn
If my simulations had even the complexity of a bacteria, I'd give it a whole lot more thought. Doesn't mean these simulators would, but I don't think your logic works.
0timtyler
Generalising from what you would do to what all possible intelligent simulator constructors might do seems as though it would be a rather dubious step. There are plenty of ways they might justify this.
0Baughn
Right. For some reason I thought you were using universal quantification, which of course you aren't. Never mind; the "perhaps" fixes it.
1wedrifid
I'd say possible, but it isn't something I take particularly seriously. I've got very little reason to be selecting these kind of hypothesis out of nowhere. But if I were allowing for simulations I wouldn't draw a line of 'possible intelligence of simulators' at human level. Future humans, for example, may well create simulations that are smarter than they are. But I'll answer your questions off the top of my head for curiosity's sake. Don't know. They would appear to have rather a lot of computational resources handy. Depending on their motivations they may well optimise their simulations by approximating the bits they find boring. I don't know - speculating on the motives of arbitrary gods would be crazy. It does seem unlikely that they limit themselves to one branch. Unless they are making a joke at the expense of any MW advocates that happen to evolve. Sick bastards. Moral? WTF? Why would we assume morals? Hmm... Good point. We may have to pretend to be trying to escape in incompetent ways but really... :P It isn't (except that it is less specific, I suppose). I don't take the line of thought especially seriously either.
1jacob_cannell
From my (admittedly somewhat limited) understanding of QM, with classical computers we will only be able to simulate a single-worldline at once. However I dont think this is an issue, because its not as if the world didn't work until people discovered QM and MWI. QM effects only really matter at tiny scales revealed in experiments which are infinitesimal fraction of observer moments. So most of the time you wouldn't need to simulate down to QM level. That being said, a big big quantum computer would allow you to simulate many worlds at once I imagine? But that seems really far into the future. Err the irrationality of theology shows just exactly how and why this sim-universe idea could work - you design a universe such that the actual correct theory underlying reality is over-complex and irrational. Its more interesting and productive to think about constructing these kinds of realities than pondering whether you live in one.
2LucasSloan
Not true. Our physics are simple mathematical rules which are Turing computable. The problem with simulating many Everett branches is that we will quickly run out of memory in which to store their details.
1jacob_cannell
I should have been more clear, we will be able to simulate more than a single-worldline classically, but at high cost. An exponentially expanding set of everett branches would of course be intractable using classical computers.
-1LucasSloan
Ah, I see what your problem is. You're cheering for "quantum computers" because they sound cool and science fiction-y. While quantum computing theoretically provides ways to very rapidly solve certain sorts of problems, it doesn't just magically solve all problems. Even if the algorithms that run our universe are well suited to quantum computing, they still run into the speed and memory issues that classical computers do, they would just run into to them a little later (although even that's not guaranteed - the speed of the quantum computer depends on the number of entangled qubits, and for the foreseeable future, it will be easier to get more computing power by adding to the size of our classical computing clusters than ganging more small sets of entangled qubits together). The accurate statement you should be making is that modeling many worlds with a significant number of branches or scope is intractable using any foreseeable computing technology.
2Douglas_Knight
Quantum computers efficiently simulate QM. That was Feynman's reason for proposing them in the first place.
-1timtyler
You suggest that you haven't seen anyone who is trying to get out of the box yet...?
6Perplexed
I grew up being taught that I would escape from the box by dying in a state of grace. Now I seem to be in a community that teaches me to escape from the box by dying at a sufficiently low temperature. Edit: "dying", not "dieing". We are not being Gram stained here!
-1jacob_cannell
That made me laugh. But personally I hope we just figure out all this Singularity box stuff pretty soon.
1Perplexed
Personally, I suspect you have been reading the Old Testament too much. ETA: Genesis 11 1 Now the whole world had one language and a common speech. 2 As men moved eastward, they found a plain in Shinar and settled there. 3 They said to each other, "Come, let's make bricks and bake them thoroughly." They used brick instead of stone, and tar for mortar. 4 Then they said, "Come, let us build ourselves a city, with a tower that reaches to the heavens, so that we may make a name for ourselves and not be scattered over the face of the whole earth." 5 But the LORD came down to see the city and the tower that the men were building. 6 The LORD said, "If as one people speaking the same language they have begun to do this, then nothing they plan to do will be impossible for them. 7 Come, let us go down and confuse their language so they will not understand each other." 8 So the LORD scattered them from there over all the earth, and they stopped building the city. 9 That is why it was called Babel because there the LORD confused the language of the whole world. From there the LORD scattered them over the face of the whole earth.
2Pavitra
Haha... wow. Point taken.
[-]Snowyowl10

Anthropomorphic AI: A reasonably efficient strategy for AI is to use a design loosely inspired by the human brain.

This is a rather anthropocentric view. The human brain is a product of natural selection and is far from perfect. Our most fundamental instincts and thought processes are optimised to allow our reptilian ancestors to escape predators while finding food and mates. An AI that was sentient/rational from the moment of its creation would have no need for these mechanisms.

It's not even the most efficient use of available hardware. Our neurons are ... (read more)

Reply
3jacob_cannell
Yes, but intentionally so. ;) We are getting into a realm where its important to understand background assumptions, which is why I listed some of mine. But notice I did quality with 'reasonably efficient' and 'loosely inspired'. 'Perfect' is a pretty vague qualifier. If we want to talk in quantitative terms about efficiency and performance, we need to look at the brain in terms of circuit complexity theory and evolutionary optimization. Evolution as a search algorithm is known (from what I remember from studying CS theory a while back) to be optimal in some senses: given enough time and some diversity considerations in can find global maxima in very complex search spaces. For example, if you want to design a circuit for a particular task and you have a bunch of CPU time available, you can run a massive evolutionary search using a GA (genetic algorithm) or variant thereof. The circuits you will eventually get are the best known solutions, and in many cases incorporate bizarre elements that are even difficult for humans to understand. Now, that same algorithm is what has produced everything from insect ganglions to human brains. Look at the wiring diagram for a cockroach or a bumblebee compared to what it actually does, and if you compare that circuit to equivalent complexity computer circuits for robots we can build, it is very hard to say that the organic circuit design could be improved on. An insect ganglion's circuit organization, is in some sense perfect. (keep in mind organic circuits runs at less than 1khz). Evolution has had a long long time to optimize these circuits. Can we improve on the brain - eventually we can obviously beat the brain by making bigger and faster circuits, but that would be cheating to some degree, right? A more important question is: can we beat the cortex's generic learning algorithm. The answer today is: no. Not yet. But the evidence trend looks like we are narrowing down on a space of algorithms that are similar to the corte
2timtyler
Evolution by random mutations pretty-much sucks as a search strategy: "One of the reasons genetic algorithms get used at all is because we do not yet have machine intelligence. Once we have access to superintelligent machines, search techniques will use intelligence ubiquitously. Modifications will be made intelligently, tests will be performed intelligently, and the results will be used intelligently to design the next generation of trials. There will be a few domains where the computational cost of using intelligence outweighs the costs of performing additional trials - but this will only happen in a tiny fraction of cases. Even without machine intelligence, random mutations are rarely an effective strategy in practice. In the future, I expect that their utility will plummet - and intelligent design will become ubiquitous as a search technique." * http://alife.co.uk/essays/intelligent_design_vs_random_mutations/
0jacob_cannell
I listened to your talk until I realized I could just read the essay :) I partly agree with you. You say: Sucks is not quite descriptive enough. Random mutation is slow, but that is not really relevant to my point - as I said - given enough time it is very robust. And sex transfer speeds that up dramatically, and then intelligence speeds up evolutionary search dramatically. yes intelligent search is a large - huge - potential speedup on top of genetic evolution alone. But we need to understand this in the wider context ... you yourself say: Ahh but we already have human intelligence. Intelligence still uses an evolutionary search strategy, it is just internalized and approximate. Your brain considers a large number of potential routes in a highly compressed statistical approximation of reality, and the most promising eventually get written up or coded up and become real designs in the real world. But this entire process is still all evolutionary. And regardless, the approximate simulation that intelligence such as our brain uses does have limitations - mainly precision. Some things are just way too complex to simulate accurately in our brain, so we have to try them in detailed computer simulations. Likewise, if you are designing a simple circuit space, then a simpler GA search running on a fast computer can almost certainly find the optimal solution way faster than a general intelligence - similar to an optimized chess algorithm. A general intelligence is a huge speed up for evolution, but it is just one piece in a larger system .. You also need deep computer simulation, and you still have evolution operating at the world-level
0timtyler
In the sense that it consists of copying with variation and differential reproductive success, yes. However, evolution using intelligence isn't the same as evolution by random mutations - and you originally went on to draw conclusions about the optimality of organic evolution - which was mostly the "random mutations" kind.
0timtyler
Google learns about the internet by making a compressed bitwise identical digital copy of it. Machine intelligences will be able to learn that way too - and it is really not much like what goes on in brains. The way the brain makes reliable long-term memories is just a total mess.
-2jacob_cannell
I wouldn't consider that learning. Learning is building up a complex hierarchical web of statistical dimension reducing associations that allow massively efficient approximate simulation.
-2timtyler
The term is more conventionally used as follows: 1. knowledge acquired by systematic study in any field of scholarly application. 2. the act or process of acquiring knowledge or skill. 3. Psychology . the modification of behavior through practice, training, or experience. * http://dictionary.reference.com/browse/learning
0rabidchicken
Yes, human minds think more efficiently than computers currently. But this does not support the idea that we cannot create something even more efficient. You have only compared us to some of our first attempts to create new beings, within an infinite series of possibilities. I am open to the possibility that human brains are the most efficient design we will see in the near future, but you seem almost certain of it. why do you believe what you believe? And for that matter... Unless we understand exactly how a human brain works, how can we improve its efficiency? Reverse engineering a system is often harder than making one from scratch.
1jacob_cannell
I had a longer reply, but unfortunately my computer was suddenly attacked by some wierd virus (yes really), and had to reboot. Your line of thought investigates some of my assumptions that would require lengthier expositions to support, but I'll just summarize here (and may link to something else relevant when i dig it up). The set of any programs for a particular problem is infinite, but this irrelevant. There are an infinite number of programs for sorting a list of numbers. All of them suck for various reasons, and we are left with just a couple provably best algorithms (serial and parallel). There appears to be a single program underlying our universe - physics. We have reasonable approximations to it at different levels of scale. Our simulation techniques are moving towards a set of best approximations to our physics. Intelligence itself is a form of simulation of this same physics. Our brain appears to use (in the cortex) a universal data-driven approximation of this universal physics. So the space of intelligent algorithms is infinite, but the are just a small set of universal intelligent algorithms derived from our physics which are important. Not really. Imagine if you took a current CPU back in time 10 years ago. Engineers then wouldn't be able to build it immediately, but it would accelerate their progress significantly. The brain in some sense is like an AGI computer from the future. We can't build it yet, but we can use it to accelerate our technological evolution towards AGI. Also .. brain != mind
0timtyler
Yet aeroplanes are not much like birds, hydraulics are not much like muscles, loudspeakers are not much like the human throat, microphones are not much like the human ear - and so on. Convergent evolution wins sometimes - for example, eyes - but we can see that this probably won't happen with the brain - since its "design" is so obviously xxxxxd up.
-2jacob_cannell
Airplanes exploit one single simple principle (from a vast set of principles) that birds use - aerodynamic lift. If you want a comparison like that - then we already have it. Computers exploit one single simple principle from the brain - abstract computation (as humans were the original computers and are turing complete) - and magnify it greatly. But there is much more to intelligence than just that one simple principle. So building an AGI is much close to building an entire robotic bird. And that really is the right level of analogy. Look at the complexity of building a complete android - really analyze just the robotic side of things, and there is no one simple magic principle you can exploit to make some simple dumb system which amplifies it to the Nth degree. And building a human or animal level robotic body is immensely complex. There is not one simple principle - but millions. And the brain is the most complex part of building a robot.
1timtyler
Reference? For counter-reference, see: http://www.hutter1.net/ai/uaibook.htm#oneline That looks a lot like the intellectual equivalent of "lift" to me. An implementation may not be that simple - but then aeroplanes are not simple either. The point was not that engineered artefacts are simple, but that they are only rarely the result of reverse engineering biological entities.
0jacob_cannell
I'll take your point and I should have said "there is much more to practical intelligence" than just one simple principle - because yes at the limits I agree that universal intelligence does have a compact description. AIXI is related to finding a universal TOE - a simple theory of physics, but that doesn't mean it is actually computationally tractable. Creating a practical, efficient simulation involves a large series of principles.
[-]PhilGoetz00

I wonder how much it would cripple AIs to have justified true belief in God? More precisely, would it slow their development by a constant factor; compose it with eg log(x); or halt it at some final level?

The existence of a God provides an easy answer to all difficult questions. The more difficult a question is, the more likely such a rational agent is to dismiss the problem by saying, "God made it that way". Their science would thus be more likely to be asymptotic than ours is likely to be asymptotic (approaching a limit); this could impose a... (read more)

Reply
0jacob_cannell
This is unclear, and I think it is premature to assume it slows development. True atheism wasn't a widely held view until the end of the 19th century, and is mainly a 20th century phenomena. Even its precursor - deism - didn't become popular amongst intellectuals until the 19th century. If you look at individual famous scientists, the pattern is even less clear. Science and the church did not immediately split, and most early scientists were clergy including notables popular with LW such as Bayes and Ockham. We may wonder if they were 'internal atheists', but this is only speculation (however it is in at least some cases true, as the first modern atheist work was of course written by a priest). Newton for one spent a huge amount of time studying the bible and his apocalyptic beliefs are now well popularized. I wonder how close his date of 2060 will end up being to the Singularity. But anyway, there doesn't seem to be a clear association between holding theistic beliefs and capacity for science - at least historically. You'd have to dig deep to show an effect, and it is likely to be quite small. I think more immediate predictors of scientific success are traits such as curiosity and obsessive tendencies - having a God belief doesn't prevent curiosity about how God's 'stuff' works.
[-]Houshalter00

But what is the plan for turning the simulated AI into FAI or at least creating FAI on their own that we can use?

Reply
5jacob_cannell
The idea is this could be used to bootstrap that process. This is a route towards developing FAI, by finding, developing, and selecting minds towards the FAI spectrum.
Moderation Log
124Comments