1. Keep the AI in a box and don't interact with it. Don't have any conversations with it whatsoever. I'll explain how this would work in a second, but first some motivation. 2. Consider all the great scientific discoveries over the course of human history -- Newtonian mechanics, relativity, algorithms...
The concept of strength is ubiquitous in our culture. It is commonplace to hear one person described as "stronger" or "weaker" than another. And yet the notion of strength is a a pernicious myth which reinforces many our social ills and should be abandoned wholesale. 1. Just what is strength,...
Here is my attempt at a calculation. Disclaimer: this is based on googling. If you are actually knowledgeable in the subject, please step in and set me right.
There are 10^11 neurons in the human brain.
A neuron will fire about 200 times per second.
It should take a constant number of flops to decide whether a neuron will fire -- say 10 flops (no need to solve a differential equation, neural networks usually use some discrete heuristics for something like this)
I want a society of 10^6 orcs running for 10^6 years
As you suggest, lets let the simulation run for a year of real time (moving away at this point from... (read more)
Still inability to realise what you are doing seems rather dangerous.
So far, all I've done is post a question on lesswrong :)
More seriously, I do regret it if I appeared unaware of the potential danger. I am of course aware of the possibility that experiments with AI might destroy humanity. Think of my post above a suggesting a possible approach to investigate -- perhaps one with some kinks as written (that is why I'm asking a question here) but (I think) with the possibility of one day having rigorous safety guarantees.
I think this calculation too conservative. The reason is (as I understand it) that neurons are governed by various differential equations, and simulating them accurately is a pain in the ass. We should instead assume that deciding whether a neuron will fire will take a constant number of flops.
I'll write another comment which attempts to redo your calculation with different assumptions.
It seems to me that by the time we can do that, we should have figured out a better way to create AI.
But will we have figured a way to reap the gains of AI safely for humanity?
but more than that, it knows that the real universe is capable of producing intelligent beings that chose this particular world to simulate.
Good point -- this undermines a lot of what I wrote in my update 1. For example, I have no idea if F = m d^3 x / dt would result in a world that is capable of producing intelligent beings.
I should at some point produce a version of the above post with this claim, and other questionable parenthetical remarks I made, deleted, or at least acknowledging that they require further argumentation; they are not necessary for the larger point, which is that as long as the only thing the superintelligence can do (by definition) is live in a simulated world governed by Newton's laws, and as long as we don't interact with it at all except to see an automatically verified answer to a preset question (e.g., factor "111000232342342"), there is nothing it can do to harm us.
I guess I am willing to bite the bullet and say that, as long as entity X prefers existence to nonexistence, you have done it no harm by bringing it into being. I realize this generates a number of repulsive-sounding conclusions, e.g., it becomes ethical to create entities which will live, by our 21st century standards, horrific lives.
At least some of them will tell you they had rather not been born.
If one is willing to accept my reasoning above, I think one can take one more leap and say that statistically as long as the vast majority of these entities will prefer existing to never having been brought into being, we are in the clear.
The theorem you cite (provided I understood you correctly) does not preclude the possibility of checking whether a program written in a certain pre-specified format will have bugs. Bugs here are defined to be certain undesirable properties (e.g., looping forever, entering certain enumerated states, etc).
Baby versions of such tools (which automatically check whether your program will have certain properties from inspecting the code) already exist.
Nice idea...I wrote an update to the post suggesting what seemed to me to be a variation on your suggestion.
About program checking: I agree completely. I'm not very informed about the state of the art, but it is very plausible that what we know right now is not yet up to task.
I'm a bit skeptical about whether you can actually create a superintelligent AI by combining sped up humans like that,
Why not? You are pretty smart, and all you are is a combination of 10^11 or so very "dumb" neurons. Now imagine a "being" which is actually a very large number of human-level intelligences, all interacting...
is very, very difficult not to give a superintelligence any hints of how the physics of our world work.
I wrote a short update to the post which tries to answer this point.
Maybe they notice minor fluctuations in the speed of the simulation based on environmental changes to the hardware
I believe they should have no ability whatsoever to detect fluctuations in the speed of the simulation.
Consider how the world of world of warcraft appears to an orc inside the game. Can it tell the speed at which the hardware is running the game?
It can't. What it can do is compare the speed of different things: how fast does an apple fall... (read more)
1. Keep the AI in a box and don't interact with it.
Don't have any conversations with it whatsoever.
I'll explain how this would work in a second, but first some motivation.
2. Consider all the great scientific discoveries over the course of human history -- Newtonian mechanics, relativity, algorithms for primality testing, etc etc.
Could a "God" who is ``bad at math'' have created the universe in order to have us make these discoveries for him?
It sounds insane: surely any being with the power to create our universe would know how to check if a number is prime.
3. But perhaps it isn't insane. Consider that at this point in its history, we
a) do not know... (read 1605 more words →)
The concept of strength is ubiquitous in our culture. It is commonplace to hear one person described as "stronger" or "weaker" than another. And yet the notion of strength is a a pernicious myth which reinforces many our social ills and should be abandoned wholesale.
1. Just what is strength, exactly? Few of the people who use the word can provide an exact definition.
On first try, many people would say that strength is the ability to lift heavy objects. But this completely ignores the strength necessary to push or pull on objects; to run long distances without exhausting oneself; to throw objects with great speed; to balance oneself on a tightrope, and so... (read 436 more words →)
Right, that is what i meant.