The Up-Goer Five Game: Explaining hard ideas with simple words

Rob Bensinger

xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.

This seemed to me and Briénne to be a really fun exercise, both for tabooing one's words and for communicating difficult concepts to laypeople. So why not make a game out of it? Pick any tough, important, or interesting argument or idea, and use this text editor to try to describe what you have in mind with extremely common words only.

This is challenging, so if you almost succeed and want to share your results, you can mark words where you had to cheat in *italics*. Bonus points if your explanation is actually useful for gaining a deeper understanding of the idea, or for teaching it, in the spirit of Gödel's Second Incompleteness Theorem Explained in Words of One Syllable.

As an example, here's my attempt to capture the five theses using only top-thousand words:

Intelligence explosion: If we make a computer that is good at doing hard things in lots of different situations without using much stuff up, it may be able to help us build better computers. Since computers are faster than humans, pretty soon the computer would probably be doing most of the work of making new and better computers. We would have a hard time controlling or understanding what was happening as the new computers got faster and grew more and more parts. By the time these computers ran out of ways to quickly and easily make better computers, the best computers would have already become much much better than humans at controlling what happens.
Orthogonality: Different computers, and different minds as a whole, can want very different things. They can want things that are very good for humans, or very bad, or anything in between. We can be pretty sure that strong computers won't think like humans, and most possible computers won't try to change the world in the way a human would.
Convergent instrumental goals: Although most possible minds want different things, they need a lot of the same things to get what they want. A computer and a human might want things that in the long run have nothing to do with each other, but have to fight for the same share of stuff first to get those different things.
Complexity of value: It would take a huge number of parts, all put together in just the right way, to build a computer that does all the things humans want it to (and none of the things humans don't want it to).
Fragility of value: If we get a few of those parts a little bit wrong, the computer will probably make only bad things happen from then on. We need almost everything we want to happen, or we won't have any fun.

If you make a really strong computer and it is not very nice, you will not go to space today.

Other ideas to start with: agent, akrasia, _{Bayes' theorem}, _Bayesianism, CFAR, cognitive bias, consequentialism, deontology, _{effective altruism}, Everett-style ('Many Worlds') interpretations of quantum mechanics, entropy, evolution, the Great Reductionist Thesis, _{halting problem}, humanism, law of nature, LessWrong, logic, mathematics, the measurement problem, MIRI, Newcomb's problem, Newton's laws of motion, optimization, _{Pascal's wager}, philosophy, preference, proof, rationality, religion, science, Shannon information, signaling, the simulation argument, singularity, sociopathy, the supernatural, superposition, time, timeless decision theory, transfinite numbers, _{Turing machine}, _{utilitarianism}, validity and soundness, virtue ethics, VNM-utility

xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.

As an example, here's my attempt to capture the five theses using only top-thousand words:

Intelligence explosion: If we make a computer that is good at doing hard things in lots of different situations without using much stuff up, it may be able to help us build better computers. Since computers are faster than humans, pretty soon the computer would probably be doing most of the work of making new and better computers. We would have a hard time controlling or understanding what was happening as the new computers got faster and grew more and more parts. By the time these computers ran out of ways to quickly and easily make better computers, the best computers would have already become much much better than humans at controlling what happens.
Orthogonality: Different computers, and different minds as a whole, can want very different things. They can want things that are very good for humans, or very bad, or anything in between. We can be pretty sure that strong computers won't think like humans, and most possible computers won't try to change the world in the way a human would.
Convergent instrumental goals: Although most possible minds want different things, they need a lot of the same things to get what they want. A computer and a human might want things that in the long run have nothing to do with each other, but have to fight for the same share of stuff first to get those different things.
Complexity of value: It would take a huge number of parts, all put together in just the right way, to build a computer that does all the things humans want it to (and none of the things humans don't want it to).
Fragility of value: If we get a few of those parts a little bit wrong, the computer will probably make only bad things happen from then on. We need almost everything we want to happen, or we won't have any fun.

If you make a really strong computer and it is not very nice, you will not go to space today.

The Halting Problem (Part Three)

Let's imagine that we have a plan for reading other plans and saying if they will end.
Our imaginary plan is called E, for Ending. We want to know if a plan like E is possible.
We do not know what the steps of plan E are.
All we know is that we are imagining that we can follow plan E to read another plan and say whether it will end or not.
(We need a name for this other plan. We'll call it X.)

But wait! We know there are plans that sometimes end, and sometimes go on forever. Here is one —

Plan Z:

Start with a number.
If our number is zero, stop.
Make our number smaller by 1.
Go to step 2.

Plan Z will always stop if the number we start with is bigger than zero and is a whole number.
But if our number is one-half (or something else not whole) then Z will never end.
That is because our number will go right past zero without ever being zero.

Plan Z is not really whole by itself.
It needs something else that we give it: the number in step 1.
We can think of this number as "food" for the plan.
The "food" is something Z needs in order to go, or even to make sense.
Some food is good for you, and some is bad for you ...
... and whether Z ends or not depends on what number we feed it.

Plan Z ends if we feed it the number 1 or 42, but not if we feed it the number one-half.

And so when we ask "Will plan X end?" we really should ask "Will plan X end, if we feed F to plan X?"

So in order to follow plan E, we need to know two things: a plan X, and a something called F.
(What kind of something? Whatever kind X wants.
If X wants a number, then F is a number. If X wants a cookie, then F is a cookie.
If X wants a plan to read, then F is a plan.)
Following E will then tell us if X-fed-with-F will end or run forever.

Now here is another plan —

Plan G:

Start with a plan. Call it plan X.
Follow plan E to read plan X, with plan X as food.
This will tell us if plan X will end or not.
Now, if E told us that X never ends, then stop!
But if E told us that X stops, then sing "The Song That Never Ends".

So when we follow plan G, we don't do the same thing that X does.
We do the other thing!
If X never ends, then G ends.
And if X ends, then G never ends.

But what happens if X is G?
G is a plan, and G wants a plan for its food. So we can feed G to G itself!
If we feed G to G, then G will end if G doesn't end.
And G will go on forever if G ends.

That does not make any sense at all.
Everything about that makes no sense!
It is like saying "If the cat is white, then the cat is not white."
It is really wrong!

What part is wrong, though?
G is very simple. There is nothing wrong with G.
The wrongness is in the thing that we imagined: Plan E, the plan that can tell us if any plan will end or not.

This means that E is not really possible.
That is the part that looked like it might make sense, but really it did not.
Oh well.
It sure would be nice if E was possible.
But it is not.

44

The Up-Goer Five Game: Explaining hard ideas with simple words

44

44

44

The Up-Goer Five Game: Explaining hard ideas with simple words

44

44