The Up-Goer Five Game: Explaining hard ideas with simple words

Rob Bensinger

xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.

This seemed to me and Briénne to be a really fun exercise, both for tabooing one's words and for communicating difficult concepts to laypeople. So why not make a game out of it? Pick any tough, important, or interesting argument or idea, and use this text editor to try to describe what you have in mind with extremely common words only.

This is challenging, so if you almost succeed and want to share your results, you can mark words where you had to cheat in *italics*. Bonus points if your explanation is actually useful for gaining a deeper understanding of the idea, or for teaching it, in the spirit of Gödel's Second Incompleteness Theorem Explained in Words of One Syllable.

As an example, here's my attempt to capture the five theses using only top-thousand words:

Intelligence explosion: If we make a computer that is good at doing hard things in lots of different situations without using much stuff up, it may be able to help us build better computers. Since computers are faster than humans, pretty soon the computer would probably be doing most of the work of making new and better computers. We would have a hard time controlling or understanding what was happening as the new computers got faster and grew more and more parts. By the time these computers ran out of ways to quickly and easily make better computers, the best computers would have already become much much better than humans at controlling what happens.
Orthogonality: Different computers, and different minds as a whole, can want very different things. They can want things that are very good for humans, or very bad, or anything in between. We can be pretty sure that strong computers won't think like humans, and most possible computers won't try to change the world in the way a human would.
Convergent instrumental goals: Although most possible minds want different things, they need a lot of the same things to get what they want. A computer and a human might want things that in the long run have nothing to do with each other, but have to fight for the same share of stuff first to get those different things.
Complexity of value: It would take a huge number of parts, all put together in just the right way, to build a computer that does all the things humans want it to (and none of the things humans don't want it to).
Fragility of value: If we get a few of those parts a little bit wrong, the computer will probably make only bad things happen from then on. We need almost everything we want to happen, or we won't have any fun.

If you make a really strong computer and it is not very nice, you will not go to space today.

Other ideas to start with: agent, akrasia, _{Bayes' theorem}, _Bayesianism, CFAR, cognitive bias, consequentialism, deontology, _{effective altruism}, Everett-style ('Many Worlds') interpretations of quantum mechanics, entropy, evolution, the Great Reductionist Thesis, _{halting problem}, humanism, law of nature, LessWrong, logic, mathematics, the measurement problem, MIRI, Newcomb's problem, Newton's laws of motion, optimization, _{Pascal's wager}, philosophy, preference, proof, rationality, religion, science, Shannon information, signaling, the simulation argument, singularity, sociopathy, the supernatural, superposition, time, timeless decision theory, transfinite numbers, _{Turing machine}, _{utilitarianism}, validity and soundness, virtue ethics, VNM-utility

xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.

As an example, here's my attempt to capture the five theses using only top-thousand words:

Intelligence explosion: If we make a computer that is good at doing hard things in lots of different situations without using much stuff up, it may be able to help us build better computers. Since computers are faster than humans, pretty soon the computer would probably be doing most of the work of making new and better computers. We would have a hard time controlling or understanding what was happening as the new computers got faster and grew more and more parts. By the time these computers ran out of ways to quickly and easily make better computers, the best computers would have already become much much better than humans at controlling what happens.
Orthogonality: Different computers, and different minds as a whole, can want very different things. They can want things that are very good for humans, or very bad, or anything in between. We can be pretty sure that strong computers won't think like humans, and most possible computers won't try to change the world in the way a human would.
Convergent instrumental goals: Although most possible minds want different things, they need a lot of the same things to get what they want. A computer and a human might want things that in the long run have nothing to do with each other, but have to fight for the same share of stuff first to get those different things.
Complexity of value: It would take a huge number of parts, all put together in just the right way, to build a computer that does all the things humans want it to (and none of the things humans don't want it to).
Fragility of value: If we get a few of those parts a little bit wrong, the computer will probably make only bad things happen from then on. We need almost everything we want to happen, or we won't have any fun.

If you make a really strong computer and it is not very nice, you will not go to space today.

The Halting Problem (Part One)

A plan is a list of things to do.
When a computer runs, it is doing the things that are written in a plan.
When you solve a problem like 23 × 3, you are also following a plan.

Plans are made of steps.
To follow a plan, you do what each plan step says to do, in the order they are written.
But sometimes a step can tell you to move to a different step in the plan, instead of the next one.
And sometimes it can tell you to do different things if you see something different.
It can say "Go back to step 4" ... or "If the water is not hot yet, wait two minutes, then go back to step 3."

Here is a plan:

Walk to the store.
Buy a food.
Come back home.
You're done!

Here is another plan:

Walk to the store.
Buy a food.
Come back home.
Go back to step 1.

There is something funny about the second plan!
If we started following that plan, we would never stop.
We would just keep walking to the store, buying a food, and walking back home.
Forever.
(Or until we decide it is a dumb plan and we should stop following it.
But a computer couldn't do that.)

You may have heard songs like "The Song That Never Ends" or "Ninety-Nine Bottles of Drinks on the Wall".
Both of these songs are like plans.
When you are done singing a part of the song, you follow a plan step to get to the next part.
In "The Song That Never Ends", you just go back to the beginning.
But in "Ninety-Nine Bottles of Drinks on the Wall", you take one away from the number of bottles of drinks.
And if there are no more bottles, you stop!
But if there are more bottles, you sing the next part.
Even though it is a very long song, it does have an end.

(There is a bad joke about people who write plans for computers.
It is also a joke about hair soap.
There is a plan written on the hair soap bottle:

Put hair soap in your hair.
Use water to get the hair soap out of your hair.
Repeat.

The person who writes plans for computers starts washing their hair and does not know when to stop!

Normal people know that "repeat" just means "do it a second time, then stop" and not "keep doing it forever". But in a computer plan, it would mean "do it forever".)

It is nice to know out how long a plan will take for us to do.
It is really nice to know if it is one of those plans that never ends.
If it is, then we could just say, "I won't follow this plan! It never ends! It is like that dumb song or the bad joke!"

The Halting Problem (Part Two)

Can we have plans for thinking about other plans? Yes, we can!

Suppose that we found a plan, and we did not know what kind of plan it is.
Maybe it is a plan for how to make a food.
Or maybe it is a plan for how to go by car to another city.
Or maybe it is a plan for how to build a house. We don't know.
Can we have a plan for finding out?

Yes! Here is a plan for telling what kind of plan it is:

Get paper and a writing stick.
Start at the first step of the plan we are reading.
Read that step.
(Do not do the things that step says to do

... (read more)

44

The Up-Goer Five Game: Explaining hard ideas with simple words

44

44

44

The Up-Goer Five Game: Explaining hard ideas with simple words

44

44