The Halting Problem (Part Two)
Can we have plans for thinking about other plans? Yes, we can!
Suppose that we found a plan, and we did not know what kind of plan it is.
Maybe it is a plan for how to make a food.
Or maybe it is a plan for how to go by car to another city.
Or maybe it is a plan for how to build a house.
We don't know.
Can we have a plan for finding out?
Yes! Here is a plan for telling what kind of plan it is:
So we can have a plan for reading and thinking about other plans.
This plan is not perfect, but it is pretty good.
It can even tell if a plan is a plan for reading plans.
This is like looking in the mirror and knowing that you are seeing yourself.
It is a very interesting thing!
But .... can we make a plan for telling if a plan will end or not?
That is a hard problem.
Edited to add the italicized comment on step 3.
The Halting Problem (Part Three)
Let's imagine that we have a plan for reading other plans and saying if they will end.
Our imaginary plan is called E, for Ending. We want to know if a plan like E is possible.
We do not know what the steps of plan E are.
All we know is that we are imagining that we can follow plan E to read another plan and say whether it will end or not.
(We need a name for this other plan. We'll call it X.)
But wait! We know there are plans that sometimes end, and sometimes go on forever. Here is one —
Plan Z:
xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.
This seemed to me and Briénne to be a really fun exercise, both for tabooing one's words and for communicating difficult concepts to laypeople. So why not make a game out of it? Pick any tough, important, or interesting argument or idea, and use this text editor to try to describe what you have in mind with extremely common words only.
This is challenging, so if you almost succeed and want to share your results, you can mark words where you had to cheat in *italics*. Bonus points if your explanation is actually useful for gaining a deeper understanding of the idea, or for teaching it, in the spirit of Gödel's Second Incompleteness Theorem Explained in Words of One Syllable.
As an example, here's my attempt to capture the five theses using only top-thousand words:
If you make a really strong computer and it is not very nice, you will not go to space today.
Other ideas to start with: agent, akrasia, Bayes' theorem, Bayesianism, CFAR, cognitive bias, consequentialism, deontology, effective altruism, Everett-style ('Many Worlds') interpretations of quantum mechanics, entropy, evolution, the Great Reductionist Thesis, halting problem, humanism, law of nature, LessWrong, logic, mathematics, the measurement problem, MIRI, Newcomb's problem, Newton's laws of motion, optimization, Pascal's wager, philosophy, preference, proof, rationality, religion, science, Shannon information, signaling, the simulation argument, singularity, sociopathy, the supernatural, superposition, time, timeless decision theory, transfinite numbers, Turing machine, utilitarianism, validity and soundness, virtue ethics, VNM-utility