xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.
This seemed to me and Briénne to be a really fun exercise, both for tabooing one's words and for communicating difficult concepts to laypeople. So why not make a game out of it? Pick any tough, important, or interesting argument or idea, and use this text editor to try to describe what you have in mind with extremely common words only.
This is challenging, so if you almost succeed and want to share your results, you can mark words where you had to cheat in *italics*. Bonus points if your explanation is actually useful for gaining a deeper understanding of the idea, or for teaching it, in the spirit of Gödel's Second Incompleteness Theorem Explained in Words of One Syllable.
As an example, here's my attempt to capture the five theses using only top-thousand words:
- Intelligence explosion: If we make a computer that is good at doing hard things in lots of different situations without using much stuff up, it may be able to help us build better computers. Since computers are faster than humans, pretty soon the computer would probably be doing most of the work of making new and better computers. We would have a hard time controlling or understanding what was happening as the new computers got faster and grew more and more parts. By the time these computers ran out of ways to quickly and easily make better computers, the best computers would have already become much much better than humans at controlling what happens.
- Orthogonality: Different computers, and different minds as a whole, can want very different things. They can want things that are very good for humans, or very bad, or anything in between. We can be pretty sure that strong computers won't think like humans, and most possible computers won't try to change the world in the way a human would.
- Convergent instrumental goals: Although most possible minds want different things, they need a lot of the same things to get what they want. A computer and a human might want things that in the long run have nothing to do with each other, but have to fight for the same share of stuff first to get those different things.
- Complexity of value: It would take a huge number of parts, all put together in just the right way, to build a computer that does all the things humans want it to (and none of the things humans don't want it to).
- Fragility of value: If we get a few of those parts a little bit wrong, the computer will probably make only bad things happen from then on. We need almost everything we want to happen, or we won't have any fun.
If you make a really strong computer and it is not very nice, you will not go to space today.
Other ideas to start with: agent, akrasia, Bayes' theorem, Bayesianism, CFAR, cognitive bias, consequentialism, deontology, effective altruism, Everett-style ('Many Worlds') interpretations of quantum mechanics, entropy, evolution, the Great Reductionist Thesis, halting problem, humanism, law of nature, LessWrong, logic, mathematics, the measurement problem, MIRI, Newcomb's problem, Newton's laws of motion, optimization, Pascal's wager, philosophy, preference, proof, rationality, religion, science, Shannon information, signaling, the simulation argument, singularity, sociopathy, the supernatural, superposition, time, timeless decision theory, transfinite numbers, Turing machine, utilitarianism, validity and soundness, virtue ethics, VNM-utility
Cognitive Biases
In the world, things happen for reasons. When anything happens ever, there's a reason for it- even if you don't know what it is, or it seems strange. Start with that: nothing has ever happened without a cause. (Here we mean "reason" and "cause" like how a ball rolling into another ball will knock it over, not like good or bad. Think about it- it makes sense.)
If you're interested in knowing more about the world, often, you want to know the real reason things happen (or the reason other things DON'T happen, which can be just as important!) If you do that, you can learn about a lot of things: why the land looks the way it does, all about the different stars, tiny things much smaller than you can see, even all about other people!
But your brain isn't the very best at doing this. Remember that idea about how animals change over time? How parents, and the parents of parents, all make a kind of animal change a little all the time, because of who lives and who doesn't? You know, the idea from the old man who said humans used to be just animals? Well, think about that- our brains let us think about so much, but they used to be just animal brains. And an animal doesn't need to worry so much about true reasons- especially for things that are too tiny to see, or big things up in the sky. Say, when an animal sees something big and bad, it would be bad if it stopped and thought about all the reasons it happened. It's best to just run away!
Human brains aren't any different, really, except in two ways: We don't want to just always run away, and we can change how we think about things! But if we don't learn how to think about real reasons, then we use animal-brain thinking- and that can make a lot of problems, especially when it's about things that animals never thought about.
If you want to learn about the real world, and real reasons, it's important to know about animal-brains and how they can be wrong. Remember, animal-brain thinking is hard to spot- if you don't look for it, it just seems like normal clear thinking. But when you see it. you can fix how it works in your own brain, and see the world a little clearer.
This works as a beginning to explaining this group of ideas. I like the focus on passed-down-change, but I want us to do more to exactly pick out what we mean here. It's especially important to note: