Complexity and Fragility of Value, My take: When people talk about the things they want, they usually don't say very many things. But when you check what things people actually want, they want a whole lot of different things. People also sometimes don't realize that they want things because they have always had those things and never worried that they might lose them.
If we were to write a book of all the things people want so a computer could figure out ways to give people the things they want, the book would probably be long and hard to write. If there were some small problems in the book, the computer wouldn't be able to see the problems and would give people the wrong things. That would probably be very, very, very bad.
Risks of Creative Super AIs: If we make computers, they will never know to do anything that people didn't tell them to do. We can tell computers to try to figure things out for themselves but even then we need to get them started on that. Computers will not know what people want except if people tell the computers exactly what they want. Very strong computers might get really stupid ideas about what they should do because they were wrong about what humans want. Also, very strong computers might do really bad things we don't want before we can turn them off.
xkcd's Up-Goer Five comic gave technical specifications for the Saturn V rocket using only the 1,000 most common words in the English language.
This seemed to me and Briénne to be a really fun exercise, both for tabooing one's words and for communicating difficult concepts to laypeople. So why not make a game out of it? Pick any tough, important, or interesting argument or idea, and use this text editor to try to describe what you have in mind with extremely common words only.
This is challenging, so if you almost succeed and want to share your results, you can mark words where you had to cheat in *italics*. Bonus points if your explanation is actually useful for gaining a deeper understanding of the idea, or for teaching it, in the spirit of Gödel's Second Incompleteness Theorem Explained in Words of One Syllable.
As an example, here's my attempt to capture the five theses using only top-thousand words:
If you make a really strong computer and it is not very nice, you will not go to space today.
Other ideas to start with: agent, akrasia, Bayes' theorem, Bayesianism, CFAR, cognitive bias, consequentialism, deontology, effective altruism, Everett-style ('Many Worlds') interpretations of quantum mechanics, entropy, evolution, the Great Reductionist Thesis, halting problem, humanism, law of nature, LessWrong, logic, mathematics, the measurement problem, MIRI, Newcomb's problem, Newton's laws of motion, optimization, Pascal's wager, philosophy, preference, proof, rationality, religion, science, Shannon information, signaling, the simulation argument, singularity, sociopathy, the supernatural, superposition, time, timeless decision theory, transfinite numbers, Turing machine, utilitarianism, validity and soundness, virtue ethics, VNM-utility