Lately I've been looking into learning the material in the MIRI research guide. After some time, I noticed that my biggest trepidation about diving in was the nagging question of, "But how will I know if I actually learned the stuff?"
Once I realized that was the thing holding me back, it became easier to think about solutions. Some of the topics correspond with courses that are common in universities, so I can pilfer final exams from their sites (woot to MIT open courseware). But for other topics I wanted to see what people here thought.
In whatever domain you specialize in, what are some examples of problems or questions that one can only answer by having a solid understanding of a huge swath of said domain? Below I've listed everything from the MIRI research guide that I could perceive as a distinct category.
(ex. If you were learning about Digital Systems and Computer Architecture, my test would be "In systemVerilog, simulate a basic 16-bit processor that can be programmed using a RISC assembly language of your design.")
(edit: I know that "huge swaths" is pretty vague, and suggestions don't have to be things that you think certify/prove that you get a topic. While a "comprehensive test" would be nice, problems/prompts like what Qiaochu commented are exactly what I'm looking for)
- Set theory
- Probability
- Probabilistic Inference
- Statistics
- Machine Learning
- Solomonoff Induction
- Naturalized Induction
- VNM Decision Theory
- Functional Decision Theory
- Logical Uncertainty
- First Order Logic
- Vingean Reflection
- Corrigibility
- Linear Algebra
- Topology
- Category Theory
- Type Theory
Even baring a full blown test, hints like (I'm going to make stuff up) "If you can prove how Solomonoff Induction across a transfinite ordinal ensures Turing-completeness, you're on the right track"