P(Luminosity fan | reads this comment) is probably not a good estimate... (count me in with a "no" data point though :)) Also, what is the ratio of "Luminosity fan because of Twilight" and "read it even though... Twilight, and liked it" populations?

(with "read Twilight because of Luminosity" also a valid case.)

Also, who is the target audience and what are the plans for reaching it? I don't think there are many people who are willing to invest time AND money into a book like this while still not having read the sequences (available freely on the web, and also in all kinds of e-book formats).

For the two use cases I imagine at the moment:

• giving it as a gift as an introduction to rationalist stuff feels better with a physical book indeed. Yes, there is a difference between buying an e-book for yourself and downloading the same stuff for free, especially in terms of motivation to actually read it, but on the receiving end e-books still might feel like being sent long pdf-s with a label "you should definitely read this", in addition to the e-book gifting weirdness (I might be wrong, I never did such a thing before).

• buying it for yourself, to be able to put it on your bookshelf. Obviously, also much harder to do with an e-book.

(I usually prefer e-books to dead-tree versions, but then I had nothing against reading the Sequences on the web either.)

The point is that it's not, but making it so is a design goal of the paper.

Example: Mario immediately jumping into a pit at level 2. According to the learned utility function of the system, it's a good idea. According to ours, it's not.

Just as with optimizing smiling faces. But while that one was purely a thought experiment, this paper presents a practical, experimentally testable benchmark for utility function learning, and, by the way, shows a not-yet-perfect but working solution for it. (After all, Mario's Flying Goomba Kick of High Munchkinry definitely satisfies our utility functions.)

I heard the opposite too: don't try to push your own research too hard, especially in the beginning, but try to find something the others in the lab group are working on, learn stuff from them, and after a while you'll end up with your own ideas anyway.

Pros and cons for both of the approaches exist, but "picking a thesis early on" might be hard as you don't necessarily know what the good problems are in your field. But that might depend on your field / advisor too.

You see that you won't get stuck: that you'll finish relatively fast.

Do you know of a way for estimating this? Every research problem (and the entire PhD itself) might look much easier before you start working on it (you don't have an outside view perspective before starting).

Also, Tom's academic website. It's the coolest academic website I've ever seen.

In the class I TA for, the students can go to the professor's office hours after the midterm / final, and if they can solve the problem there, they still get... half of the points? I wonder how that one affects test-taking performance.

Also, this whole thing seems to be annoyingly resistant to Bayesian updates... "Every time I'm anxious I perform bad, and now I'm worried about being too worried for this exam", and, since performing bad is a very valid prediction in this state of mind, worry is there to stay.

Maybe if the tests are called "quizzes" the students end up in the other stable state of "not being worried"?

Do we need a realistic simulation at all? I was thinking about how educational games could devolve into, instead of "guessing the teacher's password", "guessing the model of the game"... but is this a bad thing?

Sure, games about physics should be able to present a reasonably accurate model so that if you understand their model, you end up knowing something about physics... but with history:

actually, what's the goal of studying history?

• if the goal is to do well on tests, we already have a nice model for that, under the name of Anki. Of course, this doesn't make things really fun, but still.
• if we want to make students remember what happened and approximately why (that is, "should be able to write an essay about it"), we can make up an arbitrary, dumb and scripted thing, not even close to a real model, but exhibiting some mechanics that cover the actual reasons. (e.g. if one of the causes would have been "not enough well-trained soldiers", then make "Level 8 Advanced Phalanx" the thing to build if you want to survive the next wave of attacks.)
• if we'd like to see students discover general ideas throughout history, maybe build a game with the same mechanics across multiple levels? (and they also don't need to be really accurate or realistic.)
• and finally, if we want to train historians who could come up with new theories, or replacement emperors to be sent back in time to fix Rome... well, for that we would need a much better model indeed. Which we are unlikely to end up with. But do we need this level in most of the cases?

TL;DR by creating games with wildly unrealistic but textbook-accurate mechanics we are unlikely to train good emperors, but at least students would understand textbook things much more than the current "study, exam, forget" level.