This article was originally on the FHI wiki and is being reposted to LW Discussion with permission. All content in this article is credited to Daniel Dewey.
In value loading, the agent will pick the action:
 \sum_{u \in U} u(w)p(C(u)|w))
Here A is the set of actions the agent can take, e is the evidence the agent has already seen, W is the set of possible worlds, and U is the set of utility functions the agent is considering.
The parameter C(u) is some measure of the 'correctness' of the utility u, so the term p(C(u)|w) is the probability of u being correct, given that the agent is in world w. A simple example is of an AI that... (read 330 more words →)
I re-read Atlas Shrugged once or twice a year. One of my first posts on LW was this (and you even commented on it!):
https://www.lesswrong.com/posts/7s5gYi7EagfkzvLp8/in-defense-of-ayn-rand
Not necessarily proud of it, but it's interesting to re-read it after fully reconciling the book with my own internal principles. I can see how much I struggled with the fact that I really resonated with the idea of hero-worship, while also feeling so fragile in my own judgments, simultaneously. It really is a wonderful book, and I no longer feel the need to defend anything about it - I just get a little sad when it gets brushed off (the lord of the rings comparison joke really gets me), as an honest reading will always reveal something fundamental, even in criticism.