On the edge of sleep I hypothesized that an average human could build an FAI if they were given lots of time and paper to write down a very long chain of "Why do I believe that?"s. I then realized that such an endeavor would very quickly end in an endless loop of infinitesimal utility and a lot of wasted trees. Moral: rationality is a lot harder than the application of a few heuristics. Metamoral: Applying that heuristic doesn't make it much easier.