Thanks for engaging with me.

1. The heuristics that we use are too numerous and of too much complexity to be possible to explicitly justify all of them. Turning your mathematics analogy on its head, note that mathematicians have very little knowledge of the heuristics that they use to discover and prove theorems. Poincare wrote some articles about this; if interested see The Value of Science.

2. There are over a million charities in the US alone. GiveWell currently has (around) 5 full time staff. If GiveWell were to investigate every charity this year. each staff member would have to investigate over 500 charities per day. Moreover, doing comparison of even two charities can be exceedingly tricky. I spent ~ 10 hours a week for five months investigating the cost effectiveness of school based deworming and I still don't know whether it's a better investment than bednets. So I strongly disagree that GiveWell shouldn't use time saving heuristics.

3. As for for SIAI vs. VillageReach, it may well be that SIAI is a better fit for your values than VillageReach is. I currently believe that donating to SIAI has higher utilitarian expected value than donating to VillagReach but also presently believe that a few years of searching will yield a charity at least twice as cost-effective than either at the margin. I have been long been hoping for GiveWell to research x-risk charities. See my comment here. Over the next year I'll be researching x-risk reduction charities myself.

4. It's not clear to me that overcoming a generic bias should improve one's rationality on average. This is an empirical question with no data but anecdotal evidence. Placebo effect and selection bias may suffice to explain a subjective sense that overcoming biases is conducive to rationality. Anyway, on the matter at hand, I concur with Holden's view that relying entirely on explicit formulas does not maximize expected value and that one should incorporate some measure of subjective judgment (as to how much, I am undecided).