I want to point out these concluding paragraphs from Greene's "The secret joke of Kant's soul":

Taking these arguments seriously, however, threatens to put us on a second slippery slope (in addition to the one leading to altruistic destitution): How far can the empirical debunking of human moral nature go? If science tells me that I love my children more than other children only because they share my genes (Hamilton, 1964), should I feel uneasy about loving them extra? If science tells me that I am nice to other people only because a disposition to be nice ultimately helped my ancestors spread their genes (Trivers, 1971), should I stop being nice to people? If I care about myself only because I am biologically programmed to carry my genes into the future, should I stop caring about myself? It seems that one who is unwilling to act on human tendencies that have amoral evolutionary causes is ultimately unwilling to be human. Where does one draw the line between correcting the nearsightedness of human moral nature and obliterating it completely?

This, I believe, is among the most fundamental moral questions we face in an age of growing scientific self-knowledge, and I will not attempt to address it here. Elsewhere I argue that consequentialist principles, while not true, provide the best available standard for public decision making and for determining which aspects of human nature it is reasonable to try to change and which ones we would be wise to leave alone (Greene, 2002; Greene & Cohen, 2004).

In Greene's 2002 Ph.D. thesis, he defends consequentialism but not very strongly:

We’re just looking for a practical guideline, not an eternal moral code that handles fantasy cases as well as real ones or that tells us which among our reasons for action are philosophically privileged.

Unfortunately, for the purpose of building FAI, we are looking for an "eternal moral code".

ETA: I'm having trouble finding where Greene addresses the issue of how consequentialism handles this "slippery slope" problem. Can anyone point me to a page number, or perhaps have independent arguments for why consequentialism is less vulnerable to "growing scientific self-knowledge" than deontology?