The TLGH knows what the morally right thing to do is -- it's hardwired. Figuring out morality is harder when you don't already have a moral arrow preset for you.
Knowledge requires justificaiton. A TLGH that understands epssiemology wouild see itslef as not knowing its TLG, since "it was hardwired into me" is no justification. This applies to humans: we are capable of dounting that our evolutionarily derived moral attitudes arre the correct ones.
Evolutionary psychology tells us that our evolutionary history has given us certain moral attitudes and behaviour. So far, so good. Some scientifically minded types take this to constitute a theory of objective morality all in itself. However, that would be subject to the Open Question objection: we can ask of our inherited morality whether it is actually right. (Unrelatedly, we are probably not determined to follow it, since we can overcome strong evolutionary imperatives in, for instance, voluntay celibacy). This not a merely abstract issue either, since EP has been used to support some contentious claims; for instance, that men should be forgiven for adultery since it is "in their genes" to seek muliple partners.
Any wondering about whether a TLGH did the right thing, by itself, will be rational-ought, not moral-ought.
And if there is any kind of objective truth about which goals are the true top level goals, that is going to have to come from reasoning. Emipricism fails because there are no perceivable moral facts, and ordinary facts fall into the is-ought divide.
Rationality is probably better at removing goals than setting them, better at thous-shalt-nots than thou-shalts That is in line with the liberal-secular view of morality, where it would be strange and maybe even obnoxious for everyone to be pursuing the same aim.
Knowledge requires justificaiton. A TLGH that understands epssiemology wouild see itslef as not knowing its TLG, since "it was hardwired into me" is no justification. This applies to humans: we are capable of dounting that our evolutionarily derived moral attitudes arre the correct ones.
This only applies to humans because we are not TLGHs. Beliefs and goals require justification because we might change them. Beliefs and goals which are hardwired do not require justification; they must be taken as given. As far as I'm aware, humans only ever...
On the face of it, there is a tension in adhering both to the idea that there are facts about what it's rational for people to do and to the idea that natural or scientific facts are all the facts there are. The aim of this post is just to try to make clear why this should be so, and hopefully to get feedback on what people think of the tension.
In short
To a first approximation, a belief is rational just in case you ought to hold it; an action rational just in case you ought to take it. A person is rational to the extent that she believes and does what she ought to. Being rational, it is fair to say, is a normative or prescriptive property, as opposed to a merely descriptive one. Natural science, on the other hand, is concerned merely with descriptive properties of things -what they weigh, how they are composed, how they move, and so on. On the face of it, being rational is not the sort of property about which we can theorize scientifically (that is, in the vocabulary of the natural sciences). To put the point another way, rationality concerns what a thing (agent) ought to do, natural science concerns only what it is and will do, and one cannot deduce 'ought' from 'is'.
At greater length
There are at least two is/ought problems, or maybe two ways of thinking about the is/ought problem. The first problem (or way of thinking about the one problem) is posed from a subjective point of view. I am aware that things are a certain way, and that I am disposed to take some course of action, but neither of these things implies that I ought to take any course of action -neither, that is, implies that taking a given course of action would in any sense be right. How do I justify the thought that any given action is the one I ought to take? Or, taking the thought one step further, how, attending only to my own thoughts, do I differentiate merely being inclined to do something from being bound by some kind of rule or principle or norm, to do something?
This is an interesting question -one which gets to the very core of the concept of being justified, and hence of being rational (rational beliefs being justified beliefs). But it isn't the problem of interest here.
The second problem, the problem of interest, is evident from a purely objective, scientific point of view. Consider a lowly rock. By empirical investigation, we can learn its mass, its density, its mineralogical composition, and any number of other properties. Now, left to their own devices, rocks don't do much of anything, comparatively speaking, so it isn't surprising that we don't expect there to be anything it ought to do. In any case, natural science does not imply there is anything it ought to do, I think most will agree.
Consider then a virus particle - a complex of RNA and ancillary molecules. Natural science can tell us how it wiil behave in various circumstances -whether and how it will replicate itself, and so on- but once again surely there is nothing in biochemistry, genetics or other science which implies there is anything our very particle ought to do. It's true that we may think of it as having the goal to replicate itself, and consider it to have made a mistake if it replicates itself inaccurately, but these conceptions do not issue from science. Any sense in which it ought to do something, or is wrong or mistaken in acting in a given way, is surely purely metaphorical (no?).
How about a bacterium? It's orders of magnitude more complicated, but I don't see that matters are any different as regards what it ought to do. Science has nothing to tell us about what if anything is important to a bacterium, as distinct from what it will tend to do.
Moving up the evolutionary ladder, does the introduction of nervous systems make any difference? What do we think about, say, nematodes or even horseshoe crabs? The feedback mechanisms underlying the self-regulatory processes in such animals may be leaps and bounds more sophisticated than in their non-neural forebears, but it's far from clear how such increasing complexity could introduce goals.
To cut to the chase, how can matters be any different with the members of Homo sapiens ? Looked at from a properly scientific point of view, is there any scope for the attribution of purposes or goals or the appraisal of our behaviour in any sense as right or wrong? I submit that a mere increase in complexity -even if by many orders of magnitude- does not turn the trick. To be clear, I'm not claiming there are no such facts -far from it- just that these facts cannot be articulated in the language of purely natural science.
The upshot
The foregoing thoughts are hardly original. David Hume is famous for having observed that ought cannot be derived from is:
(the issue, together with this quote are touched on with a different point of view in this post of lukeprog's). I think they need facing up to. I see three options:
Option (1): Accept the basic point, stick resolutely to naturalism, and deny that there are any facts as to what it is rational for any given member of Homo sapiens to do. In other words, become an eliminativist about all normative concepts. Paul Churchland, I understand, is an example of an exponent of this position (or something more nuanced along these lines).
Option (2): Reject the argument above on one or another ground. Try somehow to shoehorn normative facts into a naturalistic world-view, at the possible peril of the coherence of that world-view. I acknowledge that this can be a valiant undertaking for those whose commitments suggest it. Any who embark on it should be aware that there is at least a half a century's worth of beleaguered efforts to put this Humpty together -it is not an easy task. One might want to start with the likes of Ruth Garrett Millikan or Jerry Fodor, then their various critics.
Option (3): Accept the argument above and reconcile yourself to the existence of mutually incommensurable but indispensable understandings of yourself. Be happy.
One response
This is already ponderously long, but there is one response worth anticipating, namely, that oughts can be inferred from wants (or preferences or utility functions), and that wants (preferences, utility functions) are naturalistic. The problem here is with the second conjunct - wants (etc.) are not naturalistic. At least, not obviously so (and, incidentally, the same fate befalls beliefs). My explanation is as follows.
An example of the thinking behind this proposal presumably would be something like,
P) X's wanting that X eats an apple entails, other things being equal, that X ought to eat an apple.
The force of naturalism or physicalism in this context is presumably a commitment to some empirically testable analysis concerning wants comparable to "Water is H2O", e.g.
or
or ...
Now, if both of these thoughts (the thought about wants entailing oughts and the thought about there being an empirically testable analysis) are correct, then it should be possible to substitute the analysis into (P):
P') That X's brain is in state ABC entails, other things being equal, that X ought to eat an apple.
or
P'') That X is composed in part of some structure which implements a Turing machine which is in computational state DEF entails, other things being equal, that X ought to eat an apple.
or... (your favourite theory here)
I submit that neither P' nor P'' is at all plausible, for the reasons reviewed above. A thing's merely being in a certain physical or computational state does not imbue it with purpose. Such facts do not entail that there is anything which matters to it, or which makes anything right or wrong for the thing. Concerning P'', note that although computers are sometimes thought of as having purposes in virtue of our having designed them (the computer 'makes a mistake' when it calculates an incorrect value), there is not normally thought to be any sense in which they have intrinsic purposes independent of ours, as the view under scrutiny would require of them.
There are all kinds of possible refinements of P' and P'' (P' and P'' are commonly viewed as non-starters anyway, owing to the plausibility of so-called semantic externalism which they ignore). My question is whether any more refined option shows a way to defeat the objection being raised here.
Wants do indeed imply oughts. Since there plausibly is no physical or computational state being in which implies there is anything one ought to do, wanting is not identical to being in a physical or computational state.