= 27d567bc2d48d9f40e9ccc20ea370b15)
Jack comments on Friendly AI Research and Taskification - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (44)
Why do we need to know about meta-ethics? Why would we even think there is such a thing as human meta-ethics?
Meta-ethics may be relevant for crafting ways to approach the problem. But I don't see how it is a descriptive project answerable with the techniques of cognitive neuroscience the way you describe. I also don't see why an AI would need to know whether we were cognitivists or non-cognitivists to know how we want it to act.
I can think of a way it could matter. If we're non-cognitivists, we think our moral discourse consists of speech acts, so when we argue over morality we're actually issuing imperatives, or declaring universal imperatives exist. If we're cognitivists, we're talking about what is the case. There could very well be grounds for a FAI to treat an imperative and a proposition differently. Doesn't mean the distinction actually matters, but it's not obviously irrelevant.
So it would be sloppy to code what you want an AI to do the way you code propositions/beliefs. That is, you don't want to fit the bulk of the goal architecture inside it's belief networks. Nor certainly, should you expect the AI to learn moral truths by looking at the world. Once you tell it to care about what people want, then it can look at people to find that out- but it can't learn to care about what people want just by observing the world. Those kind of moral facts don't exist. So certainly knowing things about meta-ethics will help create an FAI.
But that's an argument for smart people to spend time thinking about meta-ethics. It's not an argument for a descriptive program that finds folk-metaethics to form the the goal architecture of an AI. For one thing, most humans seem to have really confused meta-ethical beliefs.
On reflection, I think by 'metaethical thought' Mitchell probably meant the normative theory that describes human ethics. I don't think there is one of those either, but it's not obviously wrong and certainly makes more sense.
I meant: innate cognitive architecture which plays a role in metaethical thought.
You might be familiar with the idea that, according to CEV, you figure out the full complexity of human value using neuroscience (rather than relying on people's opinions about what they value), and then you "extrapolate" or "renormalize" that using "reflective decision theory" (which does not yet exist). The idea here is that the method of extrapolation should also be extracted from the details of human cognitive architecture, rather than just figured out through intuition or pure reason.
Suppose we have a person - or an intelligent agent - with a particular "value system" or "private decision theory". Note that we are talking about its actual decision theory, as embodied in its causal structure and decision-making dispositions, and not just its introspective opinions about how it decides. Given this actual value system, RDT is supposed to tell us what would happen to that value system if it were changed according to its own implicit ideals. All I'm saying is that there's a meta-ethical relativism for RDT, for large classes of decision architecture. Different theories about how to normatively self-modify a decision architecture ought to be possible, and the selection of which RDT is used should also be derived from the agent's own cognitive architecture.
Of course you can go meta again and say, maybe the RDT extraction procedure can also take different forms - etc. It's one of the tasks of the FAI/CEV/RDT research program to figure out when and how the ethical metalevels stop.