I don't see the theism/atheism debate as a policy debate. There is a factual question underlying it, and that factual question is "does God exist?" I find it very hard to imagine a universe where the answer to that question is neither 'yes' nor 'no'.
Scenario:
1) You wake up in a bright box of light, no memories. You are told you'll presently be born into an Absolute monarchy, your role randomly chosen. You may choose any moral principles that should govern that society. The Categorical Imperative would on average give you the best result.
2) You are the monarch in that society, you do not need to guess which role you're being born into, you have that information. You don't need to make all the slaves happy to help your goals, you can just maximize your goals directly. You may choose any moral principle you want to govern your actions. The Categorical Imperative would not give you the best result.
A different scenario: Clippy and Anti-Clippy sit in a room. Why can they not agree on epistemic facts about the most accurate laws of physics and other Aumann-mandated agreements, yet then go out and each optimize/reshape the world according to their own goals? Why would that make them not rational?
Lastly, whatever Kant's justification, why can you not optimize for a different principle - peak happiness versus average happiness, what makes any particular justifying principle correct across all - rational - agents. Here come my algae!
You are the monarch in that society, you do not need to guess which role you're being born into, you have that information. You don't need to make all the slaves happy to help your goals, you can just maximize your goals directly. You may choose any moral principle you want to govern your actions. The Categorical Imperative would not give you the best result.
For what value of "best"? If the CI is the correct theory of morality, it will necessarily give your the morally best result. Maybe your complaint is that it wouldn't maximise your personally utility. But I don't see why you would expect that. Things like utilitarianism that seek to maximise group utility, don't promise to make everyone blissfully happy individually. Some will lose out.
A different scenario: Clippy and Anti-Clippy sit in a room. Why can they not agree on epistemic facts about the most accurate laws of physics and other Aumann-mandated agreements, yet then go out and each optimize/reshape the world according to their own goals? Why would that make them not rational?
It would be irrational for Clippy to sing up to an agreement with Beady according to which Beady gets to turn Clippy and all his clips into beads. It is irrational for agents to sign up to anyhting which is not in their interests, and it is not in their interests to have no contract at all. So rational agents, even if they do not converge on all their goals, will negotiate contracts that minimise their disutility Clippy and Beady might take half the universe each.
Lastly, whatever Kant's justification, why can you not optimize for a different principle - peak happiness versus average happiness, what makes any particular justifying principle correct across all - rational - agents.
If you think RAs can converge on an ultimately correct theory of physics (which we don't have), what is to stop them converging on the correct theory of morality, which we also don't have?
First of all, thanks for the comment. You have really motivated me to read and think about this more -- starting with getting clearer on the meanings of "objective", "subjective", and "intrinsic". I apologise for any confusion caused by my incorrect use of terminology. I guess that is why Eliezer likes to taboo words. I hope you don't mind me persisting in trying to explain my view and using those "taboo" words.
Since I was talking about meta-ethical moral relativism, I hope that it was sufficiently clear that I was referring to moral values. What I meant by "objective values" was "objectively true moral values" or "objectively true intrinsic values".
The second sentence doesn't follow from the first.
The second sentence was an explanation of the first: not logically derived from the first sentence, but a part of the argument. I'll try to construct my arguments more linearly in future.
If I had to rephrase that passage I'd say:
If there are no agents to value something, intrinsically or extrinsically, then there is also nothing to act on those values. In the absence of agents to act, values are effectively meaningless. Therefore, I'm not convinced that there is objective truth in intrinsic or moral values.
However, the lack of meaningful values in the absence of agents hints at agents themselves being valuable. If value can only have meaning in the presence of an agent, then that agent probably has, at the very least, extrinsic/instrumental value. Even a paperclip maximiser would probably consider itself to have instrumental value, right?
If rational agents converge on their values, that is objective enough.
I think there is a difference between it being objectively true that, in certain circumstances, the values of rational agents converge, and it being objectively true that those values are moral. A rational agent can do really "bad" things if the beliefs and intrinsic values on which it is acting are "bad". Why else would anyone be scared of AI?
Analogy: one can accept that mathematical truth is objective (mathematicians will converge) without being a Platonists (mathematical truths have an existence separate from humans)
I accept the possibility of objective truth values. I'm not convinced that it is objectively true that the convergence of subjectively true moral values indicates objectively true moral values. As far as values go, moral values don't seem to be as amenable to rigorous proofs as formal mathematical theorems. We could say that intrinsic values seem to be analogous to mathematical axioms.
I fin d that hard to follow. If the test i rationally justifiable, and leads to the uniform results, how is that not objective?
I'll have a go at clarifying that passage with the right(?) terminology:
Without the objective truth of intrinsic values, it might just be a matter of testing different sets of assumed intrinsic values until we find an "optimal" or acceptable convergent outcome.
Morality might be somewhat like an NP-hard optimisation problem. It might be objectively true that we get a certain result from a test. It's more difficult to say that it is objectively true that we have solved a complex optimisation problem.
You seem to be using "objective" (having a truth value independent of individual humans) to mean what I would mean by "real" (having existence independent of humans).
Thanks for informing me that my use of the term "objective" was confused/confusing. I'll keep trying to improve the clarity of my communication and understanding of the terminology.
First of all, thanks for the comment. You have really motivated me to read and think about this more
That's what I like to hear!
If there are no agents to value something, intrinsically or extrinsically, then there is also nothing to act on those values. In the absence of agents to act, values are effectively meaningless. Therefore, I'm not convinced that there is objective truth in intrinsic or moral values.
But there is no need for morality in the absence of agents. When agents are there, values will be there, when agents are not there, the absence of values doesn't matter.
I think there is a difference between it being objectively true that, in certain circumstances, the values of rational agents converge, and it being objectively true that those values are moral. A rational agent can do really "bad" things if the beliefs and intrinsic values on which it is acting are "bad". Why else would anyone be scared of AI?
I don't require their values to converge, I require them to accept the truths of certain claims. This happens in real life. People say "I don't like X, but I respect your right to do it". The first part says X is a disvalue, the second is an override coming from rationality.
I'm not disputing that there are goals/ethics which may be best suited to take humanity along a certain trajectory, towards a previously defined goal (space exploration!). Given a different predefined goal, the optimal path there would often be different. Say, ruthless exploitation may have certain advantages in empire building, under certain circumstances.
The Categorical Imperative in all its variants may be a decent system for humans (not that anyone really uses it).
But is the justification for its global applicability that "if everyone lived by that rule, average happiness would be maximized"? That (or any other such consideration) itself is not a mandatory goal, but a chosen one. Choosing different criteria to maximize (e.g. noone less happy than x) would yield different rules, e.g. different from the Categorical Imperative. If you find yourself to be the worshipped god-king in some ancient Mesopotanian culture, there may be many more effective ways to make yourself happy, other than the Categorical Imperative. How can it still be said to be "correct"/optimal for the king, then?
So I'm not saying there aren't useful ethical system (as judged in relation to some predefined course), but that because those various ultimate goals of various rational agents (happiness, paperclips, replicating yourself all over the universe) and associated optimal ethics vary, there cannot be one system that optimizes for all conceivable goals.
My argument against moral realism and assorted is that if you had an axiomatic system from which it followed that strawberry is the best flavor of ice cream, but other agents which are just as intelligent with just as much optimizing power could use different axiomatic systems leading to different conclusions, how could one such system possibly be taken to be globally correct and compelling-to-adopt across agents with different goals?
Gandhi wouldn't take a pill which may transform him into a murderer. Clippy would not willingly modify itself such that suddenly it had different goals. Once you've taken a rational agent apart and know its goals and, as a component, its ethical subroutines, there is no further "core spark" which really yearns to adopt the Categorical Imperative. Clippy may choose to use it, for a time, if it serves its ultimate goals. But any given ethical code will never be optimal for arbitrary goals, in perpetuity (proof by example). When then would a particular code following from particular axioms be adopted by all rational agents?
But is the justification for its global applicability that "if everyone lived by that rule, average happiness would be maximized"?
Well, not, that's not Kant's justification!
That (or any other such consideration) itself is not a mandatory goal, but a chosen one.
Why would a rational agent choose unhappiness?
If you find yourself to be the worshipped god-king in some ancient Mesopotanian culture, there may be many more effective ways to make yourself happy, other than the Categorical Imperative.
Yes, but that wouldn't count as ethics. You wouldn't want a Universal Law that one guy gets the harem, and everyone else is a slave, because you wouldn't want to be a slave, and you probably would be. This is brought out in Rawls' version of Kantian ethics: you pretend to yourself that you are behind a veil that prevents you knowing what role in society you are going to have, and choose rules that you would want to have if you were to enter society at random.
My argument against moral realism and assorted is that if you had an axiomatic system from which it followed that strawberry is the best flavor of ice cream, but other agents which are just as intelligent with just as much optimizing power could use different axiomatic systems leading to different conclusions,
You don't have object-level stuff like ice cream or paperclips in your axioms (maxims), you have abstract stuff, like the Categorical Imperative. You then arrive at object level ethics by plugging in details of actual circumstances and values. These will vary, but not in an arbitrary way, as is the disadvantage of anything-goes relativism.
how could one such system possibly be taken to be globally correct and compelling-to-adopt across agents with different goals?
The idea is that things like the CI have rational appeal.
Once you've taken a rational agent apart and know its goals and, as a component, its ethical subroutines, there is no further "core spark" which really yearns to adopt the Categorical Imperative.
Rational agents will converge on a number of things because they are rational. None of them will think 2+2-=5.
Yea, honestly I've never seen the exact distinction between goals which have an ethics-rating, and goals which do not. I understand that humans share many ethical intuitions, which isn't surprising given our similar hardware. Also, that it may be possible to define some axioms for "medieval Han Chinese ethics" (or some subset thereof), and then say we have an objectively correct model of their specific ethical code. About the shared intuitions amongst most humans, those could be e.g. "murdering your parents is wrong" (not even "murder is wrong", since that varies across cultures and circumstances). I'd still call those systems different, just as different cars can have the same type of engine.
Also, I understand that different alien cultures, using different "ethical axioms", or whatever they base their goals on, do not invalidate the medieval Han Chinese axioms, they merely use different ones.
My problem with "objectively correct ethics for all rational agents" is, you could say, where the compellingness of any particular system comes in. There is reason to believe an agent such as Clippy could not exist (edit: i.e., it probably could exist), and its very existence would contradict some "'rational' corresponds to a fixed set of ethics" rule. If someone would say "well, Clippy isn't really rational then", that would just be torturously warping the definition of "rational actor" to "must also believe in some specific set of ethical rules".
If I remember correctly, you say at least for humans there is a common ethical basis which we should adopt (correct me otherwise). I guess I see more variance and differences where you see common elements, especially going in the future. Should some bionically enhanced human, or an upload on a spacestation which doesn't even have parents, still share all the same rules for "good" and "bad" as an Amazon tribe living in an enclosed reservation? "Human civilization" is more of a loose umbrella term, and while there certainly can be general principles which some still share, I doubt there's that much in common in the ethical codex of an African child soldier and Donald Trump.
Yea, honestly I've never seen the exact distinction between goals which have an ethics-rating, and goals which do not
A number of criteria have been put forward. For instance, do as you would be done by. If you don't want to be murdered, murder is not an ethical goal.
My problem with "objectively correct ethics for all rational agents" is, you could say, where the compellingness of any particular system comes in. There is reason to believe an agent such as Clippy could exist, and its very existence would contradict some "'rational' corresponds to a fixed set of ethics" rule. If someone would say "well, Clippy isn't really rational then", that would just be torturously warping the definition of "rational actor" to "must also believe in some specific set of ethical rules".
The argument is not that rational agents (for some vaue of "rational") must believe in some rules, it is rather that they must not adopt arbitrary goals. Also, the argument only requires a statistical majority of rational agents to converge, because of the P<1.0 thing.
Should some bionically enhanced human, or an upload on a spacestation which doesn't even have parents, still share all the same rules for "good" and "bad" as an Amazon tribe living in an enclosed reservation?
Maybe not. The important thing is that variations in ethics should not be arbitrary--they should be systematically related to variations in circumstances.
But claims about transfinities don't correspond directly to any object. Maths is "spun off" from other facts, on your view. So, by analogy, moral realism could be "spun off" without needing any Form of the Good to correspond to goodness.
Spun off from what, and how?
You seem to be assumig that morality is about individual behaviour. A moral realist system like utiitarianism operates at the group level, and woud take paperclipper values into account along with all others. Utilitarianism doens't care what values are, it just sums or averages them.
Speaking as a utilitarian, yes, utilitarianism does care about what values are. If I value paperclips, I assign utility to paperclips, if I don't, I don't.
Or perhaps you are making the objection that an entity woud need moral values to care about the preferences of others in the first place. That is addressed by, another kind of realism, the rationality-based kind, which starts from noting that rational agents have to have some value in common, because they are all rational.
Why does their being rational demand that they have values in common? Being rational means that they necessarily share a common process, namely rationality, but that process can be used to optimize many different, mutually contradictory things. Why should their values converge?
I was talking about rational minds converging on the moral claims, not on values.. Rational minds can converge on "maximise group utility" whilst what is utilitous varies considerably.
So what if a paperclipper arrives at "maximize group utility," and the only relevant member of the group which shares its conception of utility is itself, and its only basis for measuring utility is paperclips? The fact that it shares the principle of maximizing utility doesn't demand any overlap of end-goal with other utility maximizers.
Axioms are formal statements, intuitions are gut feelings tha are often used to justify axioms.
But, as I've pointed out previously, intuitions are often unhelpful, or even actively misleading, with respect to locating the truth.
If our axioms are grounded in our intuitions, then entities which don't share our intuitions will not share our axioms.
So do they call for them to be fired?
No, but neither do I, so I don't see why that's relevant.
Spun off from what, and how?
I am not sure I can expalin that succintly at the moment. It is also hard to summarise how you get from counting apples to transfinite numbers.
Why does their being rational demand that they have values in common? Being rational means that they necessarily share a common process, namely rationality, but that process can be used to optimize many different, mutually contradictory things. Why should their values converge?
Rationality is not an automatic process, it is skill that has to be learnt and consciously applied. Individuals will only be rational if their values prompt them to. And rationality itself implies valuing certain things (lack of bias, non arbitrariness).
So what if a paperclipper arrives at "maximize group utility," and the only relevant member of the group which shares its conception of utility is itself, and its only basis for measuring utility is paperclips? The fact that it shares the principle of maximizing utility doesn't demand any overlap of end-goal with other utility maximizers.
Utilitarians want to maximise the utiity of their groups, not their own utility. They don;t have to believe the utlity of others is utilitous to them, they just need to feed facts about group utility into an aggregation function. And, using the same facts and same function, different utilitarians will converge. That's kind of the point.
But, as I've pointed out previously, intuitions are often unhelpful, or even actively misleading, with respect to locating the truth.
Compared to what? Remember, I am talking about foundational intuitions, the kind at the bottom of the stack. The empirical method of locating the truth rests on the intuition that the senses reveal a real external world. Which I share. But what proves it? That's the foundational issue.
Mathematics isn't physics. Mathematicians prove theorems from axioms, not from experiments.
Yes, but the fact that the universe itself seems to adhere to the logical systems by which we construct mathematics gives credence to the idea that the logical systems are fundamental, something we've discovered rather than producing. We judge claims about nonobserved mathematical constructs like transfinites according to those systems,
Metaethical systems usually have axioms like "Maximising utility is good".
But utility is a function of values. A paperclipper will produce utility according to different values than a human.
I am not sure what you mean by "exist" here. Claims are objectively true if most rational minds converge on them. That doesn't require Objective Truth to float about in space here.
Why would most rational minds converge on values? Most human minds converge on some values, but we share almost all our evolutionary history and brain structure. The fact that most humans converge on certain values is no more indicative of rational minds in general doing so than the fact that most humans have two hands is indicative of most possible intelligent species converging on having two hands.
Does that mean we can;t use moral intuitions at all, or that they must be used with caution?
It means we should be aware of what our intuitions are and what they've developed to be good for. Intuitions are evolved heuristics, not a priori truth generators.
Philosphers talk about intuitions, because that is the term for something foundational that seems true, but can't be justified by anything more foundational. LessWrongians don't like intuitions, but don't see to be able to explain how to manage without them.
It seems like you're equating intuitions with axioms here. We can (and should) recognize that our intuitions are frequently unhelpful at guiding us to he truth, without throwing out all axioms.
Did you post any comments explaining to the professional philosophers where they had gone wrong?
If I did, I don't remember them. I may have, I may have felt someone else adequately addressed them, I may not have felt it was worth the bother.
It seems to me that you're trying to foist onto me the effort of locating something which you were the one to testify was there in the first place.
I don;'t see the problem. Philosophical competence is largely about understanding the problem.
And philosophers frequently fall into the pattern of believing that other philosophers disagree with each other due to failure to understand the problems they're dealing with.
In any case, I reject the notion that dismissing large contingents of philosophers as lacking in competence is a valuable piece of evidence with respect t crankishness, and if you want to convince me that I am taking a crankish attitude, you'll need to offer some other evidence.
Yes, but the fact that the universe itself seems to adhere to the logical systems by which we construct mathematics gives credence to the idea that the logical systems are fundamental, something we've discovered rather than producing. We judge claims about nonobserved mathematical constructs like transfinites according to those systems,
But claims about transfinities don't correspond directly to any object. Maths is "spun off" from other facts, on your view. So, by analogy, moral realism could be "spun off" without needing any Form of the Good to correspond to goodness.
Metaethical systems usually have axioms like "Maximising utility is good".
But utility is a function of values. A paperclipper will produce utility according to different values than a human.
You seem to be assumig that morality is about individual behaviour. A moral realist system like utiitarianism operates at the group level, and woud take paperclipper values into account along with all others. Utilitarianism doens't care what values are, it just sums or averages them.
Or perhaps you are making the objection that an entity woud need moral values to care about the preferences of others in the first place. That is addressed by, another kind of realism, the rationality-based kind, which starts from noting that rational agents have to have some value in common, because they are all rational.
Why would most rational minds converge on values?
a) they don't have to converge on preferences, since thing like utilitariansim are preference-neutral.
b) they already have to some extent because they are rational
Most human minds converge on some values, but we share almost all our evolutionary history and brain structure. The fact that most humans converge on certain values is no more indicative of rational minds in general doing so than the fact that most humans have two hands is indicative of most possible intelligent species converging on having two hands.
I was talking about rational minds converging on the moral claims, not on values.. Rational minds can converge on "maximise group utility" whilst what is utilitous varies considerably.
Philosphers talk about intuitions, because that is the term for something foundational that seems true, but can't be justified by anything more foundational. LessWrongians don't like intuitions, but don't see to be able to explain how to manage without them.
It seems like you're equating intuitions with axioms here.
Axioms are formal statements, intuitions are gut feelings tha are often used to justify axioms.
We can (and should) recognize that our intuitions are frequently unhelpful at guiding us to he truth, without throwing out all axioms.
There is another sense of "intuition" where someone feels that it's going to rain tomorrow or something. They're not the foundational kind.
And philosophers frequently fall into the pattern of believing that other philosophers disagree with each other due to failure to understand the problems they're dealing with.
So do they call for them to be fired?
In real life you sometimes get people who write, using different words, under dozen different articles: "I suspect that this all is just Eliezer's cult designed to extract money from naive people". How much of that is acceptable criticism, and how much is just annoying? Discussing that thing once, thoroughly? Yes, definitely. Dropping the idea around all the time? That's just poluting the space. Problem is that at the moment some people are already deeply annoyed, other people go meta and say we need criticism.
Democracy does not work well online. In real life, one person cannot be at more than one place. Online, one person is enough to be everywhere (within one website). In real life, you can avoid an annoying person by simply going elsewhere and taking your friends with you. Online, you must somehow stop people from doing annoying things, otherwise there is no way to avoid them, except by avoiding the whole website.
I don't have a problem with criticism. I have a problem with boring repetitive criticism. Someone says that having a ceremony is cultish. Okay. Let's discuss that. Someone says again that having a ceremony is cultish. Okay, here is some explanation, here are the differences. Someone says yet again that having a ceremony is cultish. Okay, I heard that already; give me a new information or stop repeating yourself. -- I would have no problem if someone wrote a critical article explaining the dangers of having a ceremony even in its LessWrongian variant, and proposed alternative ways to create personal connections. But dropping hostile comments to other peoples' articles is so much easier. Well, I am not impressed.
People who try hard to appear smart typically have a problem cooperating on anything. For a textbook example, visit Mensa. It is a miracle that Mensa ever gets anything done, because every time anyone proposes an idea, all people loudly disagree, to signal that they are not sheep. Okay, I get it, they are not sheep; but it is still funny how an organization consisting purely of highly intelligent people became such a laughing stock for the rest of the world. Probably they were too busy signalling that they are not sheep, so they missed the forest for the trees.
There is a time to disagree, and there is also a time to agree. If someone has a policy of e.g. never singing a song together with other people (because that might irrationally modify their feelings towards them), I accept if they decide to never sing a song together with fellow rationalists. Yes, they are consistent. I respect that. But if someone is willing to sing a song with random strangers, but would never sing a song with rationalists, that would be reversing stupidity. It means sabotaging yourself and your goals; trying to get some nonexistent points for doing things the hard way.
The proper moment for criticism is when something worth criticising happens. Not when someone merely pattern-matches something to something, and cannot stop obsessing about that. Here is a group of people who all voluntarily decided to share some powerful emotional moments together. Did they commit suicide later? No! Did they donate all their money to Eliezer? No! Did they send disconnection letters to their relatives? No! Did they refuse to talk with their friends who didn't participate in the ritual? No! Did they kill someone or send death threats? No! Are they reduced to mindless zombies? No! Are they keeping the details secret from the rest of the world, threatening to punish whistleblowers? No! -- So why the hell does someone keep repeating that it essentially is the same thing; and why should I pay any attention at all to that kind of criticism?
In real life you sometimes get people who write, using different words, under dozen different articles: "I suspect that this all is just Eliezer's cult designed to extract money from naive people". How much of that is acceptable criticism, and how much is just annoying? Discussing that thing once, thoroughly? Yes, definitely. Dropping the idea around all the time? That's just poluting the space. Problem is that at the moment some people are already deeply annoyed, other people go meta and say we need criticism.
Then write the article. (And why are some people annoyed? Can they refute the criticism or not? I find it odd how everyone is running on emotion..)
Democracy does not work well online.
Is the karma system working well?
I have a problem with boring repetitive criticism.
Then write a refutation, post it, and point people to it. (I could add it is easy to get annoyed by claims you can't refute...)
The proper moment for criticism is when something worth criticising happens.
What about criticising theories? Here's a question: has EY ever withdrawn a posting?
No! Did they donate all their money to Eliezer? No! Did they send disconnection letters to their relatives? No! Did they refuse to talk with their friends who didn't participate in the ritual? No! Did they kill someone or send death threats? No! Are they reduced to mindless zombies?
Did they do something known to stimulate the parts of the brain people are irrational with?
So why the hell does someone keep repeating that it essentially is the same thing; and why should I pay any attention at all to that kind of criticism?
They are probably repeating it because it has never been refuted. But I was not exactly saying that. I don't think LW is a full blown cult. I think it is a rather imperfect promoter of rationalism. But nuance gets lost when people get emotional.
Tying the moderator reaction to whether or not the criticism is fun to receive, rather than if it is useful to receive, is a recipe for receiving fun but useless criticisms and not receiving unfun but useful criticisms.
Useless criticisms are no fun at all.
What happened to the Search for Truth?
View more: Next
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
I broadly agree. My thinking ties shoulds and musts to rules and payoffs. Wherever you are operating a set of rules (which might be as localised as playing chess), you have certain localised "musts".
I'm very resitant to the idea, promoted by EY in the thread you refenced, that the meaning of should changes. Does he think chess players have a different concept of "rule" to poker players?