3 Since agents do not wish to be murdered, it is in their interests to agree to refrain from murder under an arrangement in which other agents agree to refrain from removing them.
So there are several things I don't like about this..
0) It's not in their interests to play the cooperative strategy if they are more powerful, since the other agent can't remove them.
1) It's not a given that all agents do not wish to be murdered. It's only luck that we wish not to die. Sentient beings could just as easily have come out of insects who allow themselves to be eaten by mates, or by their offspring.
2) So you sidestep this, and say that this only applies to beings that wish to be murdered. Well now, this is utilitarianism. You'd essentially be saying that all agents want their preferences fulfilled, therefore we should all agree to fulfill each others preferences.
You have decided that is unaccpetable because you have decided that you must not lose.
Essentially yes. But to rephrase: I know that the behavior of all agents (including myself) will work to bring about the agent's preferences to the best of the agent's ability, and this is true by definition of what a "preference" is.
Maybe. Almost everybody who has had their mind changed about sexual conduct had overridden an instinct.
I'm not sure I follow what you mean by this. My ideas about sexual conduct are in line with my instincts. A highly religious person's ideas about sexual conduct are in line with the instincts that society drilled into them. If I converted that person into sex-positivism, they would shed the societal conditioning and their morality and feelings would change. Who is not in alignment with their instincts?
(Instincts here means feelings with no rational basis, rather than genetically programmed or reflexive behaviors)
0) It's not in their interests to play the cooperative strategy if they are more powerful, since the other agent can't remove them.
I am not sure what the argument is here. The objectivist claim is not that every entity actually will be moral in practice, and it't not the claim that every agent will be interested in settling moral question: it's just the claim that agents who are interested in settling moral questions, and have the same set of facts available (ie live in the same society) will be able to converge. (Which is as objective as anything else....
Related to: Privileging the Hypothesis
-- Paul Graham
-- Doug Henwood
-- Eliezer Yudkowsky
Here are some political questions that seem to commonly get discussed in US media: should gay marriage be legal? Should Congress pass stricter gun control laws? Should immigration policy be tightened or relaxed?
These are all examples of what I'll call privileged questions (if there's an existing term for this, let me know): questions that someone has unjustifiably brought to your attention in the same way that a privileged hypothesis unjustifiably gets brought to your attention. The questions above are probably not the most important questions we could be answering right now, even in politics (I'd guess that the economy is more important). Outside of politics, many LWers probably think "what can we do about existential risks?" is one of the most important questions to answer, or possibly "how do we optimize charity?"
Why has the media privileged these questions? I'd guess that the media is incentivized to ask whatever questions will get them the most views. That's a very different goal from asking the most important questions, and is one reason to stop paying attention to the media.
The problem with privileged questions is that you only have so much attention to spare. Attention paid to a question that has been privileged funges against attention you could be paying to better questions. Even worse, it may not feel from the inside like anything is wrong: you can apply all of the epistemic rationality in the world to answering a question like "should Congress pass stricter gun control laws?" and never once ask yourself where that question came from and whether there are better questions you could be answering instead.
I suspect this is a problem in academia too. Richard Hamming once gave a talk in which he related the following story:
Academics answer questions that have been privileged in various ways: perhaps the questions their advisor was interested in, or the questions they'll most easily be able to publish papers on. Neither of these are necessarily well-correlated with the most important questions.
So far I've found one tool that helps combat the worst privileged questions, which is to ask the following counter-question:
What do I plan on doing with an answer to this question?
With the worst privileged questions I frequently find that the answer is "nothing," sometimes with the follow-up answer "signaling?" That's a bad sign. (Edit: but "nothing" is different from "I'm just curious," say in the context of an interesting mathematical or scientific question that isn't motivated by a practical concern. Intellectual curiosity can be a useful heuristic.)
(I've also found the above counter-question generally useful for dealing with questions. For example, it's one way to notice when a question should be dissolved, and asked of someone else it's one way to help both of you clarify what they actually want to know.)