= 27d567bc2d48d9f40e9ccc20ea370b15)
Roko comments on The Psychological Diversity of Mankind - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (153)
Are you sure "flummoxed" is the right word? I don't think "neurotypicals" are confused by the mathematics involved. They just dispute that the utilitarian math represents an accurate theory of ethics. Would you use the word "flummoxed" for a physicist who understands the mathematics of a theory but disputes that it says anything relevant about the real world, even if he has no alternative theory to offer?
For full disclosure, I am not convinced by utilitarian arguments at all, both in these problems you mention and in most other widely disputed ones. I understand them with perfect clarity; I just dispute that they have any relevance beyond the entertainment value of the logical exercise, and possibly propaganda value for some parties in some situations. I certainly wouldn't describe my situation as "flummoxed."
Eh, as I've argued before on LW, there are utilitarian, AS-compatible justifications for such a position: specifically, that your heroic act shuffles around the risk profiles of various activities in unpredictable ways, thus limiting the ability of people to manage risks, leading them to waste significant resources (perhaps exceeding the amount that would otherwise save more than a million lives) returning to their preferred risk profile.
The key part:
Note that this doesn't argue for a deontological prohibition, but rather, argues about the consequences of sudden deviations from social norms, without assumption of their categorical justness.
ETA: In terms of Timeless Decision Theory, you could put it this way: if people knew that bridge-walkers are drafted for deadly work on a moment's notice, it's much less likely you'd have a fat person handy to begin with. So, the way TDT calculates probabilities, the EU of pushing the fat guy off is very small on account of its low TDT-probability, eliminating the supposed utility gain.
It' isn't just about being fat while being on a bridge over trolley tracks, of course. It might be a worse world if people generally believed they should take deadly action when they see a utilitarian win.
Much less likely? That would require that such drafting be more likely on bridges than elesewhere (how often do these train accidents happen?) Also, ex ante one is more likely to find oneself one of the million saved than the one person sacrificing, so most everyone should agree to a policy that those in positions to offer incredible help be drafted.
The problem induced by pushing the fat guy off is that people don't know which zones now count as "sacrificial lamb" zones (because of the bizarreness of the deviation from social norms), except that bridges over densely-populated trolley tracks are one of them, so I think the resulting world meets this criterion.
But people are already choosing risk profiles that, under present social norms, cause them to die when near tracks that have an errant trolley coming, so it's not clear why they'd make tradeoffs (giving up other things they value) for greater near-trolley safety, and thus not clear why they'd prefer this at all.
In this case, the cost (borne by everyone in the area, not just people near tracks) is that they have to re-organize their lives around choosing routes that avoid sacrificial lamb zones. But -- by the scenario's stipulation -- people aren't currently choosing to bear the additional cost to be on the safer bridge rather than the dangerous track. (If they were, the scenario would involve millions crossing the bridge and few near the track.) What they are choosing is to bear the risk of death because of the convenience it affords.
And because the option of pushing someone off the track tells people, "Okay, you have to be a lot more risk averse to get your current level of risk", they're forced to pay more for the same safety.
On the other hand, don't forget that talk is cheap, and actions speak louder than words. I doubt that many utilitarians would be willing to follow their conclusions in practice in situations such as the fat man/trolley problem. To stress that point even further, imagine if you had to cut the fat man's throat instead of just pushing him (and feel free to increase the cost of the alternative if you think this changes the equation significantly relative to pushing). I'd bet dollars to donuts that a large majority of the contemporary genteel utilitarians couldn't bring themselves to do it, no matter how clear the calculus that -- according to them -- mandates this course of action.
This suggests to me that this "dumbfoundedness" might be in fact a consequence of more clear and far-reaching insight, not confusion. Biting moral bullets is easy in armchair discussions; what you'd actually be able to bring yourself to do is another question altogether. Therefore, when I see people who coolly affirm the logical conclusions of their favored formal ethical theories even when they run afoul of common folks' intuition, I have to ask if they are really guided by logic to an exceptional degree in their lives -- or do they simply fail to see, out of sheer mental short-sightedness, how remote their armchair theorizing is from what they'd be willing and capable to do if they, God forbid, actually found themselves in some such situation.
(This is not the reason why I don't see any validity in utilitarianism; that would be a topic for another discussion altogether. The point here is that logical consistency in ethical armchair discussions could in fact be a consequence of myopia, not logical clear-sightedness.)
I don't think this statement is logically consistent. Unless you're restrained by some outside force, if you don't do something, that means you didn't want to do it. You might hypothesize that you would have wanted it within some counterfactual scenario, but given the actual circumstances, you didn't want it.
The only way out of this is if we dispense with the concept of humans as individual agents altogether, and analyze various modules, circuits, and states in each single human brain as distinct entities that might be struggling against each other. This might make sense, but it breaks down the models of pretty much all standard ethical theories, utilitarian and otherwise, which invariably treat humans as unified individuals.
But regardless of that, do you accept the possibility that at least in some cases, bullet-biting on moral questions might be the consequence of a failure of imagination, not exceptional logical insight?
It's not always that simple. It would be inconsistent if our actions could be reduced to a simple utility function and we consistently used the word (and emotion) "want" to refer to actions that maximize that utility function, but neither of those are the case, because we're not intelligently-designed optimization processes. Our brains don't act under a single unified goal system, and very often the part of us that says it wants to do x, or the part that believes it wants to do x, or the part that would be happy if it could do x, or the part that feels bad if it doesn't do x — any of the parts where it feels like "wanting" rather than "doing" — isn't always the part that makes the decision. (In fact, in a direct causal sense, I'd say it's not the part that makes the decision, period. Sometimes it just seems like they're the same when they're properly synchronized.) Neither is the part that makes moral judgments on one's own actions and on other's actions, and so on.
Have you read any of the discussions of akrasia here? That's essentially shorthand for what we're talking about here (wanting to do something but not doing it), and if you are willing to discuss it on human terms — in terms of what humans actually mean when they say "want" rather than what a single-minded decision-theoretic reasoner would mean by it* — then such discussions can be quite fruitful, and not logically inconsistent or meaningless at all.
* If such an agent would say it at all, that is. It could be taken as a mistranslation, in the same sense that Eliezer says translating any of the Babyeaters' words about their own decisions as "right" would be a mistranslation. If a perfect decision-theoretic agent's utility function specifies some action, then by definition, it will automatically pursue that; there's no room for any "wanting" there, just deciding and doing. Indeed, the very fact that we have different words for "want" and "pursue" reflects the reality that we can and very frequently do one but not the other.
ata:
Yes, I've read lots of stuff written about akrasia on this blog. This would be a topic for a whole separate discussion, but to put it as briefly as possible, in general I'm highly suspicious of such concepts. I view them through what Bryan Caplan calls the "Gun-to-the-Head Test" (I had actually come up with the exact same argument independently before I read about it from Caplan):
Note how different this is from people who have no control of their behavior even under this test. A Parkinson patient can't stop shaking his hands, and a person with normal nerves can't refrain from the knee jerk when struck into the patellar ligament, no matter what you threaten them with.
Ultimately, I believe that people engage in akrasia and "addictive" behaviors because they sincerely want it. Procrastination and substance abuse are fun and pleasant, and may well be worth a large cost for those sufficiently fond of them. And if these people can subsequently claim that their socially disapproved behaviors were somehow against their will and this way lower their cost by assuaging their reputational consequences -- well, no wonder that such excuses are popular. Saying that you would "want" to avoid procrastination is just ritual signaling behavior, just like smokers saying that they "want" to quit.
I should add that this is a complex topic, to which this brief post doesn't do justice, but this does summarize my view on the matter.
Yes, your comment establishes that there exists a reason to make the following classifications:
a) going for a jog when you say you want to go for a jog and like the health benefits and feel good while jogging -> preference for jogging
and
b) smoking despite saying you don't want to smoke and being aware of the bad consequences for your body and lifespan and wishing smoking did not give you short-term pleasure (and vice versa) -> preference for smoking
However, to get to the root of the akrasia question, that's not enough. You would need to show that there is no significant, useful difference between those "preferences" that would justify having different labels for them. Do you really believe that the same kind of "preferring" is going on in a) as in b)?
SilasBarta:
I don't have a complete theory of akrasia and related behaviors; in fact, I don't think we know enough about these issues yet to say the final word. However, from what I've observed, I do think that the preferences in (a) and (b) are essentially the same, though of course the details of the cost/benefit calculus are different. The relevant difference between them lies in their social signaling consequences, not in the nature of the preferences as such. In the contemporary culture, exercise carries positive signals, so if you exercise, it is, if anything, something to brag about. Smoking carries negative signals, so it's in your interest to present it as something you can't control.
My further (and more controversial) relevant observation is that the contemporary public and expert opinion is biased in favor of claims of helplessness and victimhood. Thus, for example, as smoking is considered more and more immoral, smokers will be judged less negatively if they claim to be helpless addicts swindled by the predatory tobacco industry than if they just say "I like it, and it's none of your business." Similarly, people who prefer the pleasures of drinking and drugs will be viewed less judgmentally if they plead "addiction" than if they just admit that they accept the costs of these pleasures, which can sometimes be very large. (Note the change in their behavior when the cost is greatly increased in the gun-to-the-head test!)
To make such a plea, however, you need to suffer from an officially approved "addiction." You can't successfully plead helplessness and victimhood if you suffer from the urge to write blog comments instead of doing work, even though many people will testify that this urge can be far greater than the lure of officially "addictive" behaviors. (Though this might change in the future as the concept of "internet addiction" gains official circulation.) In any case, the important point here is that when you're tempted to claim that someone honestly "wants" to behave differently from what he actually does, you should consider that your judgment might be skewed by these prevailing biases. (And if you doubt the existence of these biases, just remember the likes of Tiger Woods pleading "sex addiction"!)
That said, I do find the practical discussions of akrasia and similar behaviors valuable. Someone who is truly willing to eliminate some such behavior will want to minimize the costs of eliminating it. Moreover, a good anti-acrasia technique can also work by expanding the set of choices and providing and alternative that offers as much satisfaction even without the damaging behavior.
Please don't misunderstand. I'm very sympathetic toward that view, and I agree it can explain a great number of cases. Many of the specific points you made there I agree with as well, especially about Tiger Woods and "sex addiction". I've also written diatribes (that I won't dredge up) about how people go great lengths to rationalize consumption of alcoholic drinks to make them socially acceptable, when really they just want to get high. Heck, I've even tried, long ago, getting myself addicted to legal substances that are on the socially-endorsed "addictive" list, and failed.
What I dispute is that it's a full explanation applicable to all asserted cases of akrasia. For example, it runs into these problems: If you did a gun-to-the-head test on the chain smoker and the jogger over an extended period,
the smoker would go through a kind of negative psychological stress not present in the jogger.
after a long enough time, the smoker would lose the urge to smoke, and thank the gunman[1] for having used such coercion, while the jogger would stay resentful.
ETA: the jogger would probably return to jogging thereafter, while the smoker would not return to smoking, even in private.
Also, it would require that we make no distinction between "this person is doing X because it is painful not to" vs. "this person is doing X because it is pleasurable". Our own psychological experience tells us that there is a difference between pleasure and the absence of pain, even if that difference is not relevant in every context. (Remember, rejection of the akrasia concept requires that you believe it is never a relevant distinction, not just that it's an unnecessary distinction in some contexts.)
Furthermore, it's highly probable that people dislike the impacts of e.g. smoking/drugs on them, above and beyond the social disapproval it brings on them, especially when e.g. it takes a smoker 20 cigarettes to get a minor buzz.
The above considerations keep me from cynically dismissing the concept of akrasia as means of retaining social status while acting out one's truly fundamental preferences.
[1] ETA: I'm sorry for assuming that someone who threatens lethal force against another over a long period in order to make a philosophical point is a man rather than a woman, but I don't think it's incredibly offensive to do so.
Related: The Medicalization of Everyday Life.
Stephen King (who is, incidentally, a former alcoholic) wrote a short story, "Quitters, Inc." with this as the premise. If they catch you smoking, they'll do horrible things to you and your family members.
It's not that much of a difference. Such a model could still accept that humans are unified individuals, but also attached to parts (defined as not the relevant part of the human) that interfere with the human's actions.
Roko's alternative is just to say, "X is that action that I would attempt; hardware inextricably connected to me would also stop me from doing X."
Of course, that does run into problems like, "So you agree that you're running on corrupted hardware that stops you from doing what you believe is morally right -- why should I trust you, then?
Except for very narrow definitions of "standard," this is just incorrect. Plato, Hume, Kant, and John Stuart Mill all understood and wrote about the difference between what they thought of as the rational or refined will and the more emotional appetite. Likewise Maimonides, St. Augustine, Epictetus, and a 16th century Taoist scholar whose name I can look up for you if it's actually important. In fact, an enormous part of standard ethics deals with the divergence between what we say is right and what we actually do, and tries to identify ways to help us actually do what we say is right.
The blanket assertion that anything you do without being physically restrained is what you wanted to do under the circumstances is a creature of 20th century free-market economics. While it can be part of a self-consistent moral philosophy (e.g. Ayn Rand's Objectivism), it's hardly a litmus test for sound ethical thinking. On the contrary, we should be deeply suspicious of any moral theory that tells us that whatever we do must be what we wanted to do, because it conveniently justifies a set of actions that we (apparently) find quite easy to carry out. What is easy is not always right.
Was this intended as a reply to the parent of my comment?
I was replying both to you and to Vladimir_M, because both of you seemed to me to be accepting the premise that humans (however defined) must be unitary actors in order to be amenable to coherent ethical accounts.
Understood, but just to be clear, I was only accepting that premise for purposes of argument, saying that you don't need to resort to non-unitary models to phrase Roko's position. I don't accept that premise as a general rule. (Or at least I recognize that this model quickly runs into problems -- see my exchange with Vladimir_M.)
You are mostly right, except that I disagree that such simplifications are limited to 20th century economics. I had in mind formal ethical theories that I find discussed in modern analytical philosophy, and especially utilitarianism. I honestly don't see how utilitarianism can make sense unless humans are modeled as unified agents, each with a single utility function. From what I've seen, other popular formal consequentialist approaches make analogous assumptions, for which I don't see how they could be reconciled with dissolving the concept of humans as unified agents.
But yes, considering the vast philosophical tradition you mention, my above statement definitely doesn't hold in general. However, to get back to the issue that started this discussion, I don't think that Aspergery logical consistency -- that, according to Roko, apparently makes for a good consequentialist ethicist -- would be a good guide through the works of the authors you mention!
Hm. We're a few levels down from the parent thread here, so please forgive me if I fail to focus on your main point. I'm aiming for it, but I might miss.
It seems like you're saying that, in so far as we appear to observe a unified human psychology, it may just be because of myopia -- upon truly considering a moral dilemma in all its ugly ramifications, people would approve of and adopt different courses of action.
That seems correct as far as it goes, but what if people's emotions and logic generally follow the same path? What if, upon reflection, all neurotypicals would agree that, ideally speaking, they would like to live in a world where people slit the throats of trolley-obstructors so that light rail would be safe enough to ride on, but each neurotypical individual also agrees that zie could never actually bring zerself to slit an innocent person's throat, because it would be too yucky?
That still leaves us with the vast array of neuro-atypicals in our search for diversity, but then the question of whether humankind has a unified psychology is still interesting. Instead of the response being an obvious "no, we're diverse," the response becomes an investigation of how many atypicals there are, how different their opinions really are, and perhaps which ones are worth viewing as "healthy" enough to count.
Let me just qualify that last remark. I believe there should be room for many different kinds of minds in our society, but that doesn't mean there's no such thing as mental illness. For example, a paranoid schizophrenic might have a different opinion about the trolley problem, but I'm not sure we should ask him -- maybe we should just offer him some antipsychotics and see if he calms down.
Tangent: The trolley problem actually seems like a relatively innocuous source of diversity. In terms of designing a world we would want to live in I think there is pretty broad agreement that we want our trolleys to not run out of control. Yes, the principles of the trolley controversy could end up leading to disagreement over something far more important but... right now there are people with deep, powerful desires central to their overall happiness, the fulfillment of which other people find morally repugnant and sinful. That strikes me as in many ways a much bigger problem that the deontology/consequentialism battle.
I am a bit confused OTOH why non-ADHD people (without akrasia, a term I just learned here on this webssite) find such questions interesting at all. To me, no matter what "system of morals" you may have, it's mostly useless thinking, because it's not like what I do depends that much on what I actually want to do, in my self-awareness.
So true. That's what akrasia is. But I'd be surprised if there were people who didn't experience that at least a little bit.
Interesting. This implies that there are actually two ways of interpreting such moral dilemmas: either as A) "what would you actually do in this situation", or B) "what would be the right thing to do in this situation, regardless of whether you'd actually be capable of doing it".
I've always interpreted the questions as being of type B, but the way you write suggests you're thinking of them as being type A. I wonder how much of the disagreement relating to these questions is caused by differing interpretations.
It's more complicated than that. Most people would say that there are imaginable situations where a certain course of action is right, but they'd be strongly tempted to act differently out of base motives. For example, if you ask a typical person whether it would be right to gain a large amount of money by some sort of cheating, assuming you know for sure there won't be any negative consequences, they'll immediately understand that the question is about what's normatively right, not how they'd be tempted to act. Some very sincere people would probably admit that they might yield to the temptation, even though they consider it wrong.
Now, imagine you're introduced to someone who had the opportunity to cheat a business partner for a million dollars with zero risk of repercussions, but flat-out refused to do so out of sheer moral fiber. You'll immediately perceive this person as trustworthy and desirable to deal with -- a man who acts according to high principles, not base passion and instinct. In contrast, you'd shun and despise him if you heard he'd acted otherwise.
However, let's now compare that with the extreme fat man problem (where you'd have to cut the fat man's throat to avert some greater loss of life). Imagine you're introduced to someone who was faced with it and who slit the fat man's throat without blinking. Would you feel warm and fuzzy about this person? Would any of the bullet-biting utilitarians fail to be profoundly creeped out just by the knowledge that they are standing next to someone who actually acted like that -- even though they'd all defend (nay, prescribe!) his course of action relentlessly when philosophizing? Moreover, I would again bet dollars to donuts that our genteel utilitarians would be much less creeped out by someone who couldn't bring himself to butcher the fat man.
When I think about this, I honestly can't but detect severe short-sightedness in moral bullet-biters.
I'm not sure "warm and fuzzy" is the right term, but ... I would feel a certain respect, and of course update my probability that they will fail to take the correct action out of bias or akrasia. And my probability that they will kill me.
Would you be creeped out by someone who cheerfully admitted they would kill you if you turned evil? I mean mind-control type evil? Because in fiction at least that's treated as a good thing, but still creepy.
(I think the creepiness is the fact that they can and will kill people, and there's the ever-present worry they might mistake you for a risk.)
I can believe that a neurotypical person would be more likely to imagine themselves doing the actual killing, while someone on the AS would be more likely to stay with the abstract problem.
I was going to dispute your use of "flummoxed" as well but then I realized my position on normative ethics is basically an extended defense of moral dumbfoundedness and decided that I wouldn't be the best person to make that argument.
I think anyone who is biting bullets and defending rational principles broadly applied is just more comfortable dropping intuitions (or holds them less strongly) and less comfortable with logical inconsistency (sound like anyone you know?). But I don't think that makes their claims about morality any truer than the dumbfounded. I disagree that the right answer to inconsistent intuitions is just deciding to pick some intuitions and ignore them.
You can keep all of them if you're okay saying that sometimes there are only immoral choices (or at least no moral ones) and that sometimes the action we ought to take is under-determined by our moral intuitions.
... so which is less immoral?
Yes, why should we assume that these difficult ethical conundrums have some sort of "right answer" at all? Why would asking about the "right choice" in trolley and similar problems necessarily have to have any more sense than asking about the "correct value" of 0^0?
That raises an obvious question: what do you actually do if you find yourself in a Sophie's choice, especially if the result of the null or default choice is more monstrous to you than the results of the other choices? Refusing to consider a class of decision theory problems is tantamount to precommitting to an unconsidered answer should one of them arise.
Of course, in most cases, people actually do seem to consider horrific choices once they're actually faced with one; I therefore conclude that the popular response of refusing to make an analysis of such problems is more about signaling than anything else.
Well, the correct answer could be that I don't know what I would do -- and even if I knew that I would probably act in a certain way, it wouldn't be the outcome of any rational deliberation, but just a whimsical reflex from my brain overloaded with the stress of the situation.
You'll probably agree that there are situations where this would be the only realistic answer. For example, suppose you were about to be shot in a minute and the executioner showed you two bullets and told you do choose which one will end up in your head, and also threatened to kill you in a more painful and gruesome way if you refuse to make your choice clear. What does any decision theory say about this situation? It's absurd to insist on a rational rule for decision-making here.
Now of course, you can say that I chose an example where whatever the calculus, the numbers end up being equal, since the two options are identical in every relevant respect. But why should we believe that if only the options are sufficiently different, there must be a way to impose an ordering of desirability on them? Why wouldn't the "answer undefined" response be applicable in a much broader class of situations than just those where consequentialist calculations evaluate all options the same? What property of the universe or logic (or something else?) demands otherwise?
I agree: in some cases, one can't conclude which of two awful options is least bad (or one can conclude that the difference between them isn't likely to be worth the effort of investigating further, under the circumstances), and in that case, a random selection between such options is as good as any strategy.
However, ISTM that most trolley problems don't fall into that category, and that a policy of refusing to consider them on principle is probably a signaling phenomenon (one doesn't want to appear to endorse killing the innocent, even in such a farfetched hypothetical).
That, however, is more likely to manifest itself in a decisive anti-utilitarian answer, not feigning indecisiveness. People who want to signal that they won't endorse killing the innocent will say that it's wrong to actively kill someone even if it saves other lives, so they wouldn't push the fat man etc. -- and usually this is an honest statement of how they would really act in practice. Expressions of moral intuitions that are loaded with signaling value are usually felt sincerely, and acted upon readily. Similarly, people who refuse to endorse any alternative -- who are, I believe, a small minority in the general public -- sincerely view the situation as akin to the bullet choice. It might be ultimately due to signaling, but note that among ordinary folks, this sends a very bad signal. It's not at all good to be perceived as morally indecisive and lacking in principles.
That said, I'd say your theory is applicable to enthusiastic consequentialists too, and actually more so. I have the impression that many people who bite moral bullets based on various consequentialist theories do it for signaling value. They want to signal their rationality, adherence to logic rather than emotion, bravery in face of hostile reactions from people whose moral intuitions get violated, etc. In fact, I'd venture to say that the signaling here is more transparent, since unlike the never-kill-the-innocent folks, they likely wouldn't be ready to follow what they say in practice [*].
--
[*] - This doesn't contradict what I wrote above (that signal-loaded moral statements are typically acted upon readily), because these people are signaling to a very different audience than ordinary folks, to whom that statement applies.