Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

RichardKennaway comments on The Danger of Stories - LessWrong

9 Post author: Matt_Simpson 08 November 2009 02:53AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (103)

You are viewing a single comment's thread. Show more comments above.

Comment author: RichardKennaway 09 November 2009 08:32:51AM 6 points [-]

I came to the conclusion that good and evil are not objectively real, or even useful concepts

...

I often agree that things labeled evil do more harm than good

Do more what than what?

Comment author: AdeleneDawner 09 November 2009 03:57:05PM *  0 points [-]

I haven't gotten around to deconstructing those terms yet, but off the top of my head:

A 'harmful to X' action is one that has a long-term effect on it that reduces its ability to function. Examples:

  • Taking some RAM out of a computer
  • Adding grit to a machine, increasing the rate at which it wears out
  • Injuring a person in such a way that they lose use of a body part, or develop PTSD, or are in ongoing pain (because ongoing pain reduces their ability to function, not because pain is intrinsically harmful)
  • Extremist activism, where doing so makes the movement less credible and decreases the rate at which more sensible activists can create change. (I assume here and below that, disregarding the extremism, the activism is promoting good in the sense at hand.)

A 'good for X' action, in this sense ('helpful' would be a better word), is one that has a long-term effect on it that increases its ability to function. Examples:

  • Adding RAM to a computer
  • Performing maintenance on a machine
  • Teaching a person, giving them medical help, establishing a relationship with them such that they can approach you for advice or help in the future
  • Extremest activism, where doing so moves the Overton window.

The question isn't usually whether an action does harm or good or both. The question is how much importance to give to the various harms and goods involved.

Comment author: Jack 09 November 2009 04:06:18PM *  1 point [-]

Any definition of harmful that doesn't include say, electro-shock torture and water-boarding is a really really bad definition.

Hint: Pain really is intrinsically harmful.

Comment author: AdeleneDawner 09 November 2009 04:10:18PM 2 points [-]

Those come under 'injuring in such a way as to cause the person to develop PTSD', and no, it's not.

Comment author: Jack 09 November 2009 04:21:07PM 3 points [-]

Plenty of people have been tortured and not ended up with PTSD. Moreover, we classify instances of those things as harmful long before the DSM even lets us diagnose PTSD.

Also, there are approximately fifty arguments in that post and comments, none demonstrating that pain isn't intrinsically harmful so I really have no idea what you want me to take away from that link.

Comment author: AdeleneDawner 09 November 2009 05:01:19PM 3 points [-]

PTSD or other long-term psychological (or physical) impairment, then - which may be sub-clinical or considered normal. An example: Punishment causes a psychological change that reduces the person's ability to do the thing that they were punished for. We don't (to the best of my knowledge) have a name for that change, but it observably happens, and when it does, the punishment has caused harm. (It may also be helpful, for example if the punished action would have reduced the person's ability to function in other ways. The two aren't always mutually exclusive. Compare it to charging someone money for a class - teaching is helpful, taking their money is harmful.)

Also, I do believe that there could be situations where someone is tortured and doesn't experience a long-term reduction in functionality, in which case, yes, the torture wasn't harmful. The generalization that torture is harmful is useful because those situations are rare, and because willingness to attempt to harm someone is likely to lead to harm, and should be addressed as such.

The most relevant point in the discussion of pain is right at the beginning - people who don't experience any pain tend to have very short or very difficult lives. That makes it obvious that being able to experience pain is useful to the experiencer, rather than net-harmful. So, even though some pain is observably harmful, some pain must be helpful enough to make up the difference. That doesn't jive with 'pain is intrinsically harmful', unless you're using a very different definition of the word, in which case I request that you clarify how you're defining it.

Comment author: Jack 09 November 2009 06:29:35PM 2 points [-]

Also, I do believe that there could be situations where someone is tortured and doesn't experience a long-term reduction in functionality, in which case, yes, the torture wasn't harmful.

Well, anyone else who thinks this is wrong feel free to modus tollens away the original definition. ...

I was hoping to make my point by way of counter example. Since you're not recognizing the counter example I have to go back through the whole definition and the context to see where we lost each other. But thats a mess to do because right now this is a semantic debate. To make it not one I need the cash value of you belief that something is harmful. Do you always try to avoid harm to yourself? Is something being harmful necessary for you to avoid it/avoid doing it to others? Is it sufficient? Does this just apply to you? All humans? AI? Animals? Plants? Thermostats? Futons? Is something other than help and harm at work in your decision making? You don't have to answer all of these, obviously, just give me an idea of what I should see if something is harmful so I can actually check to see if your definition works. Otherwise you can't be wrong.

Then we can see if "causing decreased functionality" leads to the right response in all the circumstances. For example, I think there are times where people want to limit their net functionality and are right to do so even and especially when they know what they're doing.

Comment author: AdeleneDawner 09 November 2009 07:24:55PM 0 points [-]

Do you always try to avoid harm to yourself?

No; if I can help someone else (or my future self) enough by harming myself or risking harm, I'll do so. Example: Giving a significant sum of money to someone in need, when I don't have an emergency fund myself.

Is something being harmful necessary for you to avoid it/avoid doing it to others?

No, there are other reasons that I avoid doing things, such as to avoid inconvenience or temporary pain or offending people.

Is it sufficient?

I use a modified version of the logic that I use to determine whether I should harm myself to decide whether it's worth it to harm others. I generally try to err on the side of avoiding harming others, because it's harder to estimate the effect that a given harm will have on their life than it is to estimate its effect on mine.

Does this just apply to you? All humans? AI? Animals? Plants? Thermostats? Futons?

My definition is meant to be general enough to cover all of those, but in each case the meaning of 'function' has to be considered. Humans get to determine for themselves what it means to function. AIs' functions are determined by their programmers (not necessarily intentionally). In practice, I consider animals on a case-by-case basis; as an omnivore, it'd be hypocritical of me to ignore that I consider the function of chickens to be 'become tasty meat', but I generally consider pets and wild animals to determine their own functions. (A common assigned function for pets, among those who do assign them functions, is 'provide companionship'. Some wild animals are assigned functions, too, like 'keep this ecosystem in balance' or 'allow me to signal that I care about the environment, by existing for me to protect'.) I lump plants in with inanimate and minimally-animate objects, whose functions are determined by the people owning them, and can be changed at any time - it's harmful to chop up a futon that was intended to be sat on, but chopping up an interestingly shaped pile of firewood with some fabric on it isn't harmful.

Is something other than help and harm at work in your decision making?

In a first-order sense, yes, but in each case that I can think of at the moment, the reason behind the other thing eventually reduces to reducing harm or increasing help. Avoiding temporary pain, for example, is a useful heuristic for avoiding harming my body. Habitually avoiding temporary inconveniences leaves more time for more useful things and helps generate a reputation of being someone with standards, which is useful in establishing the kind of social relationships that can help me. Avoiding offending people is also useful in maintaining helpful social relationships.

You don't have to answer all of these, obviously, just give me an idea of what I should see if something is harmful so I can actually check to see if your definition works. Otherwise you can't be wrong.

Comment author: Jack 10 November 2009 08:40:20AM 0 points [-]

In a first-order sense, yes, but in each case that I can think of at the moment, the reason behind the other thing eventually reduces to reducing harm or increasing help.

So If I am in extraordinary pain it would never be helpful/ not-harmful for me to kill myself or for you to assist me?

Also, where does fulfilling your function fit into this? Unless you function is just increasing functionality.

Finally, I guess you're comfortable with the fact that the function of different things is determined in totally different ways? Some things get to determine their own function while other things have people determine it for them? As far as I can tell, people determine the function of tools and this notion that people determine their own function, while true in a sense, is just Aristotelian natural law theory rearing its ugly head. It used to be that we have purposes because God created us and instilled us in one. But if there is no God it seems that the right response is to conclude that purpose, as it applied to humans, was a category error, not that we decide our own purpose.

Comment author: AdeleneDawner 10 November 2009 08:13:14PM *  2 points [-]

Reminder:

I haven't gotten around to deconstructing those terms yet, but off the top of my head:

This is beta-version-level thought. It isn't surprising that it still has a few rough spots or places where I haven't noticed that I need to explain one thing for another to make sense.


Also, where does fulfilling your function fit into this? Unless you function is just increasing functionality.

Function as I'm intending to talk about it isn't something you fulfill, it's an ability you have: The ability to achieve the goals you're interested in achieving. Those goals vary not just from person to person, but also with time, whether they're achieved or not. Also, people do have more than one goal at any given time.

I have used the word 'function' in the other sense, above, mistakenly. I'll be more careful.

So If I am in extraordinary pain it would never be helpful/ not-harmful for me to kill myself or for you to assist me?

There are two overlapping types of situations that are relevant; if either one of them is true, then it's helpful to assist the person in avoiding/removing the pain. One is that the person has 'avoid pain' as a relevant goal in the given instance, and helping achieve that goal doesn't interfere with other goals that the person considers more important. The other is that the pain is signaling harmful damage to the person's body. There are situations that don't come under either of those umbrellas - certain BDSM practices, where experiencing pain is the goal, for example, or situations where doing certain things evokes pain but not actual (relevant to the individual's goals) harm, and the only way to avoid the pain is to give up on accomplishing more important goals, which is common in certain disabilities and some activities like training for a sport or running a marathon.

Whether suicide would be considered helpful or harmful in a given situation is a function of the goals of the person considering the suicide. If you're in a lot of pain, have an 'avoid pain' goal that's very important, and don't have other strong goals or the pain (or underlying damage causing the pain) makes it impossible for you to achieve your other strong goals, the answer is fairly obvious: It's helpful. If your 'avoid pain' goal is less important, or you have other goals that you consider important and that the pain doesn't prevent you from achieving, or both, it's not so obvious. Another relevant factor is that pain can be adapted to, and new goals that the pain doesn't interfere with can be generated. I leave that kind of judgment call up to the individual, but tend to encourage them to think about adapting, and take the possibility into account before making a decision, mostly because people so often forget to take that into account. (Expected objection: Severe pain can't be adapted to. My response: I know someone who has. The post where she talks about that in particular is eluding me at the moment, but I'll continue looking if you're interested.)

If it weren't illegal or if there was a very low chance of getting caught, I'd be comfortable with helping someone commit suicide, if they'd thought the issue through well, or in some cases where the person would be unable to think it through. I know not everyone thinks about this in the same way that I do: 'If they've thought the issue through well' doesn't mean 'if they've fulfilled the criteria for me to consider the suicide non-harmful'. Inflicting my way of thinking on others has the potential to be harmful to them, so I don't.

Finally, I guess you're comfortable with the fact that the function of different things is determined in totally different ways? Some things get to determine their own function while other things have people determine it for them?

There's an underlying concept there that I failed to make clear. When it comes to accomplishing goals, it works best to consider an individual plus their possessions (including abstract things like knowledge or reputation or, to a degree, relationships) as a single unit. One goal of the unit 'me and my stuff' is to maintain a piece of territory that's safe and comfortable to myself and guests. My couch is functional (able to accomplish the goal) in that capacity, specifically regarding the subgoal of 'have comfortable places available for sitting'. My hands are similarly functional in that capacity, though obviously for different subgoals: I use them to manipulate other tools for cleaning and other maintenance tasks and to type the programs that I trade for the money I spend on rent, for example.

(This is based on the most coherent definition of 'ownership' that I've been able to come up with, and I'm aware that the definition is unusual; discussion is welcome.)

As far as I can tell, people determine the function of tools and this notion that people determine their own function, while true in a sense, is just Aristotelian natural law theory rearing its ugly head. It used to be that we have purposes because God created us and instilled us in one. But if there is no God it seems that the right response is to conclude that purpose, as it applied to humans, was a category error, not that we decide our own purpose.

I think I've already made it clear in this comment that this isn't the concept I'm working with. The closest I come to this concept is the observation that people (and animals, and possibly AI) have goals, and since those goals are changeable and tend to be temporary (with the possible exception of AIs' goals), they really are something entirely different. I also don't believe that there's any moral or objective correctness or incorrectness in the act of achieving, failing at, or discarding a goal.

Comment author: Emile 10 November 2009 09:41:04AM 0 points [-]

So if you're waiting for the bus and I kick you in the shin, there's no harm? (there would be some ongoing pain for a little while, but with no impact on what you're doing - waiting for the bus)

Comment author: rhollerith_dot_com 10 November 2009 11:23:29AM *  2 points [-]

Adelene's attitude as illustrated in this thread towards pain resembles my own.

I do not assign any intrinsic value to avoiding pain (or experiencing pleasure). (I am unsure whether Adelene goes this far.)

I must stress though that pain (and pleasure) are indispensible approximations or "predictors" for various (instrumental) values. If I had a Richard-friendly superintelligence as my constant companion, I could ignore the informational value of my pain (pleasure) sensations because I could consult the superintelligence to predict the long-term effects of the various actions I contemplate, but the way it is now, it is too expensive or impossible for me to estimate certain instrumental values (mostly around staying healthy) unless I consult my pain (and pleasure) sensations.

Moreover, I must stress that there are quite a few things that correlate with pain. Pain for example is a strong sign that I am in a mental state not conducive to learning or to the careful consideration many factors (such as is necessary to do a good job at fixing a computer program). I do not have complete control over the mental machinery that allows me to program computers, etc. I cannot for example choose to put myself in the mental state that I know to be most conductive to, e.g., computer programming while enduring certain conditions that tend to cause pain.

So, that is one thing that has not yet been mentioned in this thread that correlates with pain. Here is another. I probably cannot stay motivated to work hard at something unless I regularly take pleasure from that work (or at the least I have a realistic expectation of future pleasure resulting from the work). I do not (usually -- see next paragraph) take that to mean that what I really care about is pleasure. Rather, I take that to mean that I have imperfect control over the means (e.g., my mind) by which I get my work done and that in particular, one of the circumstances that might prevent me from achieving what I really care about is that there is no way for me to stay motivated to do the things I would need to do to achieve the things I really care about -- because my neurology just does not allow that (even though that would get me what I really care about).

Like many people in this day and age, I wish I had more motivation, that is, I wish my actual behavior was more in line with the policies and goals I have set for myself. In fact, my motivation has become so unreliable and so weak that I have entered upon an experiment in which I assume that life really is all about pleasure -- or to be more precise, all about the search for something I care enough about so that protecting it or pursuing it is "naturally plenty motivating". Nevertheless, this experiment is something I started only this year whereas the attitude towards pain (and pleasure) I describe below has been my attitude since 1992.

Moreover, the attitude toward pain (and pleasure) I describe below still strikes me as the best way to frame most high-stakes situations when it is important to avoid the natural human tendency towards self-deception, to avoid wrongly mistaking personal considerations for global considerations or to see past the cant, ideology and propaganda about morality that bombard every one of us in this day and age. There is in human nature a tension IMHO between perceiving reality correctly (and consequently avoiding the sources of bias I just listed) and having plenty of motivation.

So if you're waiting for the bus and I kick you in the shin, there's no harm?

Though the question was addressed to Adelene, I'll give my answer. If you kick me in the shin hard enough to cause pain, then there is a non-negligible probability that the kick damaged bone, skin or such. Damage of that type is probably "cumulative" in that if enough damage occurs, my mobility will be permanently impaired. So, the kick in the shins will tend to reduce the amount of insult that part of my body can endure in the future, which reduces my behavioral options.

Now if I was waiting to be executed instead of waiting for the bus, and there was no chance of my avoiding execution, I (the current me) would be indifferent to whether you kicked me (the hypothetical, doomed me) in the shins. The reason I would be indifferent is that it is not going to change anything in the long term (since I will be dead by execution in the long term).

What I just said is "big-picture" true, but not true in detail. One detail that prevents its being completely true is that your kicking me in the shin might inspire in you a taste for kicking people in the shin, which I would prefer not to happen. Another detail is that my reputation will live on after my execution, and if onlookers observe your kicking me in the shin, it could concievable affect my reputation.

If I am faced with a choice between A1 and A2 and both A1 and A2 lead eventually to the same configuration of reality ("state of affairs" as the philosophers sometimes say) then I am indifferent between A1 and A2 even if A1 causes me to experience pleasure and A2 causes me to experience pain. Why? Because subjective experiences (in themselves, not counting the conditions -- of which there are quite a few -- that correlate with the subjective experiences) are impermanent, and my reason tells me that impermanent things are important only to the extent that they have permanent effects. (And by hypothesis, the kick in the shins in our latest thought experiment has no permanent effects.)

If reality was structured in such a way that the subjective experience of pain decremented some accumulator somewhere in reality, and if that accumulator could not at trivial cost be incremented again to cancel out the decrementing caused by the pain, well, then I would have to reconsider my position -- unless I knew for sure that the contents of the accumulator will not have a permanent effect on reality.

This got a little long. I just wanted Adelene to know that not everyone here considers her comments on pain strange. (Also, apologies to those who have heard all this before more than once or twice in previous years on Overcoming Bias.)

Comment author: CronoDAS 10 November 2009 10:38:18PM 4 points [-]

If I am faced with a choice between A1 and A2 and both A1 and A2 lead eventually to the same configuration of reality ("state of affairs" as the philosophers sometimes say) then I am indifferent between A1 and A2 even if A1 causes me to experience pleasure and A2 causes me to experience pain. Why? Because subjective experiences (in themselves, not counting the conditions -- of which there are quite a few -- that correlate with the subjective experiences) are impermanent, and my reason tells me that impermanent things are important only to the extent that they have permanent effects. (And by hypothesis, the kick in the shins in our latest thought experiment has no permanent effects.)

According to several theories of cosmology, the end state of the universe is fixed: entropy will increase to maximum, and the universe will be in a state of uniform chaos. Therefore nothing we can do will have a truly permanent effect, as the final state of the universe will be the same regardless. Assuming that to be the case, are you really indifferent between being kicked in the shins and not being kicked in the shins, since the universe ends up the same either way?

Comment deleted 11 November 2009 03:17:24AM [-]
Comment author: CronoDAS 11 November 2009 08:52:23PM 0 points [-]

Also, on a more personal level, even if the universe doesn't die, you will...

Comment author: rhollerith_dot_com 14 November 2009 02:53:08AM *  0 points [-]

According to several theories of cosmology, the end state of the universe is fixed: entropy will increase to maximum, and the universe will be in a state of uniform chaos. Therefore nothing we can do will have a truly permanent effect, as the final state of the universe will be the same regardless. Assuming that to be the case, are you really indifferent between being kicked in the shins and not being kicked in the shins, since the universe ends up the same either way?

I cannot make any strong statements about how my preferences would change if I learned for sure that I definitely cannot exert any permanent effect on reality. The question does not interest me. Also, the current me does not sympathize with any hypothetical me who is stuck in a reality on which he cannot exert any permanent effect. He is a non-person to me. (Yeah, I can be pretty callous towards future versions of myself. But it is not like I can rescue him from his (hypothetical) predicament.)

Finally, and this is nothing personal, Doug, but I will probably not take the time to answer future questions from you on this subject because I have resolved to stop trying to convert anyone to any particular moral position or system of valuing things, and this question I just answered pulled me back into that frame of mind for a couple of hours.

Comment author: Cyan 14 November 2009 03:40:04AM 1 point [-]

Expounding your moral position in response to a direct question is not proselytizing.

Comment author: rhollerith_dot_com 14 November 2009 04:08:52AM *  0 points [-]

Finally, and this is nothing personal, Doug, but I will probably not take the time to answer future questions from you on this subject because I have resolved to stop trying to convert anyone to any particular moral position or system of valuing things, and this question I just answered pulled me back into that frame of mind for a couple of hours.

Expounding your moral position in response to a direct question is not proselytizing.

True. What I should have written is, Doug, please help me stop procrastinating! Please do not ask me any more questions for a while on morality or the fate of the universe.

Comment author: CronoDAS 14 November 2009 06:06:29AM 0 points [-]

Finally, and this is nothing personal, Doug, but I will probably not take the time to answer future questions from you on this subject because I have resolved to stop trying to convert anyone to any particular moral position or system of valuing things, and this question I just answered pulled me back into that frame of mind for a couple of hours.

No problem. I was just confused because "avoid pain" seems to be a goal built into most neurologically intact people at a fairly basic level (even lizard brains try to avoid pain), and that, in that context, seeing someone say that they don't care to avoid transient pain strikes me as an extremely bizarre thing to say.

Comment author: rhollerith_dot_com 14 November 2009 06:12:08PM *  0 points [-]

Finally, and this is nothing personal, Doug, but I will probably not take the time to answer future questions from you on this subject

No problem. I was just confused because . . .

seeing someone say . . . strikes me as an extremely bizarre thing to say.

And since you participated in the discussions in past years on Overcoming Bias on this subject, Doug, and since I've seen a lot of your comments about your psychology, I know the hugeness of the inferential distance I would have to bridge to remove your confusion.

Comment author: wedrifid 14 November 2009 04:21:53AM *  0 points [-]

because I have resolved to stop trying to convert anyone to any particular moral position or system of valuing things, and this question I just answered pulled me back into that frame of mind for a couple of hours.

Thanks for adding this clarification. I understand the need for restraint.

ETA: I just finished appreciating this addition when the grandchild struck it out.

Comment author: Emile 10 November 2009 03:54:52PM 2 points [-]

This isn't as much about one's personal attitude to pain, but the morality of inflicting pain to someone else.

Adelene seems to be saying, roughly, that inflicting pain on someone else is morally neutral as long as there is no long-term harm like losing an eye or developing PTSD. That seems very much at odds with most conceptions of human morality I know of.

Comment author: AdeleneDawner 10 November 2009 08:54:05PM *  1 point [-]

I agree with almost everything rhollerith said (Most obvious exception: The pain from being kicked is probably more a warning that you're in a situation that caries the danger of major damage than an indication of reduced capacity to take damage in the future.) and would like to point out that the examples I gave are not the only possible ones. In your example, the risk of miscalculating and actually doing long-term damage is relevant, as is the psychological implication of being attacked by a stranger in public. Plus, as I discussed here, enough people have a goal of avoiding pain that you can safely assume that any random stranger has that goal, so inflicting pain on them is harmful in light of that.

Edit: It'd also be harmful to me to go around kicking people in the shins - I'd quickly get a reputation as dangerous, and people would become unwilling to associate with me or help me, and there's a significant chance that I'd wind up jailed, which is definitely harmful.