notsonewuser comments on Decision Theory FAQ - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (467)
notsonewuser, a precondition of rational agency is the capacity accurately to represent the world. So in a sense, the local witch-doctor, a jihadi, and the Pope cannot act rationally - maybe "rational" relative to their conceptual scheme, but they are still essentially psychotic. Epistemic and instrumental rationality are intimately linked. Thus the growth of science has taken us all the way from a naive geocentrism to Everett's multiverse. Our idealised Decision Theory needs to reflect this progress. Unfortunately, trying to understand the nature of first-person facts and subjective agency within the conceptual framework of science is challenging, partly because there seems no place within an orthodox materialist ontology for the phenomenology of experience; but also because one has access only to an extraordinarily restricted set of first-person facts at any instant - the contents of a single here-and now. Within any given here-and-now, each of us seems to be the centre of the universe; the whole world is centred on one's body-image. Natural selection has designed us -and structured our perceptions - so one would probably lay down one's life for two of one's brothers or eight of one's cousins, just as kin-selection theory predicts; but one might well sacrifice a small third-world country rather than lose one's child. One's own child seems inherently more important than a faraway country of which one knows little. The egocentric illusion is hugely genetically adaptive. This distortion of perspective means we're also prone to massive temporal and spatial discounting. The question is whether some first-person facts are really special or ontologically privileged or deserve more weight simply because they are more epistemologically accessible? Or alternatively, is a constraint on ideal rational action that we de-bias ourselves?
Granted the scientific world picture, then, can it be rational to take pleasure in causing suffering to other subjects of experience just for the sake of it? After all, you're not a mirror-touch synaesthete. Watching primitive sentients squirm gives you pleasure. But this is my point. You aren't adequately representing the first-person perspectives in question. Representation is not all-or-nothing; representational fidelity is dimensional rather than categorical. Complete fidelity of representation entails perfectly capturing every element of both the formal third-person facts and subjective-first-person facts about the system in question. Currently, none of us yet enjoys noninferential access to other minds - though technology may shortly overcome our cognitive limitations here. I gather your neocortical representations sometimes tend causally to covary with squirming sentients. Presumably, their squirmings trigger the release of endogenous opioids in your hedonic hotspots, You enjoy the experience! (cf. http://news.nationalgeographic.com/news/2008/11/081107-bully-brain.html) But insofar as you find the first-person state of being panic-stricken as in any way enjoyable, you have misrepresented its nature. By analogy, a masochist might be turned on watching a video involving ritualised but nonconsensual pain and degradation. The co-release of endogenous opioids within his CNS prevents the masochist from adequately representing what's really happening from the first-person perspective of the victim. The opioids colour the masochist's representations with positive hedonic tone. Or to use another example, stimulate the relevant bit of neocortex with microelectrodes and you will find everything indiscriminately funny (cf.http://news.bbc.co.uk/1/hi/sci/tech/55893.stm ) - even your child drowning before your eyes. Why intervene if it's so funny? Although the funniness seems intrinsic to one's representations, they are misrepresentations to the extent they mischaracterise the first-person experiences of the subject in question. There isn't anything intrinsically funny about a suffering sentient. Rightly or wrongly, I assume that full-spectrum superintelligences will surpass humans in their capacity impartially to grasp first-person and third-person perspectives - a radical extension of the runaway mind-reading prowess that helped drive the evolution of disjunctively human intelligence.
So, no, without rewiring your brain, I doubt I can change your mind. But then if some touchy-feely superempathiser says they don't want to learn about quantum physics or Bayesian probability theory, you probably won't change their mind either. Such is life. If we aspire to be ideal rational agents - both epistemically and instrumentally rational - then we'll impartially weigh the first-person and third-person facts alike.
Hi David,
Thanks for your long reply and all of the writing you've done here on Less Wrong. I only hope you eventually see this.
I've thought more about the points you seem to be trying to make and find myself in at least partial agreement. In addition to your comment that I'm replying to, this comment you made also helped me understand your points better.
Just to clarify, you mean that human representation of others' pain is only represented using a (very) lossy compression, am I correct? So we end up making decisions without having all the information about those decisions we are making...in other words, if we computed the cow's brain circuitry within our own brains in enough detail to feel things the way they feel from the perspective of the cow, we obviously would choose not to harm the cow.
In at least one class of possible situations, I think you are definitely correct. If I were to say that my pleasure in burning ants outweighed the pain of the ants I burned (and thus that such an action was moral), but only because I do not (and cannot, currently) fully empathize with ants, then I agree that I would be making such a claim irrationally. However, suppose I already acknowledge that such an act is immoral (which I do), but still desire to perform it, and also have the choice to have my brain rewired so I can empathize with ants. In that case, I would choose not to have my brain rewired. Call this "irrational" if you'd like, but if that's what you mean by rationality, I don't see why I should be rational, unless that's what I already desired anyways.
The thing which you are calling rationality seems to have a lot more to do with what I (and perhaps many others on Less Wrong) would call morality. Is your sticking point on this whole issue really the word "rational", or is it actually on the word "ideal"? Perhaps burger-choosing Jane is not "ideal"; perhaps she has made an immoral choice.
How would you define the word "morality", and how does it differ from "rationality"? I am not at all trying to attack your position; I am trying to understand it better.
Also, I now plan on reading your work The Hedonistic Imperative. Do you still endorse it?
notsonewuser, yes, "a (very) lossy compression", that's a good way of putting it - not just burger-eating Jane's lossy representation of the first-person perspective of a cow, but also her lossy representation of her pensioner namesake with atherosclerosis forty years hence. Insofar as Jane is ideally rational, she will take pains to offset such lossiness before acting.
Ants? Yes, you could indeed choose not to have your brain reconfigured so as faithfully to access their subjective panic and distress. Likewise, a touchy-feely super-empathiser can choose not to have her brain reconfigured so she better understands of the formal, structural features of the world - or what it means to be a good Bayesian rationalist. But insofar as you aspire to be an ideal rational agent, then you must aspire to maximum representational fidelity to the first-person and the first-third facts alike. This is a constraint on idealised rationality, not a plea for us to be more moral - although yes, the ethical implications may turn out to be profound.
The Hedonistic Imperative? Well, I wrote HI in 1995. The Abolitionist Project (2007) (http://www.abolitionist.com) is shorter, more up-to-date, and (I hope) more readable. Of course, you don't need to buy into my quirky ideas on ideal rationality or ethics to believe that we should use biotech and infotech to phase out the biology of suffering throughout the living world.
On a different note, I don't know who'll be around in London next month. But on May 11, there is a book launch of the Springer volume, "Singularity Hypotheses: A Scientific and Philosophical Assessment":
http://www.meetup.com/London-Futurists/events/110562132/?a=co1.1_grp&rv=co1.1
I'll be making the case for imminent biologically-based superintelligence. I trust there will be speakers to put the Kurzweilian and MIRI / lesswrong perspective. I fear a consensus may prove elusive. But Springer have a commissioned a second volume - perhaps to tie up any loose ends.