There are many more Rubicons one can cross in this general direction, and fewer still who do cross or want to. Your journey has hardly begun, if you have the will to continue.
So do you think it's possible to deal with depression by thinking "oh, just ignore that mood. It's just a defective portion of my brain speaking."
Or is the act of getting an antidepressant med basically acting on the desire to change your own brain?
What does it say about our regard for self and willingness to change our mental structure that so many people take antidepressants? If we were uploaded, would we freely modify our minds, or fear losing ourselves in the process?
Robin, care to name the next Rubicon? Or a next Rubicon?
Michael, you can help depression by thinking "I wish my brain would stop releasing these depression neurotransmitters" - it doesn't command your brain but it does prevent you from being helplessly caught up in the feeling and swept away; it stops you from thinking that your life is inherently absolutely awful and immedicable or thinking up more reasons why you should be depressed.
Getting an antidepressant is obviously an act of rebellion against a part of one's brain (by another part of one's brain, of course!)
I think that giving anyone who hasn't shown their ability to build a Friendly AI, the ability to modify their own brain circuitry, is like giving a loaded gun to a 2-year-old. And it's not that being able to build a Friendly AI means you know enough to modify yourself. It means that you know for yourself why you shouldn't. Modifying a system as messy as a human has to be left to something smarter than human - hence the point of designing a much cleaner Friendly AI that (provably correctly) self-improves to the point where it can handle the job.
"But I still suspect that there's a little distance there, that wouldn't be there otherwise, and I wish my brain would stop doing that."
A finely crafted recursion. I salute you.
I've yet to hear of anyone turning back successfully, though I think some have tried, or wished they could.
It seems to be one interpretation of the Buddhist project
Regarding self, I tend to include much more than my brain in "I" - but then, I'm not one of those who thinks being 'uploaded' makes a whole lot of sense.
Eliezer, A few years ago I sat across from you at dinner and mentioned how much you reminded me of my younger self. I expected, incorrectly, that you would receive this with the appreciation of a person being understood, but saw instead on your face an only partially muted expression of snide mirth. For the next hour you sat quietly as the conversation continued around us, and on my drive home from the Bay Area back to Santa Barbara I spent a bit more time reflecting on the various interactions during the dinner and updating my model of others and you.
For as long as I can remember, well before the age of 4, I've always experienced myself from both within and without as you describe. On rare occasions I've found someone else who knows what I'm talking about, but I can't say I've ever known anyone closely for whom it's such a strong and constant part of their subjective experience as it has been for me. The emotions come and go, in all their intensity, but they are both felt and observed. The observations of the observations are also observed, and all this up to typically and noticeably, about 4 levels of abstraction. (Reflected in my natural writing style as well.)
This leads ea...
@Thom: Why don't you write an article / sequence of articles here, on LW, on your now significantly more coherent and extensive model of reality? I, sincerely, would be really glad to read that.
MichaelG: I see a depression as mental and emotional loop with positive reinforcement feedback.
Predisposition is required for loop to be complete:
So the loop goes as follows:
There is no actual "you" in the way that it seems to be. A persistent thought pattern / meme complex got mistaken for a "you" by awareness and, sooner or later, awareness can see through the "you", which is a tremendous relief when it occurs.
As Einstein put it:
A human being is a part of the whole, called by us, "Universe," a part limited in time and space. He experiences himself, his thoughts and feelings as something separated from the rest -- a kind of optical delusion of his consciousness.
This delusion is a kind of prison for us. . .
Matthew C quoting Einstein: "A human being is a part of the whole, called by us, "Universe," a part limited in time and space. He experiences himself, his thoughts and feelings as something separated from the rest -- a kind of optical delusion of his consciousness."
Further to this point, and Eliezer's description of the Rubicon: It seems that recognizing (or experiencing) that perceived separation is a step necessary to its eventual resolution. Those many who've never even noticed to ask the question will not notice the answer, no matter how close to them it may be.
There is no actual "you" in the way that it seems to be. A persistent thought pattern / meme complex got mistaken for a "you" by awareness and, sooner or later, awareness can see through the "you", which is a tremendous relief when it occurs.
As Einstein put it. . .
As Julian Jaynes put it:
"...this space of consciousness inside our own heads. We also assume it is there in others'. In talking with a friend, maintaining periodic eye-to-eye contact, we are always assuming a space behind our companion's eyes into which we are t...
#1 is you. #2 is an attempt by "I" to escape #1 (yourself). #3 is like #2 but a bit more complex. The internal monologue that is the "I" is terrible at dealing with anything non-technical, such as a psychological problem involving your idea of who "I" was, is, and your hope of what "I" will become. It ignores the details that are the most important to solving the problem as it exists entirely to run away from the problem. You can be entirely free from it but, as one poster mentioned before, it is a continuous process of being aware of when "I" pops in to field a problem it should not.
Jef Allbright: Eliezer, A few years ago I sat across from you at dinner and mentioned how much you reminded me of my younger self. I expected, incorrectly, that you would receive this with the appreciation of a person being understood, but saw instead on your face an only partially muted expression of snide mirth.
I can't imagine why I might have been amused at your belief that you are what a grown-up Eliezer Yudkowsky looks like.
I don't know if I've mentioned this publicly before, but as you've posted in this vein several times now, I'll go ahead and say...
I'm not sure if you'll find this interesting, but I quit smoking using something like the method you are describing. Basically I labeled the craving for nicotine as an entity apart from myself - I named it "the beast". So instead of thinking "I want a smoke" I'd think: "the beast is hungry." This didn't work all by itself (I had a lot of practice quitting) but it was the last of 5-6 attempts to quit and its stuck for nearly ten years now.
I'm confused. Eliezer, you seem to be saying that reflectivity leads to distance from one's emotions, but this completely contradicts my experience: I'm constantly introspecting and analyzing myself, and yet I am also extremely emotional, not infrequently to the point of hysterical crying fits. Maybe I'm introspective but not reflective in the sense meant here? I will have to think about this for a while.
Maybe I'm introspective but not reflective in the sense meant here?
That's right. Reflection here refers to the skill of reasoning about your own reasoning mechanisms using the same methods you that use to reason about anything else. "Solving your own psychological problems" is then a trivial special case of "solving problems," but with the bonus that solving the problem of making yourself better at solving problems, makes you better at solving future problems. Surprisingly, it turns out that this is actually pretty useful, but you probably won't understand what I'm talking about for another four years and three months.
ZM, the question is whether being more reflective makes you less passionate, not so much the absolute levels. But you're correct that being introspective is not at all the same as what I'm describing here.
Moshe Gurvich, thanks for the encouragement. I can never decide if my problem is Depression as a disease, or just reaction to my particular life circumstances.
There are people who recommend purely cognitive approaches to depression, including a lot of self-monitoring. Finding a project that engages you, so that you don't dwell on your depression, is a different approach, although also purely cognitive.
My point on the original post though was that you might naively assume that people would be scared of self-modification. But then you see people using Pro...
I gained this kind of reflectivity when I was barely able to even think and I did not know how to use it wisely. One of my first memories is relentlessly purging my early childhood personality shortly after I discovered how to perform the trick then panicking and rebuilding a new self any sort of stuff laying around. Still think that rampant self-modification left scars on my mind that are still there today. Emotional distance did increase the more I examined, altered and experimented with this and eventually caused some really painful side effects. Eek.
eliezer, with all due respect, jef's brief description of iterated reflective experience was more elucidative than yours.
i'm amused that you responded with contempt and anger to a perfectly well-intentioned comment after making the post you just made. get your shoeshine box, yudkowsky.
Eliezer, inside each of us are whole societies of mind, which form and reform coalitions depending on circumstances. Coalitions at times declare themselves to be the "real me" but treat that with the same skepticism you would apply to some particular part of the USA to calling itself the "real America."
But if that were the only issue at hand, then that would generate the prediction that I would be even more unstable (!) if I were less analytical, which is the opposite of what I would have predicted.
Yes, it could possibly be that it is this introspection/reflectivity dichotomy that's tripping me up. A deep conceptual understanding that one's self can be distinct from what-this-brain-is-thinking-and-feeling-right-now does not neces...
Cassandra, I went through something like that, but it was something like age 15-19, not a fast episode during childhood! Your story intrigues me and I am interested in knowing more; also whether you're otherwise a mathematical thinker and whether you've read/enjoyed Godel, Escher, Bach.
Robin, it seems to me that a fairly stable coalition has gained control of my fingers, so presumably you can treat with that if you're interested in my blog posts, or any source code I generate? Or to look at it another way, I understand (have a story about?) what happened...
@Eliezer: I can't imagine why I might have been amused at your belief that you are what a grown-up Eliezer Yudkowsky looks like.
No, but of course I wasn't referring to similarity of physical appearance, nor do I characteristically comment at such a superficial level. Puhleease.
I don't know if I've mentioned this publicly before, but as you've posted in this vein several times now, I'll go ahead and say it:
functional self-similarity of agency extended from the 'individual' to groups
I believe that the difficult-to-understand, high-sounding ultra-abstract co...
Mine was age 16. I don't recall having any sort of panic, but given the extent to which my adult personality resembles what I was reading at the time, I may have "rebuil[t] a new self any sort of stuff laying around." It felt intentional at the time... No painful side effects, but that may be scar tissue. (I'm a quant, and I found GEB too clever for its own good. Maybe it was unique at the time, but by the time I read it, I had already seen its better points elsewhere.)
"Davis, what you were saying made sense to me, so I'm confused as to what you could be confused about."
I came up with a nice story (successful reflection decreases passion; failed reflection increases it) that seems to fit the data (Eliezer says reflection begets emotional detachment, whereas I try to reflect and drive myself mad), but my thought process just felt really (I can think of no better word:) muddled, so I'm wondering whether I should wonder whether I made a fake explanation.
My point on the original post though was that you might naively assume that people would be scared of self-modification. But then you see people using Prozac without a second thought. More commonly, alcohol and other mood-altering substances are used. So perhaps we aren't frightened of self-modification after all.As far as we can determine, humans have always wanted to submerge their consciousness beneath various drugs and estatic practices. Thinking is unpleasant for most people, and they work hard at turning off their capacity for self-reflection.
Give...
Eliezer, I suspect the coalition in control of your fingers is not as coherent or stable as it appears. Ruling coalitions like to give the impression that they have little effective opposition and are unified without internal dissent, but the truth is usually otherwise.
I'd say my self has been rebuilt, oh, 3-5 times, first at age 12, varying in duration between hours and months, and, unlike the above anecdotes, never with conscious direction. As a gross generalization, each time has involved less "building a new self from any sort of stuff laying around" and more of a sense of being guided to inevitable rational conclusions. (Very mathematical thinker, started and enjoyed but never finished GEB.)
Jef Allbright, it seems to me that if you want Eliezer to take your criticisms seriously, you're going to need more equations and fewer words. (It would be nice if Eliezer produced some equations too.)
I'm a bit dismayed, however, by the obvious emotional response and meanness from someone who prides himself on sharpening the blade of his rationality by testing it against criticism.
Let's be fair. All "someones" operate according to the same basic Darwinian principles, which involve the subsumption of some ideas and rejection of others into a self-concept which then defends "itself" against any perceived threat. And the biggest threat, of course, is the truth that the self is not fundamentally real. When that is clearly seen, the g...
Eliezer, I suspect the coalition in control of your fingers is not as coherent or stable as it appears. Ruling coalitions like to give the impression that they have little effective opposition and are unified without internal dissent, but the truth is usually otherwise.
That comment was quintessentially Hanson, and an observation whose insight gives me much cause to believe that the coalition in control of those fingers has travelled across many a Rubicon. . .
@Cyan: "... you're going to need more equations and fewer words."
Don't you see a lower-case sigma representing a series every time I say "increasingly"? ;-)
Seriously though, I read a LOT of technical papers and it seems to me much of the beautiful LaTex equations and formulas are only to give the impression of rigor. And there are few equations that could "prove" anything in this area of inquiry.
What would help my case, if it were not already long lost in Eliezer's view, is to have provided examples, references, and commenta...
I'll be the first to say I've never been "rebuilt", but I enjoyed GEB. I have next to nothing to protect and don't even fear death so much as dying.
Mathew C: "And the biggest threat, of course, is the truth that the self is not fundamentally real. When that is clearly seen, the gig is up."
Spot on. That is by far the biggest impasse I have faced anytime I try to convey a meta-ethics denying the very existence of the "singularity of self" in favor of the self of agency over increasing context. I usually to downplay this aspect until after someone has expressed a practical level of interest, but it's right there out front for those who can see it.
Thanks. Nice to be heard...
Based on the disproportionate reaction from our host, I'm going to sit quietly now.
Someday, the ruling coalition laying claim to the title of "Robin Hanson" will not include the member who remembers what their password is, and we will never find out about it.
On Eliezer's comment about abstract stuff = fake, nothing to say:
If you really want to communicate your ideas, transfer them all the way to another brain, you would try harder to present them so that almost anyone who wants to understand them (with the right level of background info) has no hard time doing so. Instead it's like you cram whole functions or classes into convoluted one-liners like some extreme programmer showing off his chops. Yeah, it may work, but you got to show that your code really runs for us mortals.
I understand Eliezer's ideas witho...
I had crossed when I was much younger, without realizing what I'd done or the consequences. I wish I was informed, but it's too late now. I guess I committed myself to this path, I might as well see where it leads.
Eliezer: If there is more than one rubicon to cross, is it possible to skip one? Does the question make any sense?
Robin: What coalitions should I expect to see? Who's in charge of Robin Hanson right now?
Jef: Give me exactly one reason why I should listen to you. Ignore his current inability in FAI: nothing you've said has convinced me that he is making a mistake that matters. If the mistake is that big, I can discover the ramifications for myself after I know what's going on.
@Partiallybright: "If you really want to communicate your ideas, transfer them all the way to another brain, you would try harder to present them so that almost anyone who wants to understand them (with the right level of background info) has no hard time doing so."
Yes, criticism fully accepted.
Partiallybright: "Instead it's like you cram whole functions or classes into convoluted one-liners like some extreme programmer showing off his chops."
Well, I code in Python most of the time, and I tend to write in functional/imperative style bec...
I have enjoyed what parts I read of Godel, Escher, Bach but I have yet to finish it. As far as being a mathematical thinker... I haven't really identified how my thought process works other than being confused much of the time. I do enjoy math but I don't seem to have much of a talent for it. Haven't really found anything I do have a talent for. I am trying to become more of a mathematical thinker and to construct a good foundation to build a system of knowledge on but I have this strong natural inclination to trust anyone and that tends to undermine some ...
Thanks, Cassandra, that was the information I was looking for.
It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here.
The idea of having to reweave yourself out of immediately available parts is startling to me - I remember my own recovery as being more like a chain of "Undos" to restore the original state. I wonder what the alternative would have been like, but I'm not going to try it again to see!
In regards to referring to yourself as a coalition. I am not so sure that would be a good idea. Cassie would be a good little hive drone because she never had any sense of strong identity to begin with. And I like to speak in the third person. ;) Seriously I am mildly uncomfortable with even referring to myself as 'I' these days because I try to keep very careful record of which factors influence my mind and how they influence me and after I add all this up it seems pretty clear to me that I do not exist. Perhaps I am simply wrong but this is the conclusion I have come to.
"As far as my childhood goes I created a lot of problems for myself by trying to force myself into a mold which conflicted strongly with the way my brain was setup."
"It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here."
For some mysterious reason, my younger self was so oblivious to the world that I never experienced (to my recollection) a massiv...
...it seems to me much of the beautiful LaTex equations and formulas are only to give the impression of rigor.
I didn't suggest equations to enforce some false notion of rigor -- I suggested them as an aid to clear communication.
When I was a teenager I had a concept I referred to as "the double edged sword of apathy". It was precisely the concept that separating oneself from certain aspects of oneself (which at the time I called fostering apathy) is a destructive tool which can be either positive or negative. Care must be taken not to slice your own arm off.
I don't believe that this danger should be removed though, at least I wouldn't personally allow it. I hold "self-modifying" to be the deepest aspect of life. When we finally get the technology to do source level modifications I won't let an AI do the job: I'll do it myself, regardless of the risk.
I grow more interested in the ideas of William James on this subject. Statements such as:
The passing Thought itself is the only verifiable thinker
Thought is a passing thought that incessantly remembers previous thoughts and appropriates some of them to itself
There is a "judging Thought" that identifies and owns some parts of the stream of consciousness while disowning others
The next moment another Thought takes up the expiring Thought and appropriates it. It binds the individual past facts with each other and with itself.
In this way, what ho...
This is an awesome and freaky topic.
"It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here."
Haha... this is indeed kind of weird to see. I am very familiar with this experience.
"Seriously I am mildly uncomfortable with even referring to myself as 'I' these days because I try to keep very careful record of which factors influence my mind and how they in...
"I gained this kind of reflectivity when I was barely able to even think and I did not know how to use it wisely. One of my first memories is relentlessly purging my early childhood personality shortly after I discovered how to perform the trick then panicking and rebuilding a new self any sort of stuff laying around. Still think that rampant self-modification left scars on my mind that are still there today. Emotional distance did increase the more I examined, altered and experimented with this and eventually caused some really painful side effects. Eek."
Yes, yes, yes.
I'm liking my analogy with "the Way" more and more. I was unsure about it at first.
" " Mathew C: "And the biggest threat, of course, is the truth that the self is not fundamentally real. When that is clearly seen, the gig is up."
Spot on. That is by far the biggest impasse I have faced anytime I try to convey a meta-ethics denying the very existence of the "singularity of self" in favor of the self of agency over increasing context. I usually to downplay this aspect until after someone has expressed a practical level of interest, but it's right there out front for those who can see it. "
I think you are m...
EY: I remember my own recovery as being more like a chain of "Undos" to restore the original state.
Presuming you're referring to your religious upbringing, that seems like a funny way of characterizing it. Virtually every old primitive civilization that we know about had religious superstitions that all look pretty similar to each other and whose differences are mostly predictable given the civilization's history and geography. (I say "virtually" just as cover; I don't actually know any exceptions). Modern Judaism is a whole lot sane...
Sorry, I botched the second-to-last sentence. It should read "For example, if you were a cult member between 2004 and 2006, and you_2008 consider both you_2005 and you_2002 to be fools, you_2005 considers you_2002 a fool, but you_2008 consider you_2002 wiser than you_2005, then count that as one improvement rather than two."
I'm pretty consequentialist and its hard to predict the effects of Eliezer's harsh criticism of Jef Albright, but in the absence of a compelling argument that it is for the better, I would not be so mean.
That said, Jef's comments were (perhaps unintentionally) condescending.
As for their substantive disagreement, I have little to add to what they and other commentators are contributing.
It seems that Tom McCabe and Daniel Franke didn't go through anything like what Cassandra and I went through, since they have no referent for at all for the experience, and tried to map it onto belief systems or religions.
If I understand C correctly, the mutual experience that we're describing is nothing like that. You find out how to disable pieces of yourself. Then one day you find that you've disabled too much. It doesn't necessarily have anything to do with religion or even with beliefs, except for whatever beliefs spurred you to start deleting pieces of yourself.
EY: You find out how to disable pieces of yourself. Then one day you find that you've disabled too much. It doesn't necessarily have anything to do with religion or even with beliefs, except for whatever beliefs spurred you to start deleting pieces of yourself.
Okay, I now see what you're saying. I haven't experienced it. I understand the trick of disabling pieces of oneself, but I've never in my recollection abused it. However, I can understand what it would be like because I've experienced something that I'm guessing is similar: I'm a high-functioning autistic, and I've had to put considerable effort into software emulation of the emotional hardware that I'm missing.
Everything I am, is surely my brain; but I don't accept everything my brain does, as "me".
Such an awkwardly phrased and punctuated sentence is evidence of cognitive failure, or at least a hiccup. There's a fundamental mistake you are trying to paper over right at the start of this essay, which goes downhill from there.
Why are hardcore materialists, who presumably have no truck with Cartesian mind/body dualism, so eager to embrace brain/body dualism? Or software/hardware dualism?
So you start by restricting your self to your brain (at least, I...
trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it.
I cannot recall ever trying to delete or even control a large part of myself, so no opinion there, but "distancing ourselves from ourselves" sounds a lot like developing what some have called an observing self, which is probably a very valuable thing for an person wishing to make a large contribution to the world IMHO.
A person worried about not feeling alive enough would probably get more bang for his buck by avoiding exposure to mercury, which binds permanently to serotonin receptors, causing a kind of deadening.
mtraeven: Why are hardcore materialists, who presumably have no truck with Cartesian mind/body dualism, so eager to embrace brain/body dualism? Or software/hardware dualism?
Has anyone but me brought up software/hardware dualism? I'm only using it metaphorically. I'm not claiming any fundamental, bright-line distinction.
Hmm. I remember being non-reflective in first grade but not in second grade. One consequence was that I couldn't re-write explicit beliefs in response to new information and I saw general injunctions and commands as relatively binding and automatic. Conflicting commands couldn't be accommodated, nor could common sense. I don't think that my emotions were any more intense. I never re-wrote myself, or noticed a change at the time, but I notice it in my memories. Early ones don't include the question "why am I doing this?" or "why do this...
I think I prefer, and should prefer, my smoothed out highs and lows. During a finite manipulation sequence of a galactic supercluster, whose rules I pre-established, I wouldn't necessarily need to feel much -- since that might feel like 'a lot of pointless muscle straining' -- other than a modest, homo sapiens-level positive reinforcement that it's getting done. Consciousness, if I may also give my best guess, is only good for abstract senses (with and without extensions), and where these abstract senses seem concrete, even to an infinite precision, not "highs" and certainly not "lows" are necessary.
My mom tells me that when I was 2 or 3 years old and in preschool, one day they had a group of kids come up and recite poems or sing songs or something. I got really scared and wouldn't do my poem. Riding home afterwords, I said "I'm ready to say my poem now."
"You can't honey, everybody's already gone," my mom said.
She explained to me why it's important to be ready when it's time to do something.
"I don't want to be scared. I want to be a person who does things when it's time," I said.
I have no recollection of this event, but e...
The ability to become emotionally detached is a useful skill (e.g. if you are being tortured) but when it becomes an automatic reflex to any emotion, it can take all the colour out of life.
Sometimes highly intelligent people are also overwhelmingly sensitive/empathetic so detaching is very tempting. The first few minutes of this video with the genius girl walking around the spaceship shows what it's like to be highly empathetic (Firefly).
http://www.youtube.com/watch?v=MsyuTLYx59g
But also: emotions come from the subconscious, and the subconscious contains...
"You find out how to disable pieces of yourself. Then one day you find that you've disabled too much. It doesn't necessarily have anything to do with religion or even with beliefs, except for whatever beliefs spurred you to start deleting pieces of yourself."
you're trying to describe a very common experience using uncommon (and frankly, bizarre - "disabling pieces of yourself"?) language to make yourself feel special. sorry, but that's what many will see when they read through this stuff.
cassandra had a set of dispositions or preferenc...
I have no memory of a time when I didn't think self-reflexively, I'm pretty sure I was doing it as far back as kindergarden. Though I'm not sure I took it to the extremes of some of you, I only ever did modest personality modification :). I realized I could like any previously hated food just by trying. It's somewhat useful having such great control over thoughts and emotions. Like I've amplified my fear to help me in facing the feared situation (this might sound nonsensical, but it really works). In 6th grade I was confused when the teacher talked about w...
no system of interesting complexity, including humans, actually attains long-term goals so much as it simply tries to null out the difference between its (evolving) internal model and its perceptions of its present reality.
That's no reason not to talk about goals, and instead only mention something like "utility". Humans are psychologically goal-oriented - i.e. if you talk about goals, people understand what you mean.
Talk about "goals" can be formally translated into talk about "utility", by considering utility to be estim...
This is a beautiful comment thread. Too rarely do I get to hear anything at all about people's inner lives, so too much of my theory of mind is generalizations from one example.
For example, I would never have guessed any of this about reflectivity. Before reading this post, I didn't think there was such a thing as people who hadn't "crossed the Rubicon", except young children. I guess I was completely wrong.
Either I feel reflective but there's higher level of reflectivity I haven't reached and can't even imagine (which I consider unlikely but am ...
Yvian, I too am surprised to be told that there are many people who aren't at stage 2. It's not a bit surprising that most people aren't at stage 3. I've been capable of that kind of thought for as long as I can remember, but it's only since maybe 17 (I'm currently 23) that I've actually had a habit of thinking that way.
I'm amused by the number of people on this thread saying that they've acquired thought habit X through overcoming a mental disorder.
[Eliezer] It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here.
I think it's only the fact of having consciously distanced yourself that's unusual; I believe it's very common for someone to unconsciously disidentify with some part of their self that they'd be better off having, and have to reintegrate it.
I think this character Eliezer Yudkowsky is straight out of some hard sci-fi novel. A cross-breed between man and machine, or an AI emulating humanness and only partially pulling it off. No way a non-fictional person can come up with all of this ... words fail ... stuff.
But hey, it's the new millenium and it's supposed to be a frikkin' sci-fi life, with humans turned into sci-fi characters, right?!
Out of the 6 billion of us, I think he's one of the most ready for uploading. Meet you on UploadEarth 2045, dude.
Partly on topic, perhaps someone here can give me a helping hand in my attempt to level-up intellectually. A heavy obstacle for me is that I have a hard time thinking in terms of math, numbers and logic. I can understand concepts on the superficial level and kind of intuitively "feel" their meaning in the back of my mind, but I have a hard time bringing the concepts into the frond of my mind and visualize them in detail using mathematical reasoning. I tend to end up in a sort of "I know that you can calculate X with this information, and kno...
@Eliezer: There's emotion involved. I enjoy calling people's bluffs.
Jef, if you want to argue further here, I would suggest explaining just this one phrase "functional self-similarity of agency extended from the 'individual' to groups".
Eliezer, it's clear that your suggestion isn't friendly, and I intended not to argue, but rather, to share and participate in building better understanding. But you've turned it into a game which I can either play, or allow you to use it against me. So be it.
The phrase is a simple one, but stripped of context, as...
Jef: That's more like it. Though part of your explanation is still unnecessarily convoluted and nested (try using shorter sentences), now I get it. It's an alright, non-fake concept/observation. But that still doesn't mean the perception of fakeness/having nothing to say isn't valid. But it may be just a perception in many if not all cases - you need to work harder to avoid being perceived as having nothing to say, that's all - talking a lot and saying a lot instead of talking a lot and saying little/nothing.
@Tim Tyler: "That's no reason not to talk about goals, and instead only mention something like "utility"."
Tim, the problem with expected utility maps directly onto the problem with goals. Each is coherent only to the extent that the future context can be effectively specified (functionally modeled, such that you could interact with it and ask it questions, not to be confused with simply pointing to it.) Applied to a complexly evolving future of increasingly uncertain context, due to combinatorial explosion but also due to critical und...
You find out how to disable pieces of yourself. Then one day you find that you've disabled too much.
This definitely happened to me. I realized at some point (a few years ago) that in trying to force-fit myself into roles I thought I needed to fill in order to be a "responsible person", I'd succeeded in turning off aspects of my brain that are actually pretty vital for me to learn effectively. I didn't do anything about this realization, though, until I experienced an Epic Fail that showed me that the way I was trying to operate was neither usefu...
Why must I be like that?
Why must I chase the cat?
Nothin' but the dog in me.
--George Clinton - Atomic Dog
gaffa: A heavy obstacle for me is that I have a hard time thinking in terms of math, numbers and logic. I can understand concepts on the superficial level and kind of intuitively "feel" their meaning in the back of my mind, but I have a hard time bringing the concepts into the frond of my mind and visualize them in detail using mathematical reasoning. I tend to end up in a sort of "I know that you can calculate X with this information, and knowing this is good enough for me"-state, but I'd like to be in the state where I am using the i...
Tim, the problem with expected utility maps directly onto the problem with goals. Each is coherent only to the extent that the future context can be effectively specified (functionally modeled, such that you could interact with it and ask it questions, not to be confused with simply pointing to it.) Applied to a complexly evolving future of increasingly uncertain context, due to combinatorial explosion but also due to critical underspecification of priors, we find that ultimately (in the bigger picture) rational decision-making is not so much about "...
Main post: Everything I am, is surely my brain.
It would seem that, as far as causes go, everything about any of us is contained in the zygote, long preceding any sort of "brain". Indeed, it would seem to go far more basic than that, as discussed in the Quantum Mechanics series. These recent discussions about ethics, morality, concept of self, etc. seem to be effects, rather than causes, the results of external forces interacting with the original selection and sequence of a relative few chemicals. Who can say that the eventual outcome and expres...
I find it difficult to be empathetic with people who have had to reject religious thought, because I've never been religious. It has always been clear to me that my parents' beliefs on the subject were absurd; the only change has been in my response to them (patient tolerance to impatient intolerance). I was a conscious atheist even before I realized there was no Santa Claus.
Perhaps that experience was something like what other people go through when they reject religion... except that I cared little about the Santa Claus myth in itself, and was traumatized more by the realization that my parents were willing to lie to me merely for their own personal pleasure than anything else.
Interesting discussion.
Eli,
First, since no one has come out and said it yet, maybe it's just me but this post was kind of whiny. Maybe everyone else here is more in-tune with you (or living in your reality distortion field), but the writing felt like you were secretly trying to make yourself out to be a martyr, fishing for sympathy. Based on my knowledge of you from past interactions and your other writings I doubt this to be the case, but none the less it's the sense I got from your writing.
Second, I, too, have been through a similar experience. When I...
gaffa, have you tried any Raymond Smullyan? If you want logic and mathematical reasoning in a game-like structure, he has several books (like the linked) that present them as riddles. There is no shame in not getting how they work until you read the solution to at least one in each chapter, but you get the pattern of how to think through certain sorts of logical structures.
When I first came across Eliezer's writings, it struck me that what I read felt so true to me, that for the first time I felt like I had found someone I could relate to.
I have been avidly reading everything from him I could come across, as long as it "felt" right, which was often. With time I noticed that we didn't think in the same way, and it felt to me Eliezer was much more rational, scientifical, structured, than I was.
I immediately felt that the desirable thing to do would be to read even more of him, so as to "absorb" those traits...
Eliezer: "I don't know if I've mentioned this publicly before..."
You definitely haven't mentioned that publicly in any place that I read, which makes me glad I decided to dip into the comments of this post. I always felt a tacit acceptance, or at least no active disagreement on your part of Jef's posts on similar subjects on SL4 and other online fora. (at least any available to immediate recall)
The subject of what parts of my influences, tendencies and opinions, and identifiable hardware quirks I call 'myself' is a driver of cycles of stress and ...
It's not a one-way street; with proper technique (e.g. NLP anchoring and reframing methods, to name just a couple) you can change the cached "meaning" of a certain class of events so that they have pretty much any emotional content you choose.
Granted, my personal experiences run in the direction of modifying "negative" things to be positive, and I haven't had much call for keeping around any negative feelings.
Truth is, your concern about losing the negative feelings is irrational... probably based on a science fictional ideal of "not being as human" if you lose the negative emotions. I used to be bothered by that, but then I got rid of the negative emotion I associated with getting rid of negative emotions. ;-)
Seriously, though, you need to distinguish between suppression or detachment/disassociation of a negative emotion, and not having it in the first place. It's like Spock vs the Dali Lama: big difference. At the point where you merely disidentify from an emotion, it's a step backwards.
What's necessary is to detach the emotional "tags" from with the experience - specifically, your brain's cached predictions of the future that will ari...
I see "Me" as all that is within my skin.
I find it helpful to think of different bits of me. My "Inner Toddler" is the bit which Wants things, or is upset by things. If I just tell it to shut up, it will become recalcitrant. If I listen to it, even though I will not necessarily do what it says, it is happier and I am happier.
This is why I am not Rationalist. I try to use Rationality to understand the World, and "myself", but I use emotion to set my goals, just as it gives me my rewards.
And- I get a great deal from the theories of Carl Rogers, who postulated an Organismic Self, or a "Me" which is my whole organism, and a "Self-concept", which is those bits of me which I allow myself to be conscious of, excluding bits which I am too ashamed of to be conscious of them, and including bits which are not really me, but which I pretend are me because of my concept of "good". I also see myself as an evolved being, and so draw from this that I fit into that habitat which I find myself: if an ancestor did not fit enou...
Jef Allbright:
So "Functional self-similarity of agency extended from the 'individual' to groups.", in other words, means "groups of humans follow similar practices to achieve their goals"? Or am I missing some mystic subtlety in the choice of "functional" over "similar", "self-similarity" over "a grouping like its parts", and agency over "method of achieving goals"? You took a lot of time to dance around the point that "groups also exclude of parts of themselves for similar reasons...
Responding to Gaffa (I kind of intended to respond right after the comment, bot got sidetracked):
When approaching a scientific or mathematical problem, I often find myself trying hard to avoid having to calculate and reason, and instead try to reach for an "intuitive" understanding in the back of my mind, but that understanding, if I can even find it, is rarely sufficient when dealing with actual problems.
I would advise you to embrace calculation and reason, but just make sure you think about what you are doing and why. Use the tools, but try...
I seem to have taken step #3 in the other direction. The bundle of desires, urges, and subconscious cognition that makes me feel things is "me", while that running voice in my head that wonders about why I feel what I feel, is, well, just a running voice in my head doing some analysis, but with no power to affect much of anything. My mind has basically never been able to overrule the parts of the brain that feel like "me", which gives my parents no end of frustration, as they repeatedly insist that people have to learn to do things they...
Eliezer: "It makes you more moral (at least in my experience and observation) because it gives you the opportunity to make moral choices about things that would otherwise be taken for granted, or decided for you by your brain."
I have to take specific issue with this (despite being further down in the comments than I think will attract anyone's attention). This post and its comments discuss a process by which a mind can modify its behavior through reflection, but while Eliezer and many others may use this power to strengthen their morality, it ca...
I know this is a few days late, but I couldn't help but notice that no one mentioned how your "Zenlike but not Zen" philosophy is basically just a weak version of Stoicism (weak in that you seem to desire some passion, whereas a stoic would advocate distancing yourself from all highs and lows). There is no need to create techniques to do this from scratch, the path has already been laid out. I would encourage anyone interested in the topic to research Stoic teachings, particularly Epictetus, if you haven't done so already.
[I recommend Epictet...
Say, Eliezer: Have you considered identifying yourself only with the part of your brain which asks clearly stated questions? This questioner is most clearly "you", IMO.
For example, I could draw the boundary of my selfhood as including the times/ parts of my brain, when I ask clear and well stated questions like: "What is this guy saying?"
If death is horrible then I should fight death, not fight my own grief.
Yes, but how do you tell whether death is horrible or not?
In other words, how is one supposed to know which of the following is true?
That's not really a fair comparison, is it? There is no reason to choose world B since in world A nobody feels grief about dying either (since nobody dies).
To make it fair, I think we need to change world A so that nobody dies, but everyone feels grief at random intervals, so that the average amount of grief is the same as in our world. Then it's not clear to me which world I should prefer...
Notably the rabbis of old made a step in the right direction: they created a vision of "the evil desire" as an external, anthropomorphic, force. This is already helpful to create a layer between the emotions and the "person", and muster the natural aggressive emotions against it. (and no, they did not think devil is an actual sadist with horns).
My criticism of this approach is that this framework is not fine enough to give you an advantage battling the "evil inclination": viewing the immoral desire as an external force, rather...
Though I don't like the fact, it does seem in my case to be true, that reflecting upon a passion can diminish it. It does seem that in taking a step back from myself to look at what my brain is doing, that this introduces a greater emotional distance from the world, than when I am not reflecting.
That's exactly what (Far Eastern) meditation is about.
Robin Hanson comments that the "I" even in a reflective person's mind is an unstable coalition.
My guess is that Eliezer knows this, and is defining his "self" to mean something like "the shifting coalition within this brain, that is trying to save the world". If this guess is wrong, I'd love to find out, this seems like the crucial bit to me.
I draw the lines differently.
I include the thoughts that I do not want to have as parts of my self, parts that I do not fully understand or control. I observe these thoughts, and I take care not to act on them, but I do not reject them from my definition of self.
I think something like "I want to be a person who does not think or feel X on a regular basis, but right now I am a person who is experiencing X." I put it in more neutral terms than rejection. I do not think of it as an imposition by something separate, although sometimes I get frustr...
Everything I am, is surely my brain; but I don't accept everything my brain does, as "me".
Though I don't like the fact, it does seem in my case to be true, that reflecting upon a passion can diminish it. It does seem that in taking a step back from myself to look at what my brain is doing, that this introduces a greater emotional distance from the world, than when I am not reflecting. Though I am extremely cautious when it comes to the moral issue of how to modify human nature, this is one thing that I would change, I think.
The point at which you start thinking that not everything that goes on your brain is to be identified with "you", is something of a Rubicon, I fear. And those who prize the intensity of a simple passionate existence, above success in life or a complex layered personality, should not cross that Rubicon.
(Though in this case you should accept advice from rationalists about practical policies for guarding yourself, because this is a dangerous world in which to be unreflective. For example: if you don't follow the link to the rest of this post, then sign the #$!@ up for cryonics regardless of what your simple passions say on the subject.)
Still here? Then here is the Rubicon of which I speak:
The first scenario is what I would call the intensity of simple passion; nothing between the world and the emotions. The sentence contains no "I" to get in the way. There is nothing to think about except the world itself, the sadness and the missing person.
In the second scenario it is not the world that is sad, but some particular person, an "I"; and the sadness of this "I" is something that can be called into question.
And in the third scenario, the borders of "I" have been drawn in a way that doesn't include everything in the brain, so that "I" is the victim of the brain, struggling against it. And this is not paradoxical. Everything that I am, has to be in my brain somewhere, because there is nowhere else for it to be. But that doesn't mean I have to identify with everything that my brain does. Just as I draw the border of "me" to include my brain but exclude my computer's CPU - which is still a sensible decision at least for now - I can define the border of myself to exclude certain events that happen in my brain, which I do not control, do not want to happen, and do not agree with.
That time I faced down the power-corrupts circuitry, I thought, "my brain is dumping this huge dose of unwanted positive reinforcement", and I sat there waiting for the surge to go away and trying not to let it affect anything.
Thinking "I am being tempted" wouldn't have quite described it, since the deliberate process that I usually think of as "me" - the little voice inside my own head - was not even close to being swayed by the attempted dose of reward that neural circuit was dumping. I wasn't tempted by power; I'd already made my decision, and the question was enforcing it.
But a dangerous state of mind indeed it would have been, to think "How tempting!" without an "I" to be tempted. From there it would only be a short step to thinking "How wonderful it is to exercise power!" This, so far as I can guess, is what the brain circuit is supposed to do to a human.
So it was a fine thing that I was reflective, on this particular occasion.
The problem is when I find myself getting in the way of even the parts I call "me". The joy of helping someone, or for that matter, the sadness of death - these emotions that I judge right and proper, which must be me if anything is me - I don't want those feelings diminished.
And I do better at this, now that my metaethics are straightened out, and I know that I have no specific grounds left for doubting my feelings.
But I still suspect that there's a little distance there, that wouldn't be there otherwise, and I wish my brain would stop doing that.
I have always been inside and outside myself, for as long as I can remember. To my memory, I have always been reflective. But I have witnessed the growth of others, and in at least one case I've taken someone across that Rubicon. The one now possesses a more complex and layered personality - seems more to me now like a real person, even - but also a greater emotional distance. Life's lows have been smoothed out, but also the highs. That's a sad tradeoff and I wish it didn't exist.
I don't want to have to choose between sanity and passion. I don't want to smooth out life's highs or even life's lows, if those highs and lows make sense. I wish to feel the emotion appropriate to the event. If death is horrible then I should fight death, not fight my own grief.
But if I am forced to choose, I will choose stability and deliberation, for the sake of what I protect. And my personality does reflect that. What you are willing to trade off, will sometimes get traded away - a dire warning in full generality.
This post is filed under "morality" because the question "Which parts of my brain are 'me'?" is a moral question - it's not predicted so much as chosen. You can't perform a test on neural tissue to find whether it's in or out. You have to accept or reject any particular part, based on what you think humans in general, and yourself particularly, ought to be.
The technique does have its advantages: It brings greater stability, being less subject to sudden changes of mind in the winds of momentary passion. I was unsettled the first time I met an unreflective person because they changed so fast, seemingly without anchors. Reflection conveys a visibly greater depth and complexity of personality, and opens a realm of thought that otherwise would not exist. It makes you more moral (at least in my experience and observation) because it gives you the opportunity to make moral choices about things that would otherwise be taken for granted, or decided for you by your brain. Waking up to reflection is like the difference between being an entirely helpless prisoner and victim of yourself, versus becoming aware of the prison and getting a chance to escape it sometimes. Not that you are departing your brain entirely, but the you that is the little voice in your own head may get a chance to fight back against some circuits that it doesn't want to be influenced by.
And the technique's use, to awaken the unreflective, is as I have described: First you must cross the gap between events-in-the-world just being terribly sad or terribly important or whatever, of themselves; and say, "I feel X". Then you must begin to judge the feeling, saying, "I do not want to feel this - I feel this way, but I wish I didn't." Justifying yourself with "This is not what a human should be", or "the emotion does not seem appropriate to the event".
And finally there is the Rubicon of "I wish my brain wouldn't do this", at which point you are thinking as if the feeling comes from outside the inner you, imposed upon you by your brain. (Which does not say that you are something other than your brain, but which does say that not every brain event will be accepted by you as you.)
After crossing this Rubicon you have set your feet fully upon the reflective Way; and I've yet to hear of anyone turning back successfully, though I think some have tried, or wished they could.
And once your feet are set on walking down that path, there is nothing left but to follow it forward, and try not to be emotionally distanced from the parts of yourself that you accept as you - an effort that a mind of simple passion would not need to make in the first place. And an effort which can easily backfire by drawing your attention to the layered depths of your selfhood, away from the event and the emotion.
Somewhere at the end of this, I think, is a mastery of techniques that are Zenlike but not Zen, so that you have full passion in the parts of yourself that you identify with, and distance from the pieces of your brain that you reject; and a complex layered personality with a stable inner core, without smoothing out those highs or lows of life that you accept as appropriate to the event.
And if not, then screw it, let's hack the brain so that it works that way. I have no confidence in my ability to judge how human nature should change, and would sooner leave it up to a more powerful mind in the same metamoral reference frame. But if I had to guess, I think that's the right thing to do.