Jeez, what's next? "Optimism Bias and My First Kiss"?

As an avid reader, I have to say I'm enjoying the personal revelations. They have much in common with certain varieties of religious exuberance. In fact the whole sequence has the feel of an Olde Tyme Rationalist Revival.

But butt smacking recollections from summer camp? Surely there has to be a line somewhere.

I know. I can't help it. That was my first step into advanced rationality. I mean, what was I supposed to say? It was an important point and I wanted to get it across.

The truth doesn't always save a person. It especially doesn't always save a person from embarrassment.

I prefer to see examples like this. There are lots of good reasons for grounding these discussions in the concrete.

Did you ever actually assert that your reasoning was out of principle, only to suddenly remember the experience through an unrelated pathway and thus the remember the real reason of your refusal later on, contradicting that initial assertion, or was it simply that you had never actually considered your reason of refusal until you were 15, and that it had only felt like you had been making the assumption that you had refused out of principle when you had never asserted that as truth- you had just never considered your reason, and you knew when you were 15 that there was reason to refuse- because it was a negative-sum game.

So, Eliezer. You're interested in the Singularity. And the Singularity will come about by means of self-improving artificial intelligence. If I understand correctly. Meanwhile, you're also interested in self-improvement of the human wetware - hence your contributions to this website. There's an obvious similarity between these two interests, possibly even an overlap.

Took me a while to notice it.

Richard,

By examining many such transitions, I discovered some generalizations about the unconscious function that takes thought N to thought N+1.

I'm very interested, please elaborate!

Eliezer,

You started noticing the value of beating specific biases at age 15, but apparently didn't get the importance of heuristics and biases in general till 2003. Why did that reflective generalisation take so long, when in hindsight it's so vital for your chosen path? That's not meant as criticism - I've done this too. I want to know how to fix it.

Incentive to think about this: doing so might lead you to notice something vital for safe seed AI.

RI, the problem was simply the size of the planet and its scientific literature. I just didn't run into the field of heuristics and biases until that late. I realized it was important the first time I read a mention-in-passing on a webpage - though I didn't realize how much there was to know and how little of it I knew; Emil Gilliam finally mailed me a copy of "Judgment Under Uncertainty".

At the age of fifteen, I realized the importance of defeating biases in general, I just didn't know there was a field that studied them. I knew the importance of rationality in general, but I'd never heard the word "Bayesian". It's a frighteningly big world out there - not like a hunter-gatherer tribe, where you can presume you know everything anyone else knows.

-- and since most writings in psychology are worthless, it is easy to give up on the whole field before one discovers worthwhile writings like "Judgement Under Uncertainty" and "The Moral Animal".

RI, a large part of my motivatation was simply to practice a mental skill: it is a delightful feeling to improve drastically one's ability to observe one's own deliberations. Three decades and a severe bump on the head separate my teenage years from today, and today I am almost completely unable to do this exercise.

BTW, it is my guess that the exercises Eliezer and I describe will confer most of their benefits on exercisers who are still teenagers.

RI, to answer your question: the function that takes thought N into thought N+1 is complex enough that I did not learn anything that could be put into neat sentences, nor do I retain any declarative memories of what I learned except that the deliberation proceeded in a much more "predictable-in-retrospect" manner when I thought about some themes than when I thought about others. E.g., I remember that thinking about my mom produced very opaque chain of thoughts.

The practice Eliezer describes strikes me as of greater potential benefit than the one I describe, but perhaps the one I describe can be accomplished by a greater fraction of teenagers reading these words. Very few individuals are blessed with the delightful hardware that the teenage Eliezer had available for such exercises.

Thanks for your replies,

and since most writings in psychology are worthless, it is easy to give up on the whole field

Fair point. But if it annoys us, think how the rational minds in the field must feel!

...it is a delightful feeling... ...did not learn anything that could be put into neat sentences... ...much more "predictable-in-retrospect"...

Are you sure the model that your mind was using to report its own thought process corresponded to what was really going on under the bonnet? I've barely scratched the surface yet in studying bias and rationality (the 2 books you recommended are queuing on my shelf) but a recurring theme seems to be that introspection about one's own thought processes is often demonstrably unreliable, even among highly intelligent subjects working within their fields of expertise and reporting very high confidence in their introspections.

Eliezer,

Maybe I'm just biased towards cross-fertilization between disciplines. I was just wondering about other vital but relatively obscure knowledge being missed for similar reasons, and if there might be a way to attack the problem. Anyhow, please thank Emil for us, a lot of people are getting stronger thanks to her little gift! :-)

...her little gift...

That's what I get for posting at 1am. Apologies to Emil, who I see is an SIAI donor and therefore one of my favorite people :-)

That's a horrible game. Did you have a name for the game as well? Spank-to-spank?

Interesting. As a child I thought I could remember everything, so lying was easy, it's own memory of having lied, and thus distinct from reality. It was only much later that I realized it was even possible for the two to become horribly confused in one's own mind. But did you lie and forget the lie? Or did you just incompletely remember and add the part about negative sums as an explanation of the behavior later on? I think the "sweeping" of our minds' corners is also prone to give us false memories. We think, "well, why did I do that? Oh yeah! Now I remember..." You might have just been lucky to stumble upon what you now believe to be the real memory, but introspection, especially of the past isn't always so accurate.

Are you sure the model that your mind was using to report its own thought process corresponded to what was really going on under the bonnet?

I was very confident that I could reliably retain or reconstruct the last five or so actual thoughts of my conscious deliberation -- and the actual ordering or sequencing of those thoughts. Although recalling or reconstructing a thought sometimes required a few seconds of effort or patience, the reconstruction sure felt like a reliable observation rather than an interpretation or the updating of a model or the use of a learned model. As for extracting generalizations or regularities from those observations, like I said, I can recall making only one (declarative) generalization, but I have no reason to believe that I could not bring the same regularity-noticing, causality-extracting powers to bear on those observations that I could to any observations made by my senses. (When observing a very complex process, the regularity-noticing consists mostly of implicit learning rather than explicit, declarative learning -- or so it seems to me.)