Jonathan_Graehl comments on Anticipation vs. Faith: At What Cost Rationality? - Less Wrong

8 Post author: Wei_Dai 13 October 2009 12:10AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (105)

You are viewing a single comment's thread.

Comment author: Jonathan_Graehl 13 October 2009 12:22:38AM 2 points [-]

Do you think an intellectual argument is all you need to self-modify so drastically? Or is this just in anticipation of some future mind-surgery technology?

Comment author: Wei_Dai 13 October 2009 12:56:31AM 1 point [-]

That's part of what I'm asking, I guess. Does anyone value rationality for its own sake, enough to give up anticipation if it turns out to be irrational, purely on intellectual grounds? And what would you do if mind copy/merging technology becomes a reality in the future (which I assume most of us here think is more likely than not)?

Comment author: Eliezer_Yudkowsky 13 October 2009 01:26:07AM 15 points [-]

Does anyone value rationality for its own sake, enough to give up anticipation if it turns out to be irrational, purely on intellectual grounds?

You don't make a conscious decision to give up something like that, if it needs giving up. You learn more, see that what you once thought was sense was in fact nonsense, and in the moment of realization, you have already lost that which you never had. Really this is the wrong way to phrase the question: you should properly ask, "If the idea of anticipation is complete nonsense and all our thoughts about it are mere helpless clinging to our own confusion, would you rather know what was really going on?" and to this I answer "Yes."

If someone offered to tell me the Real Story, saying, "Once you learn the Real Story, you will lose your grasp of that which you once called 'anticipation'; the concept will dissolve, and you will find it difficult to remember why you ever once believed such a notion could be coherent; just as you once lost 'time'," I would indeed reply "Tell me, tell me!"

Comment author: Wei_Dai 13 October 2009 09:17:26AM *  6 points [-]

I think when I wrote my previous response I may have missed your point somewhat. I guess what you're really saying is that, if anticipation is truly irrational, then once we sufficiently understand why it's irrational, we won't value it anymore, and it won't require any particular "effort" to give it up. Is this a better summary of your position?

If so, are you really sure it's true, that the human mind has that much flexibility and meta-rationality? Why? (Why do you believe this? And why would evolution have that much apparent foresight?)

Comment author: Eliezer_Yudkowsky 13 October 2009 03:30:43PM 3 points [-]

It is a better summary; and I can give no better answer than, "It's always worked that way for me before." I think the real difficulty would come for someone who was told that they had to give up anticipation, rather than seeing it for themselves in a thunderbolt of dissolving insight.

Comment author: Wei_Dai 13 October 2009 06:01:35PM 2 points [-]

My reasoning here is that evolution in general has very limited foresight, therefore there must be a limit to human rationality somewhere that is probably far short of ideal rationality. "It's always worked that way for me before" doesn't seem like very strong evidence in comparison to that argument.

Comment author: CronoDAS 13 October 2009 08:59:47AM *  6 points [-]

So, it might very well be the case that "time" is something you can get rid of from fundamental physics equations and still get the right answers.

But clocks still work. Even if time is only an "emergent" phenomenon (tee hee), it's still something that's there...

Comment author: aausch 16 October 2009 02:11:40AM 3 points [-]

If someone offered to tell me the Real Story, saying, "Once you learn the Real Story, you will lose your grasp of that which you once called 'anticipation'; the concept will dissolve, and you will find it difficult to remember why you ever once believed such a notion could be coherent; just as you once lost 'time'," I would indeed reply "Tell me, tell me!"

What about this situation:

"As a significant shortcut to developing an understanding of the Real Story, you can follow a formula which begins with a forced loss of your grasp of that which you once called 'anticipation'. I can promise that, once you do understand the Real Story, you will find it difficult to remember why you ever once believed the notion of 'anticipation' could be coherent. I have never found it useful to think about reversing the shortcut formula, so I cannot promise that the process is reversible"

Comment author: MichaelVassar 16 October 2009 09:50:35PM 2 points [-]

I'll do that too. Lots of chemicals can help with this.

Comment author: Jonathan_Graehl 13 October 2009 06:36:25PM 3 points [-]

The possibility of losing the natural feeling of anticipation, or time, isn't really on the table (yet). Knowing the real nature of things intellectually is always good, but does knowing that a feeling is an illusion remove its interference with comfort in the face of a rational decision?

Part of the thrill in bungee jumping is in the overriding. Are you saying that you can manipulate your decision making so that counterproductive instincts fade away?

Comment author: Wei_Dai 13 October 2009 03:10:42AM *  2 points [-]

You don't make a conscious decision to give up something like that, if it needs giving up.

I don't agree with this. I tend to make all other important decisions consciously. What's so special about this one? (ETA: Also, one potential way of giving it up is to edit my brain using some future technology. I think I definitely want to make that decision consciously.)

The rest of your comment seems to be saying that you're not yet convinced that anticipation is irrational. That's fair enough, but doesn't really address the main point of my post, which is that we regard some parts of our decision making process as having terminal values, and may decide to keep them (as luxuries, more or less) even if we come to believe that they no longer have positive instrumental value as decision subroutines.

Comment author: MichaelVassar 16 October 2009 09:49:28PM 0 points [-]

I'd like to vote this up several times.

Comment author: Wei_Dai 16 October 2009 11:07:52PM 3 points [-]

Can you explain why? I personally can't get used to this writing style, and it took me a few hours to figure out what Eliezer was getting at. I also don't understand why he chose to use a tone of high confidence, on something that he has rather flimsy evidence about.

Comment author: Douglas_Knight 16 October 2009 11:34:56PM 1 point [-]

If you spent hours to figure out what something meant, it's probably worth writing it out in your own words. At least it should help people who find the first style natural understand and communicate with you.

Comment author: Wei_Dai 16 October 2009 11:50:46PM 2 points [-]

I did. See here.

Comment author: Psychohistorian 17 October 2009 01:23:59AM *  0 points [-]

Does anyone value rationality for its own sake, enough to give up anticipation if it turns out to be irrational, purely on intellectual grounds?

Anticipation is an experience. I don't really see how one could decide to give up anticipation because it's irrational any more than they could decide to give up hunger just because it's irrational.

At least, so long as anticipation refers to "the good (bad) feeling one gets when thinking about an upcoming good (bad) event." I'm not really sure what else you'd mean by it, and I'm not sure how the truth could hope to destroy it.

Comment author: pengvado 17 October 2009 02:25:58PM *  0 points [-]

I'm pretty sure that everyone here who's considering giving up "anticipation", uses that term to mean not just any thinking about future experiences, but a consistent method of assigning concrete probabilities to future experiences. And the hypothesis about the irrationality of experiencing such anticipation, is merely a corollary of that factual hypothesis: If your probability estimates on some particular question consistently fail describe good bets, then binding emotions to those probabilities motivates bad decisions.