Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Kaj_Sotala comments on The curse of identity - Less Wrong

125 Post author: Kaj_Sotala 17 November 2011 07:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (300)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kaj_Sotala 17 November 2011 04:03:55PM 11 points [-]

but I wondered why Kaj hadn't included any solutions

I've been somewhat helped by simply realizing the problem. For example, recently I was struggling with wanting to study a lot of math and mathy AI, because that's the field that my brain has labeled the most prestigious (mostly as a result of reading Eliezer et al.). When I realized that I had been aiming at something that I felt was prestigious, not something that was actually my comparative advantage, it felt like a burden was lifted from my shoulders. I realized that I could actually take easier courses, and thereby manage to finish my Master's degree.

Comment author: [deleted] 17 November 2011 04:24:22PM *  4 points [-]

My understanding is the quote: "It's better to be a big fish in a small pond than a small fish in a big pond." is substantially related to status.

If I try to apply it to your situation to find isomorphisms, I find a lot:

Rather than being a small fish(struggling with math) in a big pond(Eliezer et al.), you want to be the big fish(actually my comparative advantage) in the small pond(take easier courses.)

Considering this, are you sure you've left the status framework? If so, why?

(Edited after comment from TheOtherDave for brevity.)

Comment author: Kaj_Sotala 17 November 2011 04:36:41PM *  17 points [-]

I want to pursue my comparative advantage because that's the best way that I can help SIAI and other good causes, regardless of status considerations. Pursuing mathy stuff is only worthwhile if that's my best way of helping the causes I consider valuable.

Or to put it more succinctly: if being a big fish in a small pond, or even a small fish in a small pond, lets me make money that can be used to hire big fish in big ponds, then I'd rather do that than be a small fish in a big pond.

(I won't try to claim that I've left the status framework entirely, just to some extent on this particular issue. Heck, I'm regularly refreshing this post to see whether it's gotten more upvotes.)

Comment author: [deleted] 17 November 2011 05:11:53PM 1 point [-]

That's a fair point, but because money is so fungible, it's exactly the same kind of statement that you would be making if you were in fact selfish and didn't care about existential risk at all. In the same sort of way that both a new FAI and a new UFAI may have one of their tasks be to ask for some computing power.

So while that may be the right thing to do, I'm not sure if that in of itself can be taken as evidence that you care more about existential risk than status. Although, if you take that into account, then it really does work, because you aren't getting the status that you would get if you immediately helped the SIAI, you are instead ignoring that for a later boost that will really help more.

Honestly, the more I talk about this topic, the less I feel like I actually have any concrete grasp of what I'm saying. I think I need to do some more reading, because I feel substantially too confused.

Comment author: Kaj_Sotala 17 November 2011 05:24:58PM *  1 point [-]

True. It also isn't very reassuring to know that some of the paths which I'm now pursuing will, if successful, give me high status (within a different community) in addition to the status boost one gets from being rich. I do know that I'm still being somewhat pulled by status considerations, but at least now I'm conscious of it. Is that enough to avoid another hijack? Probably not merely by itself. I'll just have to try to be careful.

Comment author: pjeby 17 November 2011 05:32:31PM 5 points [-]

Why are you even trying to avoid status considerations? How does avoiding status considerations help you reach your instrumental goals?

Or, more precisely: what makes you think that conscious awareness and attempting to avoid status considerations will be any more successful at changing your actual behavior than any other activity undertaken via willpower?

Comment author: Kaj_Sotala 17 November 2011 05:48:12PM 10 points [-]

I'm not trying to avoid status considerations. I'm only trying to avoid them hijacking my reasoning process in such a way that I think the best ways of achieving status are also the best ways of achieving my non-status goals.

I can't completely ignore status considerations, but I might be able to trade a high-status path that achieves no other goals, to a path that is somewhat lower in status but much better at achieving my other goals. But that requires being able to see that the paths actually have different non-status outcomes.

Comment author: rysade 18 November 2011 08:50:46AM 0 points [-]

This is very clear. Others should refer back to this for a refresher if the topic becomes confusing. I know it's set my head spinning around sometimes.

Comment author: Eliezer_Yudkowsky 17 November 2011 07:33:21PM 16 points [-]

Comparative advantage is eating the sort of food that most greatly increases your fish size in the pond whose size implies the greatest marginal payoff for adding fish of the size you can become if you enter that pond.

Comment author: [deleted] 17 November 2011 08:54:03PM *  0 points [-]

When I combine what you said with:

I don't try to not seek status, I try to channel my status-seeking drive into things that will actually be useful.

I think I may have dissolved my confusion. You could separate it out into two pieces:

1: Comparative advantage - An Optimization Process

2: Things that will actually be useful. - Being Friendly

My confused feeling seems like it might have been from setting these things as if they were opposed and you could only maximize one.

But if you figure the two are multiplied together, it makes much more sense to attempt to balance both correctly, to maximize the result.

Utility functions aren't quite as simple as multiplying two numbers, but the basic idea of maximizing the product of comparative advantage and usefulness sounds a lot more reasonable in my head then maximizing one or the other.


Comment author: TheOtherDave 17 November 2011 04:33:30PM -2 points [-]

...but not edit.

Comment author: [deleted] 17 November 2011 04:55:46PM *  2 points [-]

(Edited after comment from pjeby for brevity.)

I suppose I could simplify this to "There are layers of status seeking. So it's very easy to think you aren't making a Status0 play, because you are making a Status1 play, and this can recurse easily to Status2, Status3, or Status4 without conscious awareness."

Comment author: TheOtherDave 17 November 2011 06:04:01PM 4 points [-]

Erg. That sounds really insane. Which is bizarre, because although it sounds insane when I actually say it, my brain normally handles it without too much self awareness, and could go back to doing so if I wasn't specifically trying to analyze it in the context of this discussion.

FWIW, that sense of "this sounds insane when I say it explicitly but feels natural if I don't think about it" is an experience I often have when I am becoming aware of my real motives and they turn out to conflict with preconceived ideas I have about myself or the world. Usually, either the awareness or the preconceived ideas tend to fade away pretty quickly. (I endorse the latter far more than the former.)

Comment author: pjeby 17 November 2011 05:22:11PM 2 points [-]

Except, pjeby essentially said that "But if you were a truly good person, you would acknowledge that you were a status seeking hypocrite."

Uh, no. That is so far off from what I said that it's not even on the same planet.

See, "good" and "hypocrite" are just more status labels. ;-)

What I was saying is, if you acknowledge your actual goals, you might have a better chance of sorting out conflicts in them. Nowhere does labeling yourself (or the goals) good or bad come into it. In fact, in the discussion on solutions, I explicitly pointed out that getting rid of such labels is often quite useful.

And I most definitely did not label anyone's goals hypocritical or advise them to aspire to goodness. In fact, I said that the original questioner's behavior may well have been optimal, given their apparent goals, provided that they didn't think too much about it.

In much the same way that your comment would've been more workable for you, had you not thought too deeply about it. ;-)

Comment author: [deleted] 17 November 2011 06:22:21PM 0 points [-]

In much the same way that your comment would've been more workable for you, had you not thought too deeply about it. ;-)

Upon additionaI retrospection, (and after lunch) I agree. I'll edit those down to the more workable parts.

Since there doesn't appear to be a way to do partial strikethrough, I guess I can just save the removed/incomplete parts in a text file if for some reason anyone really wants to know the original in the near future.