k64

Replying toI Can Tolerate Anything Except The Outgroup

I Can Tolerate Anything Except The Outgroup

I just tried criticizing my ingroup. Did my blood boil? No. My Scotsmen got truer. Every time I could identify a flawed behavior, it felt inappropriate to include those people in my "real ingroup". Now, if I had a more objectively defined group based on voting record or religious belief or something, then maybe I'd be able to force my brain to keep them in my ingroup, but right now, my brain flips to "sure, I'm happy to criticize those people giving us a bad name. Look, I'm criticizing my ingroup!"

I tried 2 other experiments:
1. Think about criticisms toward my ingroup that do make me angry - maybe those are the ones... (read more)

Replying toWhy does Eliezer make abrasive public comments?

k641mo

Why does Eliezer make abrasive public comments?

I think it's worth noting that I also have had times where I was impressed with your tact. The two examples that jump to mind are 1) a tweet where you gently questioned Nate Silver's position that expressing probabilities as frequencies instead of percentages was net harmful, and 2) your "shut it all down" letter to NYT, especially the part where you talk about being positively surprised by the sanity of people outside the industry and the text about Nina losing a tooth. Both of those struck me as emotionally perceptive.

Replying toHow I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)

k641mo

How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)

The thing I wonder every time this topic comes up is: why is this the question raised to our attention? Why aren't we instead asking whether AlphaFold is conscious? Or DALL-E? I'd feel a lot less wary of confirmation bias here if people were as likely to believe that a GPT that output the raw token numbers was conscious as they are to believe it when those tokens are translated to text in their native language.

Also, I think it is worth separating the question of "can LLMs introspect" (have access to their internal state) vs "are LLM's conscious".

Replying toAyn Rand’s model of “living money”; and an upside of burnout

k641mo

Ayn Rand’s model of “living money”; and an upside of burnout

I'm curious how you'd see moral and long-term considerations playing into this. For instance:
1. Saving for retirement produces no experienced benefit for many years and will only ever complete a single investment cycle in a lifetime.
2. Donating to or working on global health, x-risk, etc. produces no experienced benefit ever in most cases.
Yet, in both cases, individuals seem capable of exercising willpower to do these activities for many years.

I can think of 3 models currently that could explain this:
1. They just are "dead willpower", but your willpower system gets enough income from shorter term investments to allow it to continue to invest in things that will not pay out any time... (read more)

Replying toWhy does Eliezer make abrasive public comments?

k642mo

Why does Eliezer make abrasive public comments?

I do think that this is probably part of my misprediction - that I simply idealize others too much and don't give enough credit to how inconsistent humans actually are. "Idealize" is probably just the Good version of "flatten", with "demonize" being the Bad version, both of which are probably because it takes less neurons to model someone else that way.

I actually just recently had the displeasure of stumbling upon that reddit and it made me sad that people wanted to devote their energies to just being unkind without a goal. So I'm probably also not modeling how my own principle of avoiding offense unless helpful would erode over time. I've seen it happen to many public figures on twitter - it seems to be part of the system.

Replying toWhy does Eliezer make abrasive public comments?

k642mo

Why does Eliezer make abrasive public comments?

I like this perspective. I would agree that there is more to knowing and being known by others than simply Aumann Agreement on empirical fact. I also probably have a tendency to expect more explicit goal-seeking from others than myself.

I haven't thought this through before, but I notice two things that affect how open I am. The first is how much the communication is private, has non-verbal cues, and has an existing relationship. So right now, I'm not writing this with a desired consequence in mind, but I am filtering some things out subconsciously - like if we were in person talking right now, I might launch into a random anecdote, but... (read more)

•••

Replying toWhy does Eliezer make abrasive public comments?

k642mo*

Why does Eliezer make abrasive public comments?

I accept your correction that I misquoted you. I paraphrased from memory and did miss real nuance. My bad.

Looking at the comment now, I do see that it has a score of -43 currently, and is the only negative karma comment on the post. So maybe a more interesting question is why I (and presumably several others) interpreted it as insult when logical content of "Intelligence(having <30y timeline in 2025) > Intelligence(potted plant)" doesn't contain any direct insult. My best guess is that people are running informal inference on "do they think of me as lower status", and any comparison to a lower intelligence entity is likely to trigger that. For instance,... (read more)

•••

Replying toWhy does Eliezer make abrasive public comments?

k642mo

Why does Eliezer make abrasive public comments?

I suspect that some of my dissonance does result from an illusion of consistency and a failure to appreciate how multi-faceted people can really be. I naturally think of people as agents and not as a collection of different cognitive circuits. I'm not ready to assume that this explains all of the gap between my expectations and reality, but it's probably part of it.

Replying toWhy does Eliezer make abrasive public comments?

k642mo

Why does Eliezer make abrasive public comments?

I think this is an important perspective, especially for understanding Eliezer, who places a high value on truth/honesty, often directly over consequentialist concerns.

While this explains true but unpleasant statements like "[Individual] has substantially decreased humanity's odds of survival", it doesn't seem to explain statements like the potted plant one or other obviously-not-literally-true statements, unless one takes the position that full honesty also requires saying all the false and irrational things that pass through one's head as well. (And even then, I'd expect to see an immediate follow-up of "that's not true of course").

Replying toWhy does Eliezer make abrasive public comments?

k642mo

Why does Eliezer make abrasive public comments?

I agree with this decision. You reference the comment in one of your answers. If it starts taking over, it should be removed, but can otherwise provide interesting meta-commentary.

Why does Eliezer make abrasive public comments?

k64

k64, Eliezer Yudkowsky

2mo

I don't want to ruffle any feathers, but this has been bugging me for a while and has now become relevant to a decision since MIRI is fundraising and is focused on communication instead of research.

I love Eliezer's writing - the insight, the wit, the subversion. Over the years though, I've seen many comments from him that I found off-putting. Some of them, I've since decided, are probably net positive and I just happen to be in a subgroup that they don't work for (for example, I found Dying with Dignity discouraging, but saw enough comments that it had been helpful for people that I've changed my mind to think it was... (read 239 more words →)

How I'm telling my friends about AI Safety

k64

9mo

One of the comments on the new book post asked how to tell normie friends about AI safety. I don't have any special credentials here, but I thought it'd be worthwhile to share the facebook post I've drafted, both to get feedback and to give an example of one way a post could look. There exist articles and blogs that already do this well, but most people don't read shared articles and it's helpful to have a variety of ways to communicate. My goal here is to grab attention, diffuse densiveness with some humor, and try to make the problem digestable to someone who isn't immersed in the topic or lingo. Let... (read 1845 more words →)

Can you donate to AI advocacy?

k64

9mo

I posted a quick take that advocacy may be more effective than direct donation to alignment research. I am not an AI researcher and I'm not an influencer, so I'm not well positioned to do either. I see on the "How can I help" FAQ that there are options to donate, but they look like donating to research directly.

My question is: is there a way to donate to AI safety advocacy efforts? I'm also ok with donating to an organization or grantmaker that explicitly considers funding advocacy efforts. And of course, maybe I'm missing something, like advocacy being the type of thing you can't pay for, or some clear reason why AI safety advocacy will not be effective.

Note: Eliezer and Soares wrote a new book and say that pre-orders will help, so that's a way to donate $15 - $28 toward advocacy.

Edit: Based on a suggestion by Yaroslav, I also asked this question on the EA forum.

k64's Shortform

k64

9mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Unlike most other charitable causes, AI safety affects rich people. This suggests to me that advocacy may be a more effective strategy than direct funding.

Why Aren't Rationalists Winning (Again)

k64

9mo

Yes, this topic has been discussed multiple times. But at this point, long enough ago that my choices for joining the conversation are [Necro] or [New post], so here we are. (You can jump to the 2nd to last paragraph if you want to just see my conclusion.)

Background: Why am I asking this question?

After reading some comments I realize I should give some background on what I mean and why I'm asking this question.

Winning here refers to gaining utility, aka achieving your goals, whatever those goals may be, and is in reference to this post.

The reason for asking the question at all is that I personally expected a large improvement across

... (read 1420 more words →)

Minds are magic

k64

9mo

As we know, brains are incredibly complex connections of neurons. If physics is deterministic so are our brains. But that's not how we think about brains. We think about them as MINDS. And minds are neat and consistent.

There are a lot of benefits to thinking about brains as minds:

Studies have shown higher success for people with internal loci of control and for individuals with growth mindsets.
From personal experience, I have a much easier time making predictions about people when I think of them as minds, and some of those predictions even come close to reality.
Much of psychology and philosophy make use of the MIND model.

However, there are also drawbacks to... (read 346 more words →)

Any Trump Supporters Want to Dialogue?

k64

k64, Shankar Sivarajan

It's that time of year - the time when rationality seems increasingly scarce as political tensions rise. I find myself wishing I could have one of the people I see reaching super different conclusions shoot me with a POV gun so I could understand what it's like being on the other side.
I'm not strongly left-leaning, so I don't have trouble understanding why people may have some concerns about the left - but I have 0% support for Donald Trump, so if you want to explain to me why you think he's great, go for it. I also think that the election is close to 50/50 currently, so if you think it's... (read more)

Doing Nothing Utility Function

k64

One of the questions I've heard asked is "how do you design a utility function that would make the AI do nothing?" That is, how could we put a pause button on an AI so that we could pause it if we wanted to? I had an idea about how one could do this, and am sure it has already been thought of, so I'm curious to know why this doesn't end up working.

Why can't we just write something like:

If (paused) and not (taking actions): utility=PredictedUtilityIfNotPaused

If not (paused): utility=NormalUtilityFunction

Anti-Parfit's Hitchhiker

k64

Thinking about Parfit's Hitchhiker, an alternative example occurred to me:
You're lost in the desert and this time Aul Peckman drives up and tells you "I will give you a ride back to town iff you would have stiffed my nemesis Paul Eckman." After reading Parfit's Hitchhiker, you had pre-committed to pay Paul Eckman if this happened to you, or chosen a decision theory that would cause you to do that, so you try telling Aul Peckman that you would stiff his nemesis in this situation, but he knows you're lying and drives off. If only you weren't so timelessly rational!

Obviously, one can argue that you're more likely to encounter agents who will want to get paid than who will want you to not pay someone, and so if you're in a world where that is true, you still have positive EV from running TDT/UDT, but is this an example of regretting TDT rationality?

Is it rational to modify one's utility function?

k64

Rationality is often informally defined by means-end reasoning or utility maximization. However, this idea becomes less clear when faced with the option of modifying one's own utility function. Does rationality prescribe avoiding any change to one's current utility function because such a change would obviously reduce expected utility under the current function, or does it prescribe taking actions which result in the highest utility by whatever means necessary, in which case a change would be rational iff the new utility function yields higher expected utility given known background info about the world?

This is obviously relevant to AI alignment, where one concern is that AI may hack their own utility functions and another... (read more)

LESSWRONG
LW

LESSWRONG
LW

Why does Eliezer make abrasive public comments?

Can you donate to AI advocacy?

Any Trump Supporters Want to Dialogue?

Doing Nothing Utility Function

k64

k64

Why does Eliezer make abrasive public comments?

How I'm telling my friends about AI Safety

Can you donate to AI advocacy?

k64's Shortform

Why Aren't Rationalists Winning (Again)

Minds are magic

Any Trump Supporters Want to Dialogue?

k64

Why does Eliezer make abrasive public comments?

Can you donate to AI advocacy?

Any Trump Supporters Want to Dialogue?

Doing Nothing Utility Function

k64

k64

Why does Eliezer make abrasive public comments?

How I'm telling my friends about AI Safety

Can you donate to AI advocacy?

k64's Shortform

Why Aren't Rationalists Winning (Again)

Minds are magic

Any Trump Supporters Want to Dialogue?