LWLW

Hey Steven this is unrelated but I wanted to say I really appreciate your posts and comments here! 

I’ve found the best way to get out of philosophical rabbit holes is to spend more time living. It provides far more reassurance and wisdom than spending all day trying to solve the problem of evil. I think Hume found something similar and that’s deeply reassuring to me.

The idea of a superintelligence having an arbitrary utility function doesn’t make much sense to me. It ultimately makes the superintelligence a slave to its utility function which doesn’t seem like the way a superintelligence would work.

Has anybody checked if finetuning LLMs to have inconsistent “behavior” degrades performance? Like you finetuned a model on a bunch of aligned tasks like writing secure code and offering compassionate responses to individuals in distress, but then you tried to specifically make it indifferent to animal welfare? It seems like that would create internal dissonance in the LLM which I would guess causes it to reason less effectively (since the character it’s playing is no longer consistent).

My guess is that finetuning an LLM turns it into a p-zombie. I don’t think the architecture is complicated enough to support consciousness. There’s zero capacity for choice involved, which seems to be what consciousness is all about.

I think if one could formulate concepts like peace and wellbeing mathematically, and show that there were physical laws of the universe implying that eventually the total wellbeing in the universe grows monotonically positively then that could show that certain values are richer/“better” than others.

If you care about coherence then it seems like a universe full of aligned minds maximizes wellbeing while still being coherent. (This is because if you don’t care about coherence you could just make every mind infinitely joyful independent of the universe around it, which isn’t coherent).

but if human intelligence and reasoning can be picked up from training, why would one expect values to be any different? the orthogonality thesis doesn't make much sense to me either. my guess is that certain values are richer/more meaningful, and that more intelligent minds tend to be drawn to them.

and you can sort of see this with ASPD and NPD. they're both correlated with lower non-verbal intelligence! and ASPD is correlated with significantly lower non-verbal intelligence.

and gifted children tend to have a much harder time with the problem of evil than less gifted children do! and if you look at domestication in animals, dogs and cats simultaneously evolved to be less aggressive and more intelligent at the same time.

contra the orthogonality thesis.

if you want to waste a day or two, try to find an eminent mathematician or physicist who had NPD or ASPD. as far as i can tell, i haven't been able to find any successful ones who had either disorder.

as far as the research goes, ASPD is correlated with significantly lower non-verbal intelligence. and in one study i found, NPD wasn't really correlated with any parts of intelligence except with lower non-verbal intelligence.

which can lead to the idea that everbody starts out aligned, and then when those with less cognitive reserve are confronted with trauma, more serious misalignment/personality disorders can arise.

even if you look at kids with ASPD who try to murder their siblings when they're two, most of the time it's a younger sibling. which can be explained by the kid being jealous that they get less attention than their younger sibling. which can be entirely explained by a lower (emotional) pain tolerance.

I’m sorry are you really saying you’d rather have Ted bundy with a superintelligent slave than humanity’s best effort at creating a value-aligned ASI? You seem to underestimate the power of generalization.

If an ASI cares about animal welfare, it probably also cares about human welfare. So it’s presumably not going to kill a bunch of humans to save the animals. It’s an ASI, it can come up with something cleverer. 

Also I think you underestimate how devastating serious personality disorders are. People with ASPD and NPD don’t tend to earn flourishing lives for themselves or others.

Also, if a model can pick up human reasoning patterns/intelligence from pretraining and RL, why can’t it pick up human values in its training as well?

This just seems incoherent to me. You can’t have value-alignment without incorrigibility. If you’re fine with someone making you do something against your values, then they aren’t really your values.

So it seems like what you’re really saying is that you’d prefer intent-alignment over value-alignment. To which I would say your faith in the alignment of humans astounds me.

Like is it really safer to have a valueless ASI that will do whatever its master wants than an incorrigible ASI that cares about animal welfare? What do you expect the people in the Epstein files to do with an ASI/AGI slave? 

A value-aligned ASI completely solves the governance problem. If you have an intent-aligned ASI then you’ve created a nearly impossible governance problem.

I think a lot of people are confused by good and courageous people and don’t understand why some people are that way. But I don’t think the answer is that confusing. It comes down to strength of conscience. For some people, the emotional pain of not doing what they think is right hurts them 1000x more than any physical pain. They hate doing what they think is wrong more than they hate any physical pain. 

So if you want to be an asshole, you can say that good and courageous people, otherwise known as heroes, do it out of their own self-interest.

Is intology a legitimate research lab? Today they talked about having an AI researcher that performed better than humans on RE-bench at 64 hr time horizons. This seems really unbelievable to me. The AI system is called Locus.

I just can’t wrap my head around people who work on AI capabilities or AI control. My worst fear is that AI control works, power inevitably concentrates, and then the people who have the power abuse it. What is outlandish about this chain of events? It just seems like we’re trading X-risk for S-risks, which seems like an unbelievably stupid idea. Do people just not care? Are they genuinely fine with a world with S-risks as long as it’s not happening to them? That’s completely monstrous and I can’t wrap my head around it.  The people who work at the top labs make me ashamed to be human. It’s a shandah.

This probably won’t make a difference, but I’ll write this anyways. If you’re working on AI-control, do you trust the people who end up in charge of the technology to wield it well? If you don’t, why are you working on AI control?

Fun Fact of the Day: Kanye West’s WAIS is within two points of a fields medalist’s (the fields medalist is Richard Borcherds, their respective IQs are 135 and 137).

Extra Fun Fact: Kanye West was bragging about this to Donald Trump in the Oval Office. He revealed that his digit span was only 92.5 (which is what makes me think he actually had a psychologist-administered WAIS).

Extra Extra Fun Fact: Richard Borcherds was administered the WAIS-R by Sacha Baron Cohen's first cousin.