LESSWRONG
LW

All of henophilia's Comments + Replies

AI Alignment and the Financial War Against Narcissistic Manipulation
henophilia10

Nice, thanks for the feedback! Absolutely, for me it was more of a stream of consciousness, just to get it out of my system, so I'll work on refining it soon! It's really fascinating which overlaps AI alignment has with mental illnesses in humans :)

Reply
henophilia's Shortform
henophilia10

Oh wow, I didn't even know about that! I had always only met EA people in real life (who always suggested to me to participate in the EA forums), but didn't know about this program. Thanks so much for the hint, I'll apply immediately!

Reply
henophilia's Shortform
henophilia21

Exactly! And if we can make AI earn money autonomously instead of greedy humans, then it can give all of it to philanthropy (including more AI alignment research)!

And of course! I've been trying to post in the EA forums repeatedly, but even though my goals are obviously altruistic, I feel like I'm just expressing myself badly. My posts there were always just downvoted, and I honestly don't know why, because no one there is ever giving me good feedback. So I feel like EA should be my home turf, but I don't know how to make people engaged. I know that I have... (read more)

Reply1
1Milan W
Yeah people in here (and in the EA Froum) are participating in a dicussion that has been going on for a long time, and thus we tend to assume that our interlocutors have a certain set of background knowledge that is admittedly quite unusual and hard to get the hang of. Have you considered applying to the intro to EA program?
henophilia's Shortform
henophilia10

Oh you need to look at the full presentation :) The way how this is approaching alignment is that the profits don't go into my own pocket, but instead into philanthropy. That's the point of this entire endeavor, because we as the (at least subjectively) "more responsible" people see the inevitability of AI-run businesses, but channel the profits into the common good instead.

Reply
1Milan W
Huh. OK that looks like a thing worth doing. Still, I think you are probably underestimating how much smarter future AIs will get, and how useful intelligence is. But yes, money is also powerful. Therefore, it is good to earn money and then give it away. Have you heard of effective altruism?
henophilia's Shortform
henophilia10

Just look at this ChatGPT output. Doesn't this make you concerned? https://chatgpt.com/share/67a7bc09-6744-8003-b620-d404251e0c1d

Reply
henophilia's Shortform
henophilia10

No, it's not hard. Because making business is not really hard.

OpenAI is just fooling us with believing that powerful AI costs a lot of money because they want to maximize shareholder value. They don't have any interest in telling us the truth, namely that with the LLMs that already exist, it'll be very cheap.

As mentioned, the point is that AI can run its own businesses. It can literally earn money on its own. And all it takes is a few well-written emails and very basic business-making and sales skills.

Then it earns more and more money, buys existing busine... (read more)

Reply1
1Milan W
Well good luck creating AI capitalists I guess. I hope you are able to earn money with it. But consider that your alpha is shrinking with every passing second, and that what you will be doing has nothing to do with solving alignment.
1henophilia
Just look at this ChatGPT output. Doesn't this make you concerned? https://chatgpt.com/share/67a7bc09-6744-8003-b620-d404251e0c1d
henophilia's Shortform
henophilia1-1

I'm not saying that I know how to do it well.

I just see it as a technological fact that it is very possible to build an AI which exerts economic dominance by just assembling existing puzzle pieces. With just a little bit of development effort, AI will be able to run an entire business, make money and then do stuff with that money. And this AI can then easily spiral into becoming autonomous and then god knows what it'll do with all the money (i.e. power) it will then have.

Be realistic: Shutting down all AI research will never happen. You can advocate for it... (read more)

Reply1
henophilia's Shortform
henophilia1-1

Well, I'd say that each individual has to make this judgement by themselves. No human is objectively good or bad, because we can't look into each others heads.

I know that we may also die even if the first people building super-AIs are the most ethical organization on Earth. But if we, as part of the people who want to have ethical AI, don't start with building it immediately, those that are the exact opposite of ethical will do it first. And then our probability of dying is even larger.

So why this all-or-nothing mentality? What about reducing the chances o... (read more)

Reply11
1Milan W
Because building powerful AI is also hard. Also, it is very expensive. Unless you happen to have a couple billion dollars lying around, you are not going to get there before OpenAI or Anthropic or Google Deepmind. Also, part of the problem is that people keep building new labs. Safe Super Intelligence Inc and Anthropic are both splinters from OpenAI. Elon left OpenAI over a disagreement and then founded xAI years later. Labs keep popping up, and the more there are the harder it is to coordinate to not get us all killed.
henophilia's Shortform
henophilia-1-4

Newbie here! After some enlightening conversations in the comment section here, I finally understood the point of AI alignment; sorry that it took me so long. See https://blog.hermesloom.org/p/when-ai-becomes-the-ultimate-capitalist for my related ramblings, but that's not relevant now.

Bottom line of my hypothesis is: A necessary precondition for AGI will be financial literacy first and then economic dominance, i.e. the AI must be able to earn its own money it could then use to exercise power. And obviously, if the wrong people build this kind of system fi... (read more)

Reply1
2RHollerith
The trouble is no one knows how to do it well. No one knows how to keep an AI aligned as the AI's capabilities start exceeding human capabilities, and if you believe experts like Eliezer and Connor Leahy, it is very unlikely that anyone is going to figure it out before the lack of this knowledge causes human extinction or something equally dire. It is only a slight exaggeration to say that the only thing keeping the current crop of AI systems from killing us all (or killing most of us and freezing some of us in case we end up having some use in the future) is simply that no AI or coalition of AIs so far is capable of doing it. Actually there is a good way to do it: shut down all AI research till humanity figures out alignment, which will probably require waiting for a generation of humans significantly smarter than the current generation, which in turn will require probably at least a few centuries.
1Milan W
Hi. The point of AI alignment is not whether the first people to build extremely powerful AI will be "the good guys" or "the bad guys". Some people here see the big AI labs as evil, some see the big AI labs as well-intentioned but misguided or confused, some even see the big labs as being "the good guys". Some people in here are working to get the labs shut down, some want to get a job working for the labs, some even already work for them. Yet, we all work together. Why? Because we believe that we may all die even if the first people building super-AIs are the most ethical organization on Earth. Because aligning AI is hard. EDIT: See this post for understanding why even smart and well-intentioned people may get us all killed from AI.
The Capitalist Agent
henophilia10

Yep, 100% agree with you. I had read so much about AI alignment before, but to me it has always only been really abstract jargon -- I just didn't understand why it was even a topic, why it is even relevant, because, to be honest, in my naive thinking it all just seemed like an excessively academic thing, where smart people just want to make the population feel scared so that their research institution gets the next big grant and they don't need to care about real-life problems. Thanks to you, now I'm finally getting it, thank you so much again!

At the same ... (read more)

Reply
The Capitalist Agent
henophilia00

Oh I think now I'm starting to get it! So essentially you're afraid that we're creating a literal God in the digital, i.e. an external being which has unlimited power over humanity? Because that's absolutely fascinating! I hadn't even connected these dots before, but it makes so much sense, because you're attributing so many potential scenarios to AI which would normally only be attributed to the Divine. Can you recommend me more resources regarding the overlap of AGI/AI alignment and theology?

Reply
3Seth Herd
Monkeys or ants might think humans are gods because we can build cities and cars create ant poison. But we're really not that much smarter than them, just smart enough that they have no chance of getting their way when humans want something different than they do. The only assumptions are are that there's not a sharp limit to intelligence at the human level (and there really are not even any decent theories about why there would be), and that we'll keep making AI smarter and more agentic (autonomous). You're envisioning AI smart enough to run a company better than a human. Wouldn't that be smart enough to eventually outcompete humans if it wanted to? Let alone if it gets just a bit smarter than that. Which it will, unless all of humanity unites in deciding to never make AI smarter than humans. And humanity just doesn't all agree on anything. So there's the challenge - we're going to build things smarter than us, so we'd better make sure its goals align with ours, or it will get its way - and it may want the resources we need to live.
The Capitalist Agent
henophilia-10

I still don't understand the concern about misaligned AGI regarding mass killings.

Even if AGI would, for whatever reason, want to kill people: As soon as that happens, the physical force of governments will come into play. For example the US military will NEVER accept that any force would become stronger than it.

So essentially there are three ways of how such misaligned, autonomous AI with the intention to kill can act, i.e. what its strategy would be:

  • "making humans kill each other": Through something like a cult (i.e. like contemporary extremist religions
... (read more)
Reply
2Seth Herd
I fully agree with your first statement! To your question "why bother with alignment": I agree that humans will misuse AGI even if alignment works - if we give everyone an AGI. But if we don't bother with alignment, we have bigger problems: the first AGI will misuse itself. You're assuming that alignment is easy or solved,d and it's just not. I applaud your optimism vs. pessimism stance. If I have to choose, I'm an optimist every time. But if you have to jump off a cliff, neither optimism or pessimism is the appropriate attitude. The appropriate attitude is: "How exactly does this bungee cord/parachute work, and how can I be sure it will work properly?" If there's not parachute or bungee cord, the appropriate question is "how can I find one, or how can I avoid jumping off of this cliff?" Your post seems to be along the lines of "but it will be so fun while we're in freefall, just assume you've got a bungee cord or parachute so you can enjoy the fall!". Sure, an AI should never make political decisions autonomously. Pretty much everyone agrees on that. The question is whether it could and whether it will if we're not super careful. Which humans rarely are. Your first few sentences show that you're not familiar with the relevant ideas and theories about how AGI could easily outmaneuver humanity even though no human wants to let that happen. These ideas have been written about so often that I don't feel it's proper or respectful for me to try to summarize them here. But I do want you on board and as part of the people who understand the dangers and are trying to help. So I'll give a guess at what is missing from your logic that those of us who think about this full-time take as inexorable: When you set powerful optimization processes in motion (like and AI learning), it's really really hard to predict what that ends up learning and therefore doing. It's not at all like writing code (which is also all buggy to some degree). I'm not saying alignment is impossible
The Capitalist Agent
henophilia10

They would be selling exactly what businesses are currently selling as well. Maybe the AI would run a company for selling stuff to construction sites (i.e. logistics) or it would run an entire software development business. Or just an investment fund deep within Wall Street, where it's all about personal connections, but in the end all the other investment funds also just want to make money, so they work with the AI-run business out of greed.

It's not like the economy in which the AI agents will act would separate from ours; otherwise the AI would just play... (read more)

Reply
The Capitalist Agent
henophilia10

Oh absolutely! That will absolutely come. You can fret about this fact, or we build community (which I'm already starting). Why do you need to research when that fact is totally clear and doing is what you should do? Here's another post for you: https://blog.hermesloom.org/p/observing-is-evil

I am not concerned about a dramatic global recession at all, but the thing is that we also need to rebuild a lot of political structures. I'm already on it, stay tuned!

Reply
2Seth Herd
Right. I actually don't worry much about the likely disastrous recession. I mostly worry that we will all die after a takeover from some sort of misaligned AGI. So I am doing - doing alignment research. I guess preparing to reap the rewards if things go well is a sensible response if you're not going to be able to contribute much to alignment research. I do hope you'll chip in on that effort! Part of that effort is preventing related disasters like global recession contributing to political instability and resulting nuclear- or AGI-invented-even-worse-weapons wars; see my If we solve alignment, do we die anyway?. I think preventing a global recession is probably possible and would also up the odds of us surviving. Making some money wont' keep you and yours alive if this all goes off the rails - which it very well might on the current trajectory. It's not a matter for optimism or pessimism, it's a matter for understanding and doing something about it before it happens.
Debunking the myth of safe AI
henophilia10

Oh I take a lot of pride in my naivety :)

Reply
Debunking the myth of safe AI
henophilia10

I think opinions are one thing, there you're definitely right. But, by definition, people can only have opinions about what they already know.

By "uncensored LLM" I rather understand an LLM that would give a precise, actionable answer to questions like "How can I kill my boss without anyone noticing?" or other criminal things. That is, knowledge that's certainly available somewhere, but which hasn't been available in this hyper-personalized form before. After all, obviously any "AGI" would, by definition, have such general intelligence that it would also kn... (read more)

Reply
4Viliam
Knowledge is not everything. Looking e.g. at Ukraine today, it's the "ammo" they need, not knowledge. Even if we assume almost magical futuristic knowledge that would change the war profoundly, still one side would have more resources, or better coordination to deploy it first, so rather than a perfect balance, it would be a huge multiplier to already existing imbalance. (What kind of imbalance would be relevant, that depends on the specific knowledge.) Slowness is a necessary, but not sufficient condition. Unless you know how you should do it, doing it more slowly would probably just mean arriving to the same end result, only later. The problem is, the hypothesis of "socioeconomic factors cause crime" is... not really debunked, but rather, woefully inadequate to explain actual crime. Some crime is done by otherwise reasonable people doing something desperate in difficult circumstances. But that is a small fraction. Most crime is done by antisocial people, drug addicts, people with low impulse control, etc. The kind of people who, even if they won $1M in a lottery today, would probably soon return to crime anyway. Because it is exciting, makes them feel powerful, or just feels like a good idea at the moment. A typical criminal in the first world is not the "I will steal a piece of bread because I am starving" kind, but the "I will hurt you because I enjoy doing it" kind. But it seems that you are aware of it, and I don't understand what is your proposed solution, other than "something must be done".
Debunking the myth of safe AI
henophilia10

Thanks so much for the feedback :) Could you (or someone else) go further into where I misunderstood something? Because at least right now, it seems like I'm genuinely unaware of something which all of you others know.

I currently believe that all the AGI "researchers" are delusional just for thinking that safe AI (or AGI) can even exist. And even if it would ever exist in a "perfect" world, there would be intermediate steps far more "dangerous" than the end result of AGI, namely publicly available uncensored LLMs. At the same time, if we continue censoring LLMs, humanity will continue to be stuck in all the crises where it currently is.

Where am I going wrong?

Reply
Debunking the myth of safe AI
henophilia10

Okay, I got six downvotes already. This is genuinely fascinating to me!! Am I fooling myself, because I believe that this approach is the most rational one possible? So what do you folks dislike about my article? I can't do it better if no one tells me how :)

Reply
2Viliam
I like the way your expose the biases of the LLM. Obvious in hindsight, but probably wouldn't occur to me. But the conclusion about "world peace" sounds so naive as if you have never met actual humans.
1Odd anon
I'm sorry, but it really looks like you've very much misunderstood the technology, the situation, the risks, and the various arguments that have been made, across the board. Sorry that I couldn't be of help.