Buddhism seems more about the reduction of suffering than the cultivation of happiness and joy. I believe the result of this isn't happiness, but rather stillness / peace of mind. This state is probably more postive than negative, but it still sounds like the inverse of manic depression, in that I expect both the highs and lows to be relatively close to zero.

I'd describe the opposite of buddhism to be the Dionysian (Nietzsche's concept of it, at least)

This book mystifies happiness, and tries to solve it from within the aspects of human nature which gatekeeps your happiness in order to prevent you from wireheading yourself. That you're actually under these constraints is merely what your mind wants you to think.

The only thing which prevents you from being happy all the time, is actually that you do not give yourself permission to be happy all the time, which is because you don't feel like it's justified. That emotions have to be justified, e.g. that you have to work for happiness as a reward, that you're only allowed to be happy when you do well or when things are good, is a defensive mechanism of the mind. You can do away with these mechanisms if you want, as long as you can convince your mind that the negative emotion is harming you, or that more positive emotion would aid your chances of survival (you likely enjoy the higher needs like self-expression, but the brain tends to focus on lower needs like security)

Your enemies are not innately evil

This quote is a lot like "You're not your experiences", "This too shall pass", "You're not the ego", "Nobody can make you feel bad without your permission", and this is because they're all insights into the self, which allows one to remove some of the illusions that the mind places itself under in order to create that punishment/reward loop in which most people live. I think these illusions are the mind making reinforcement learning with the environment possible. Of course, society has already subverted this system, for all the rewards are unhealthy superstimuli like porn, drugs, fats and sugars, ASMR, gambling, reaction videos and so on, so we might as well design a better system before we teach ourselves that inaction is the only good strategy).

The reason that happiness is a skill is because the mind fights back, and because there's many things to manipulate and many mechanisms to understand. When you try to lucid dream and the dream characters notice what you're doing, they usually attack you because your mind wants to prevent what you're doing. These are likely evolved defense mechanisms against wireheading behaviour (perhaps lucid dreaming prevents deep sleep, meaning that people who did it too much didn't pass on their genes). And when the mind fights back, it's likely because you know that you're doing something that you shouldn't, which means that your mind knows that you're doing something that you shouldn't. In other words, it is mostly subjective, and if you change your beliefs about what you're doing, your mind will change its response to what you're doing as well.

You don't need wisdom, you don't need to understand anything about life, you just need to understand your own brain, how to make agreements with yourself, and how your belief structure relates to your subjective experience of the world.

Compassion does not lead to happiness unless perhaps you're rewarding yourself for being a compassionate person, but your brain really likes the feeling of fitting into a group, and social relationships are a great source of meaning. It's seems that the brain uses similar mechanisms to simulate other people as it does to understand itself, meaning that people who experience others at a surface level also tends to experience themselves at a surface level (and vice versa). I can't explain it nearly as well as this article does, so I highly recommend at least skimming it.

This is basically all you need to know. If you want stronger, broader, more positive valence, you need to become more sensitive, meaning that you need to feel yourself, feel your body, feel other people, feel alive, feel immersed in different contexts, and to have beliefs (conscious or unconscious) which makes this valence more positive than negative (so that your mind doesn't choose numbness as a defense mechanism against it, harming your efforts to become sensitive).

Another thing I recommend is decreasing the abstract distance of the relation of negative experiences. If you feel bad when you eat meat because you know that farming animals suffer, and because you know that farming animals is necessary in order to produce the meat that you're eating, and because you dislike the idea of other beings suffering, we could call this a distance of 3-4 steps. The more steps you allow negative things to spread, the less you will enjoy reality, as your mind will fight against them by infusing them with negative valence. The less distance, the more you can enjoy things. The same is true across time - live in the moment and you will likely experience that nothing is wrong, but simulate 2 months of the past and 2 months of possible future consequences of each choice you make, and you will find that most actions will feel bad: "I'm wasting time", "If I do this good thing, they will just expect more good things of me in the future", "My current behaviour is not consistent with the other behaviour I've shown these exact people, I need to conform to their model of me", etc.

All of these pain-in-the-ass restrictions aren't as real as they look. If you feel bad, it's ultimately because you choose to feel bad, even if your brain coerced you into making that choice. If you want to feel good, just do so.

Edit: I used to be interested in lucid dreaming, so I'd read about it on online forums. Breaking the 4th wall seemed to end badly for the vast majority of people, but there's some level of "dream characters react how you expect them to react" involved as well. Everything else is just what I learn naturally, or my own original research. I'd also love to read scientific papers about these things if any exist.  

Same here. I think it's because working on myself made me neglect my connection to everyday life. Working too much on yourself, in your own way, makes you a little bit incompatible with everything and everyone else. Wizard power (as I see it: A powerful mentality, independence, the ability to go ahead of everyone else) is best developed in isolation or smaller groups, but social connections are important for success in life.

The feeling which caused me to switch was that of a pyramid being upside down. If you work on top of the needs hierarchy while neglecting the bottom like I did, you're putting yourself at risk, and this creates anxiety which is justified and thus hard to shake off again. 

In my way of looking at this, Einstein had a bit of wizard power, and he had trouble getting recognized for it once he made his discoveries. Most people did not have much reason to believe that he was a genius, as he didn't have a lot of social proof. Tesla also leaned too much towards his intellectual pursuits, I think. He didn't have much in terms of money and friends, and this seemed to cause him difficulties even though he was such a competent person.

An alternative route I've thought of is becoming a social wizard. Those people who have an unnatural amount of charisma and the ability to read people like books.

About being nerd sniped - I think many things do bring some benefits. The problem is, even though you can do anything, you can not do everything. There simply isn't enough time. I like this quote, attributed to Leonardo da Vinci: "As every divided kingdom falls, so every mind divided between many studies confounds and saps itself."

A thing which I can't put into words well but only warn about vaguely is the consequence of learning many new things too fast. When I do this, I face the same thing as OP does - I basically become another person. If I want to get back, I have to reverse my mindset, attitude, values, cognitive models, priorities, etc. and not just remember what I learned before, so for me, learning isn't purely additive unless the new material is close to what I learned before. Even switching from "work mode" to "socializing mode" takes me a few hours at least.

I agree with the top part. I think it's naive to believe that AI is helping anyone, but what I want to talk about is why this problem might be unsolvable (except by avoiding it entirely). 

If you hate something and attempt to combat it, you will get closer to it rather than further away, in the manner which people refer to when they say "You actually love what you say you hate". When I say "don't think about pink elephants", the more you try, the more you will fail, and this is because the brain doesn't have subtraction and division, but only addition and multiplication.

You cannot learn about how to defend yourself against a problem without learning how to also cause the problem. When you learn self-defense you will also learn attacks. You cannot learn how to argue effectively with people who hold stupid worldviews without first understanding them and thus creating a model of the worldview within yourself as well.

Due to mechanics like these, it may be impossible to research "AI safety" in isolation. It's probably better to use a neutral word like "AI capabilities" which include both the capacity for harm and defense against harm so that we don't mislead ourselves with words. It can cause untold damage, much like viewing "good and evil" as opposites, rather than two sides of the same thing, has.

I also want to warn everyone that there seems to be an asymmetry in warfare which makes it so that attacking is strictly easier than defending. This ratio seems to increase as technology improves.

I don't think it's possible without changing the people into weird types who really don't care too much about the social aspects of life because they're so interested in the topics at hand. You can try rewarding truth, but people still stumble into issues regarding morality, popularity of ideas, the overton window, some political group that they dislike randomly hitting upon the truth so that they look like supporters for stating the same thing, etc.

I think prediction markets are an interesting concept, but it cannot be taken much further than it is now, since the predictions could start influencing the outcomes. It's dangerous to add rewards to the outcomes of predictions, for when enough money is involved, one can influence the outcome.

The way humans in general differ from truth-seeking agents makes their performance downright horrible on some specific areas (if the truth is not in the overton window for instance). These inaccuracies can cascade and cause problems elsewhere, since they cause incorrect worldviews even in somewhat intelligent people like Musk. There's also a lot of information which is simply getting deleted from the internet, and you can't "weight both sides of the argument" if half the argument is only visible on the waybackmachine or archive.md.

I guess it's important to create a good atmosphere and that everyone is having fun theorizing and such, but some of the topics we're discussing are actually serious. The well-being of millions of people depend on the sort of answers and perspectives which float around puvlic discourse, and I find it pathetic that ideas are immediately shut down if they're not worded correctly or if they touch a growing list of socially forbidden hypotheses.

Finally, these alternative rewards have completely destroyed almost all voting systems on the internet. There's almost no website left on which the karma/thumb/upvote/like count bears any resemblence to post quality anymore. Instead, it's a linear combination of superstimuli like 'relatability', 'novelty', 'feeling of importance (e.g. bad news, danger)', 'cuteness', 'escapism', 'sexual fantasy', 'romantic fantasy', 'boo outgroup', 'irony/parody/parody of parody/self-parady/nihilism', 'nostalgia', 'stupidity' (I'm told it's a kind of humor if you're stupid on purpose, but I think "irony" is a defence mechanism against social judgement). It's like a view into the unfulfilled needs of the population. Youtube view count and subscriptions, Reddit karma, Twitter retweets, all almost gamed to the point that they're useless metrics. Online review sites are going in the same direction. It's like interacting with a group of mentally ill people who decide what you're paid each day. I think it's dangerous to upvote comments based on vibes as it takes very little to corrupt these metrics, and it's hard to notice if upvotes gradually come to represent "dopamine released by reading" or something other than quality/truthfulness.

My previous criticism was aimed at another post of yours, it likely wasn't your main thesis. Some nitpicks I have with it are:

"Developing AGI responsibly requires massive safeguards that reduce performance, making AI less competitive" you could use the same argument for AIs which are "politically correct", but we still choose to take this step, censorsing AIs and harming their performance, thus, it's not impossible for us to make such choices as long as the social pressure is sufficiently high.

"The most reckless companies will outperform the most responsible ones" True in some ways, but most large companies are not all that reckless at all, which is why we are seeing many sequels, remakes, and clones in the entertainment sector. It's also important to note that these incentives have been true for all of human nature, but that they've never mainfested very strongly until recent times. This suggests that that the antidote to Moloch is humanity itself, good faith, good taste and morality, and that these can beat game theoritical problem which are impossible when human beings are purely rational (i.e. inhuman). 

We're also assuming that AI becomes useful enough for us to disregard safety, i.e. that AI provides a lot of potential power. So far, this has not been true. AIs do not beat humans, companies are forcing LLMs into products but users did not ask for them. LLMs seem impressive at first, but after you get past the surface you realize that they're somewhat incompetent. Governments won't be playing around with human lives before these AIs provide large enough advantages.

"The moment an AGI can self-improve, it will begin optimizing its own intelligence."
This assumption is interesting, what does "intelligence" mean here? Many seems to just give these LLMS more knowledge and then call them more intelligent, but intelligence and knowledge are different things. Most "improvements" seem to lead to higher efficiency, but that's just them being dumb faster or for cheaper. That said, self-improving intelligence is a dangerous concept.

I have many small objections like this to different parts of the essay, and they do add up, or at least add additional paths to how this could unfold.

I don't think AIs will destroy humanity anytime soon (say, within 40 years). I do think that human extinction is possible, but I think it will be due to other things (like the low birthrate and its economic consequences. Also tech. Tech destroys the world for the same reasons that AIs do, it's just slower).

I think it's best to enjoy the years we have left instead of becoming depressed. I see a lot of people like you torturing themselves with x-risk problems (some people have killed themselves over Roko's basilisk as well). Why not spend time with friends and loved ones?

Extra note: There's no need to tie your identity together with your thesis. I'm the same kind of autistic as you. The futures I envision aren't much better than yours, they're just slightly different, so this is not some psychological cope. People misunderstand me as well, and 70% of the comments I leave across the internet get no engagement at all, not even negative feedback. But it's alright. We can just see problems approaching many years before they're visible to others. 

I've read some of your other replies on here and I think I've found a pattern, but it's actually more general than AI. 

Harmful tendencies outcompete those which aren't harmful

This is true (even outside of AI), but only at the limit. When you have just one person, you cannot tell if he will make the moral choice or not, but "people" will make the wrong choice. The harmful behaviour is emergent at scale. Discrete people don't follow these laws, but the continous person does.

Again, even without AGI, you can apply this idea to technology and determine that it will eventually destroy us, and this is what Ted Kaczynski did. Thinking about incentives in this manner is depressing, because it feels like everything is deterministic and that we can only watch as everything gets worse. Those who are corrupt outcompete those who are not, so all the elites are corrupt. Evil businessmen outcompete good businessmen, so all successful businessmen are evil. Immoral companies outcompete moral companies, so all large companies are immoral.

I think this is starting to be true, but it wasn't true 200 years ago. At least, it wasn't half as harmful as it is now, why? It's because the defense against this problem is human taste, human morals, and human religions. Dishonesty, fraud, selling out, doing what's most efficient with no regard for morality. We consider this behaviour to be in bad taste, we punished it and branded it low-status, so that it never succeeded in ruining everything.

But now, everything could kill us (if the incentives are taken as laws, at least), you don't even need to involve AI. For instance, does Google want to be shut down? No, so they will want to resist antitrust laws. Do they want to be replaced? No, so they will use cruel tricks to kill small emerging competitors. When fines for illegal behaviour are less than the gains Google can make by doing illegal things, they will engage in illegal behaviour, for that is the logical best choice available to Google if all which matters is money. If we let it, Google would take over the world, in fact, it couldn't do otherwise. You can replace "Google" with any powerful structure in which no human is directly in charge. When it starts being more profitable to kill people than it is to keep them alive, the global population will start dropping fast. When you optimize purely for Money, and you optimize strongly enough, everyone dies. An AI just kills us faster because it optimizes more strongly, we already have something which acts similarly to AI. If you optimize too hard for anything, no matter what it is (even love, well-being, or happiness), everyone eventually dies (hence the paperclip maximizer warning).

If this post gave you existential dread, I've been told that Elinor Ostrom's books make for a good antidote.

A lot of the words we use are mathematical and thus more precise and with less connotations that people can misunderstand. This forum has a lot of people with STEM degrees, so they use a lot of tech terms, but such vocab is very useful for talking about AI risk. The more precise language is used, the less misunderstandings can occur.

Moloch describes a game theory problem, and these problems generally seem impossible to solve. But even though they're not possible to solve mathematically doesn't mean that we're doomed (I've posted about this on here before but I don't think anyone understood me. In short, game theory problems only play out when certain conditions are met, and we can prevent these conditions from becoming true).

I haven't read all your posts from end to end but I do agree with your conclusions that alignment is impossible and that AGI will result in the death or replacement of humanity. I also think your conclusions are valid only for LLMs which happen to be trained on human data. Since humans are deceptive, it makes sense that AIs training on them are as well. Since humans don't want to die, it makes sense that AIs trained on them also don't want to die. I find it unlikely that the first AGI we get is a LLM since I expect it to be impossible LLMs to improve much further than this.

I will have to disagree that your post is rigorous. You've proven that human errors bad enough to end society *could* occur, not not that they *will* occur. Some of your examples have many years between them because these events are infrequent. I think "There will be a small risk of extinction every year, and eventually we will lose the dice throw" is more correct.
Your essay *feels* like it's outlining tendencies in the direction of extinction, showing transitions which look like the following:
A is like B
A has a tendency for B
For at least some A, B follows.
If A, then B occurs with nonzero probability.
If A, then we cannot prove (not B).
If A, then eventually B.

And that if you collect all of these things in to directed acyclic graph, that there's a *path* from our current position to an extinction event. I don't think you've proven that each step A->B will be taken, and that it's impossible with a probability of 1 to prevent it (even if it's impossible to prevent it with a possibility of 1, which is a different statement)
I admit that my summary was imperfect. Though, if you really believe that it *will* happen, why are you writing this post? There would be no point in warning other people if it was necessarily too late to do anything about it. If you think "It will happen, unless we do X", I'd be interested in hearing what this X is.

I'm afraid "Good at presenting their ideas in a persuasive manner" is doing all the heavy lifting here.

If the community had a good impression of him, they'd value his research over that of a PhD. If the community had a bad impression of him, they'd not give a second of thought towards his "research" and they would refer to it with the same mocking quotation marks that I just used. However, in the latter case, they'd find it more difficult to dismiss his PhD.

In other words, the interpretation depends if the community likes you or not. I've been in other rationalist communities and I'm speaking from experience (if I'm less vague that this, I'd be recognizable, which I don't want to be). I saw all the negative social dynamics that you'd find on Reddit or in young female friend groups with a lot of "drama" going on, in case you're unfortunate enough to have an intuition for such a thing.

In any "normie" community there's the staff in charge, and a large number of regular users who are somewhat above the law, and who feel superior to new users (and can bully them all they want, as they're friends with the staff). The treatment of users users depend on how well they fit in culturally, and it requires that they act as if the regulars are special (otherwise their ego is hurt). Of course, some of these effects are borderline invisible on this website, so they're either well-hidden or kept in check.

Still, this is not a truth-maximizing website, the social dynamics and their false premises (e.g. the belief that popularity is a measure of quality) are just too strong. The sort of intllectuals who don't care about social norms, status or money are better at truth-seeking and generally received poorly by places like this.

First, a few criticisms which I feel are valid:

1: Your posts are quite long.

2: You use AI in your posts, but AIs aren't able to produce high enough quality that it's worth posting.

3: Some of your ideas have already been discovered before and have a name on here. "Moloch" for instance is the personification of bad nash equilibriums in game theory. It generally annoys people if you don't make yourself familiar with the background information of the community before posting, but it's a lot of work to do so.

Your conclusion is correct, but it boils down to very little: "greedy local optimization can destroy society". People who already know that likely don't want to read 30 pages which makes the same point. "Capitalism" was likely the closest word you knew, but there's many better words, and you sadly have to be a bit of a nerd to know a lot of useful words.

Here's where I think you're right:

This is not a individualist website for classic nerds with autism who are interested in niche topics, it's a social and  collectivism community for intellectual elites who care about social status and profits.

Objective truth is not of the highest value.
Users care about their image and reputation.
Users care about how things are interpreted (and not just what's written).
Users are afraid of controversies. A blunt but correct answer might net you less karma than a wrong answer which shows good-will.
Users value form - how good of a writer you are will influence the karma, regardless of how correct or valuable your idea is. Verbal intelligence is valued more than other forms.
The userbase has a left-wing bias, and as does the internet (as if about 8 years ago), so you can find lots of sources which argue in favor of things which are just objectively not true. But it's often difficult to find a source which disproves the thing, as they're burried. Finally, as a social website, people value authority and reputation/prestige, and it's likely that the websites they feel are "trustworthy" only include those controlled by left-wing elites.
Users value knowledge more than they value intelligence. They also value experience, but only when some public institution approves of it. They care if you have a PhD, they don't care if you have researched something for 5 years in your own free time.
 

You're feeling the consequences of both. I think most of the negative reaction comes from my first 3 points, and that the way it manifests is a result of the social dynamics.