Viliam

Replying to"Recursive Self-Improvement" Is Three Different Things

"Recursive Self-Improvement" Is Three Different Things

Just nitpicking, but I would classify self-play as a subset of R&D. The AI is exploring some... space... by experimenting and learning, it's just a very narrow space.

-2

Replying toCoping with Deconversion

Viliam9h

Coping with Deconversion

Why should anything matter if we're just arbitrary self-reinforcing arrangements of chemicals?

Why should anything matter if we were something else?

(This is probably not helping, but... the thing you imagine that you have lost, perhaps you have never had it.)

Suppose that you learn that instead of being chemicals, we are actually particles of a fart of a great god Gugulag. Why specifically should learning this information make you feel happier, and your life more meaningful?

Is it because you think that Gugulag is cool (after all, your entire village worships him), so being connected to him in any way makes you cool? So, basically, is this about your perception of your own social status?

Or... (read more)

Viliam9hQuick Take

If Hitler is our default example for "someone who killed (caused to die) a lot of people on purpose", who could be our default example for "someone who killed (caused to die) a lot of people by negligence"?

Disclaimer: I am not implying that Hitler killed more people that Stalin or Mao, etc. I am just saying that he is a convenient example; if you mention him, people know what you meant. I am looking for similarly convenient examples of criminal negligence.

(Inspired by this thread, but didn't want to go off topic.)

Replying toDoes focusing on animal welfare make sense if you're AI-pilled?

Viliam10h

Does focusing on animal welfare make sense if you're AI-pilled?

Thanks for the explanation, but I am not completely satisfied with it. You can explain a lot of what humans do by functional advantages, so... to put it bluntly, is it morally okay to torture people who are insufficiently introspective? Are Buddhist meditators utility monsters?

Replying toIs a constitution a "noble lie"?

Viliam10h

Is a constitution a "noble lie"?

The historically most famous example was "The proletarians have nothing to lose but their chains". (It turned out that they actually had something to lose, such as their lives, and the lives of their families.)

Or, if you ask a random adolescent about how they would fix some frustrating aspect of the world, the answer would usually make things much worse, if you think about it for five minutes. "Kill the outgroup" is quite popular, or "just tell them to do X and kill/imprison anyone who resists".

There is also a difference between parents expressing frustration about vaccination, like maybe it happens to early or too frequently, but their reactions will be different if they have e.g. "vaccines cause autism" as a rallying point.

Basically, a meme can offer a "cure" that is worse than the thing you were originally frustrated about, but it sounds attractive because the frustration happens here and now, and the concerns about the "cure" seem too hypothetical.

Replying toStone Age Billionaire Can't Words Good

Viliam10h

Stone Age Billionaire Can't Words Good

That billionaires are unpopular is what makes expropriating their stuff politically feasible.

That's why when Robert Fico in Slovakia wanted to increase the taxes on the middle class, he called it a "millionaire tax".

(The average voter is unbelievably stupid. The argument "it is called a 'millionaire tax', but in fact the law is written as an extra tax for people with monthly income between €2000 and €3000" is just met with dumb stares and repeating "but it is a millionaire tax, so it only applies to millionaires, and they already have more than they need, why are you concerned about it if you are not a millionaire?")

Replying toStone Age Billionaire Can't Words Good

Viliam11h

Stone Age Billionaire Can't Words Good

When I grew up in a communist country, they taught us that there was a difference between "personal property" (like your shoes and books and things like that) which was good, and "private property" (like owning a factory) which was bad.

The line between these two was basically that "personal property" is for consumption, but "private property" is the stuff that makes more stuff, it makes the initial small differences grow, so if we don't want it to grow out of control, we better nip it in the bud. (Only the Party, and the experts it appoints, are qualified enough to control the stuff that makes more stuff.)

I had a question that seemed... (read more)

Replying toOn Resolving the Great Matter

Viliam12h

On Resolving the Great Matter

As I see it, my death is "worlds that exist without me". If I die today, tomorrow the Sun will still shine, only I won't be there to see it. The people I know will be there, only I won't be with them. It's similar to imagining people living on a different planet. Things will happen, beyond my control, without me being able to see the result, just like if they happened in a distant galaxy.

I will not be there. There will be no me anywhere. Ever. That kinda sucks, because right now I have some wishes about things I would like to do and experience and learn; and if I die,... (read more)

Replying toIs a constitution a "noble lie"?

Viliam12h

Is a constitution a "noble lie"?

in most cases, dissatisfaction with existing institutions leverages critical theory to attack them. But the failure of existing institutions causes the unrest, not critical theory itself.

You make a good and interesting point here. On the other hand, "dissatisfaction" is not a binary thing; people can be more or less frustrated about something. Some memes can be helpful to help mildly frustrated people create mayhem that would otherwise only be created by the extremely frustrated ones. And because nothing is perfect, there will always be some mildly frustrated people.

So it's a combination of how much things really suck, and how much people believe that if something frustrates you, you should burn everything down, that determines how many things get burned down.

Viliam13h

Maybe an AI should automatically read the article and generate an image based on it. At least that would give more information than a random (albeit beautiful) color thing.

Not sure whether this is a good idea, but it seems like an interesting experiment. Perhaps you could try this with some selection of high-karma (so the readers are more likely to remember them) recent articles that did not contain a picture, and post the results?

If the prompt "draw something that is associated with the text of the article" is not sufficient, maybe it could include some nudges, such as "check whether the article contains math? is about AI? is fiction?" and then maybe make some specific recommendations (e.g. use a specific artistic style for fiction).

-9

Halfhaven Forever

Viliam

1mo

Scott Alexander, a famous blogger with about 100 000 subscribers, once said:

"Whenever I see a new person who blogs every day, it's very rare that that never goes anywhere or they don't get good. That's like my best leading indicator for who's going to be a good blogger."

We know that "correlation is not causation" and that "when a measure becomes a target, it ceases to be a good measure". But we also know that "practice makes perfect", so...

In November 2025, Lightcone Infrastructure organized The Inkhaven Residency, where 30 people posted 1 blog per day during 30 days, with feedback and mentorship from other successful writers.

For people who couldn't take a month off... (read 985 more words →)

Resources for parents

Viliam

2mo

According to recent surveys, the average age of LessWrong readers is 30±10 years, and about 15% of readers have one or more children. That means that although most readers are childless, there are enough parents here to have a discussion about parenting.

There are various topics that parents can be interested it.

Some advice will apply to children in general. Why should we discuss it on LessWrong, if there are already thousands of websites dedicated to this topic? The other websites disagree with each other, and we may want to separate good advice from superstition.

But we also need advice that applies more specifically to our community. According to surveys, the average IQ of LessWrong... (read 323 more words →)

Do we have some page containing resources for rationalist parents, or generally for parents of smart children? Such as recommended books, toys, learning apps, etc.

I found tag https://www.lesswrong.com/w/parenting but I was hoping for some kind of best textbooks / recommendations / reference works but for parents/children.

I am fascinated by how often I read something about LLMs and it seems to illustrate something about human psychology. I wonder how many psychologists think about these things. (I suspect not many, because psychologists typically don't read technical articles about LLMs.)

For example, in "GDM: Consistency Training Helps Limit Sycophancy and Jailbreaks in Gemini 2.5 Flash" the part "Bias-augmented Consistency Training", specifically "Train the model via SFT to give the clean response ... when shown the wrapped prompt"... that reminds me strongly of "Asch’s Conformity Experiment", "On Expressing Your Concerns". Specifically that it becomes much easier to resist pressure when you have seen an example of resisting the pressure.

Halfhaven halftime

Viliam

3mo

Halfhaven is a virtual blogger camp, an online alternative to Inkhaven Residency.

The rules are simple:

every day post max 1 article with min 500 words (or equivalent effort)
try to get 30 by the end of November (but there are no hard lines)

The invitation links keep expiring, the current one is: https://discord.gg/rZv3a2Jt

If you wanted to join during November, but you couldn't join because you didn't have the link, I apologize; I was on a vacation without internet. If you published your articles online anyway, feel free to add them retroactively in Halfhaven with their actual publication date. In general, post a link in Halfhaven the same day you publish the post.

Here are the posts... (read 1142 more words →)

Before we had LLMs, "sycophancy" was called "how to win friends and influence people".

Halfhaven virtual blogger camp

Viliam

4mo

If you are a blogger, or an aspiring blogger, and want to improve your skills, you can spend a month in Inkhaven Residency (homepage, Less Wrong post) and keep writing and publishing every day during November 2025. At the end, you will be a great writer! (At least, that is the hypothesis behind the project.) There will also be mentorship from successful bloggers, writing workshops, and feedback.

Some of us have a school / job / family / other excuse, and can't simply take a month off.

Halfhaven is an alternative, online blogging camp that allows you to produce at half the speed, from the comfort of your home. It is a Discord group... (read 515 more words →)

As an alternative to Inkhaven Residency, for the people who can't take a 30 day break, I propose an alternative. I am writing about it here, in case someone else also wants to do something, so that we could coordinate. Please respond here if you plan something, otherwise I will announce my plan officially in a LW post on Sept 30.

My current vision, which I provisionally call Halfhaven, is to make an online blog-writing group, probably on Discord, that will try to produce the same total output in twice as much time. That is 30 articles posted during October and November 2025, so on average one article in two days. Feels slightly... (read 466 more words →)

Why do AIs write using the "AI style"? (Overuse of words like "delve" or constructions like "it is not X, it is Y".) I mean, they are trained on human texts, people call them "stochastic parrots", so how is it possible for them to develop a distinct style? Isn't that like the only thing they - practically by definition - shouldn't do? I would expect them to write text that superficially seems like written by humans, but if you look closely, you see that the style is there but the substance is not. Instead, the style is wrong.

There are many things that I don't understand about LLMs (practically all of them), but this one bothers me because I would expect the exact opposite.

(Is it possible that the "AI style" is actually a style of some category of humans that produce a lot of texts that for some mysterious reason most people never see, so those texts are over-represented in the learning data? Journalists maybe?)

Wikipedia, but written by AIs

Viliam

6mo

I got this crazy idea; I wonder if anyone could try it. Let's make an online encyclopedia, similar to Wikipedia, with one difference: all articles would be edited by AIs.

Why? (I mean, other than "because it's possible and sounds kinda cool".)

First, because it's possible. If an AI can give you a report on a certain topic, it might as well create an encyclopedia article on the topic. But unlike asking the AI directly, when you read the encyclopedia you know that you are reading the same version everyone else is.^[1] This avoids the problem of the AI telling you exactly what it thinks you want to hear. No more sycophancy - now the... (read 1074 more words →)

I propose a term "Pascal's abuser" (analogical to Pascal's mugger) to describe something that seems to happen repeatedly in the rationalist community, to make it easier to notice the same pattern in the future.

Pascal's abuser is a person or an organization who go like: "Yeah, what we are doing here hurts people, but it is okay because there is a tiny chance it might contribute to saving the world."

Examples: Black Lotus, Leverage Research... and maybe also MAPLE. But more importantly, other such people and organizations that I expect to appear in the rationality community in future, so I want us to recognize the pattern faster.

In Slovakia, we have a tradition of correspondence math competitions where the math problems are embedded in (extremely simple) stories. Do similar things exist in English-speaking countries? If yes, please give me some links.

Here is an example https://pikomat.sk/s/archiv but you would have to translate the PDF documents, not sure how difficult that is.

The incentives in scientific publishing are even more perverse than I assumed.

Recently I talked to someone who is an editor in a small scientific journal. They said that the work feels meaningful to them, because they get a few good scientific articles that probably couldn't get published elsewhere (they are good, but not so important in the large picture). Then they complained about also getting lots of articles that suck (such as: fail to consider obvious alternative explanations, or the conclusions in the text contradict the attached data), mostly from various African and Chinese wannabe scientists.

"So you publish the good articles, and reject the bad ones, right?" I asked.

"I wish I could",... (read more)

Learned helplessness about "teaching to the test"

Viliam

8mo

I keep wondering why is there so much learned helplessness about "teaching to the test".

It is often used as an example of Goodharting (e.g. here), and the implied conclusion seems to be... that it is wrong to try testing students systematically, because it is known that it inevitably causes teaching to the test, which ruins education?

Every time I read something like that, I think: Why can't we simply fix the test, so that "teaching to the test" either becomes impossible, or becomes the right thing to do? To me this seems like the obvious reaction, so I am surprised that I don't see it more often.

Let me give you a fictional example:... (read 835 more words →)

This is how I lost my weight on orforglipron: graph. The X axis is about 3 years, yellow interval is more than 1 year when I took the pills.

[Book Translation] Three Days in Dwarfland

Viliam

9mo

There was a book written half a century ago in Soviet Union, where a group of three kids travel to the land of arithmetic, meet anthropomorphic numbers, and learn a few mathematical concepts in a way appropriate for kids about 10 years old...

And I thought: "What a pity this was never translated to English."

Well, thanks to LLMs, this is not a problem anymore: download here 💾.

I hope you enjoyed it!

That said, the translation is very far from perfect. I tried to fix the most obvious mistakes made by the automatic translation, but English is not my first language. Plus there are a few poems I didn't have patience for, beyond telling the... (read 152 more words →)

The first AI war will be in your computer

Viliam

10mo

The first AI war will be in your computer and/or smartphone.

Companies want to get customers / users. The ones more willing to take "no" for an answer will lose in the competition. You don't need a salesman when an install script (ideally, run without the user's consent) does a better job; and most users won't care.

Sometimes Windows during a system update removes dual boot from my computer and replaces it with Windows-only boot. Sometimes a web browser tells me "I noticed that I am not your default browser, do you want me to register as your default browser?" Sometimes Windows during a system update just registers current Microsoft's browser as a default... (read 701 more words →)

Two hemispheres - I do not think it means what you think it means

Viliam

I am going to address some misconceptions about brain hemispheres -- in popular culture, and in Zizian theory. The latter, because the madness must stop. The former, because it provided a foundation for the latter.

Two hemispheres in popular culture

About 99% of animals are bilaterally symmetric -- the left side and the right side of the body are approximately each other's mirror images. The symmetry is not perfect. For example the human heart is situated slightly on the left side, and its left and right halves have slightly different functions. But in general, it seems like once Evolution Fairy has decided that bilateral symmetry is a good idea, it was easier to keep... (read 3967 more words →)

112

Trying to be rational for the wrong reasons

Viliam

Rationalists are people who have an irrational preference for rationality.

This may sound silly, but when you think about it, it couldn't be any other way. I am not saying that all reasons in favor of rationality are irrational -- in fact, there are many rational reasons to be rational! It's just that "rational reasons to be rational" is a circular argument that is not going to impress anyone who doesn't already care about rationality for some other reason.

So when there is a debate like "but wouldn't the right kind of self-deception be more instrumentally useful than perfectly calibrated rationality? do you care more about rationality or about winning?", well... you can make... (read 715 more words →)

LESSWRONG
LW

LESSWRONG
LW

Two hemispheres - I do not think it means what you think it means

Bring up Genius

Require contributions in advance

Impostor Syndrome as skill/dominance mismatch

Viliam

Viliam

Halfhaven Forever

Resources for parents

Halfhaven halftime

Halfhaven virtual blogger camp

Wikipedia, but written by AIs

Learned helplessness about "teaching to the test"

[Book Translation] Three Days in Dwarfland

Viliam

Two hemispheres - I do not think it means what you think it means

Bring up Genius

Require contributions in advance

Impostor Syndrome as skill/dominance mismatch

Viliam

Viliam

Halfhaven Forever

Resources for parents

Halfhaven halftime

Halfhaven virtual blogger camp

Wikipedia, but written by AIs

Learned helplessness about "teaching to the test"

[Book Translation] Three Days in Dwarfland

Two hemispheres in popular culture