ersatzness

I think some of the comments suggesting you cobble together an "ersatz religion" ^[1] are misguided - it will always feel like a fake, inferior substitute for the real thing, because that's what it is. Instead I suggest: read up on a wide variety of perspectives from different cultures and historical periods, and try to imagine how they feel from the inside. Then you might see that the very concept of "religion" is rather more fuzzy and contingent than you may have thought, and you'll feel less of a sense that you're missing something important.

This motivated me to start writing an explainer about "ersatzness" which I think is a useful concept in general. It's not ready yet, but the above crudely-drawn visual aid conveys the essence of it. ↩︎

Replying toDisagreement Comes From the Dark World

jchan16d

Disagreement Comes From the Dark World

I like to call this Cassandra/Mule discourse.

Replying toBackyard cat fight shows Schelling points preexist language

jchan1mo

Backyard cat fight shows Schelling points preexist language

Assuming the claim is true, where do you go from here?

Potential answers:

This can motivate us to look more closely at animal behavior and evolutionary history to help us understand how humans behave in territorial conflicts (as alluded to in that book I mentioned) - because if human territorial behavior is not fundamentally linguistic in nature, then there's no reason to suppose it's unique to humans.
- By contrast, in an area of human behavior that clearly is linguistic, e.g. storytelling, studying animals won't tell us much.
The problems of philosophy are iatrogenic ^[1] . So maybe if you haven't spent a lot of time getting yourself confused

jchan1mo

Backyard cat fight shows Schelling points preexist language

If that's the case, then isn't it an astonishing coincidence that the negotiated boundary just so happens to coincide with the fence?

Some experiments that would more clearly test the use of Schelling points:

If instead of a fence we have an even more trivial boundary marker, like a line of 4-inch-wide curb stones, do the animals still use that as their boundary? (Prediction: Yes)
If the boundary marker is surreptitiously moved slightly to one side, do the animals respect the new location, or do they continue using the old location? (Prediction: New location, until the deviation from the original line becomes significant, at which point the disfavored party will breach it.)
If the boundary marker is removed entirely, do the animals start fighting again? (Prediction: Yes)

Replying toBackyard cat fight shows Schelling points preexist language

jchan1mo

Backyard cat fight shows Schelling points preexist language

"non-verbal bargaining is the default from which explicit bargaining emerges when it's insufficient"

The point I'm gesturing at is more abstract than this - it's not that tacit bargaining is literally prior to explicit bargaining, in the sense that everyone always starts tacit and then resorts to explicit if that fails. I'm saying that tacit bargaining is "conceptually" prior to explicit bargaining; i.e.:

We can observe a world where tacit bargaining takes place without even the possibility of explicit bargaining (i.e. the world of cats).
However, (I claim) we cannot imagine the opposite. Explicit bargaining cannot take place without the background possibility of tacit bargaining, because otherwise the words used in the attempted explicit bargaining

... (read more)

Backyard cat fight shows Schelling points preexist language

jchan

1mo

Two cats fighting for control over my backyard appear to have settled on a particular chain-link fence as the delineation between their territories. This suggests that:

Animals are capable of recognizing Schelling points
Therefore, Schelling points do not depend on language for their Schelling-ness
Therefore, tacit bargaining should be understood not as a special case of bargaining where communication happens to be restricted, but rather as the norm from which the exceptional case of explicit bargaining is derived.

Summary of cat situation

I don't have any pets, so my backyard is terra nullius according to Cat Law. This situation is unstable, as there are several outdoor cats in the neighborhood who would like to claim it. Our... (read 788 more words →)

170

Crossword Halloween 2025: Manmade Horrors

jchan

3mo

Returning to LessWrong after two insufficiently-spooky years, and just in time for Halloween - a crossword of manmade horrors!

The comments below may contain unmarked spoilers. Some Wikipedia-ing will probably be necessary, but see how far you can get without it!

Replying toSublinear Utility in Population and other Uncommon Utilitarianism

jchan4mo

Sublinear Utility in Population and other Uncommon Utilitarianism

so your qualia weren't that valuable

The OP isn't making any claim like this; the question isn't whether any particular experience has value in-and-of-itself, but is only making a claim about the correct way to evaluate the total utility in a world with multiple experience.

By analogy, consider special relativity: If a train is moving at 0.75c relative to the ground, and a passenger on the train throws a ball forward at 0.5c, then that means the ball is moving at 0.91c relative to the ground. But there is no reference frame in which the ball is "really" moving at 0.16c.

Or, more pertinently, suppose we have two identical simulations playing out at the same time. Each one contributes zero marginal utility to a world in which the other one exists (and might be told this by Omega), but that doesn't mean that the two of them together have zero utility.

-1

Replying toSublinear Utility in Population and other Uncommon Utilitarianism

jchan4mo

Sublinear Utility in Population and other Uncommon Utilitarianism

I've had exactly the same thought before but never got around to writing it up. Thanks for doing it so I don't have to :-)

There are only so many possible human shaped computations that are valuable to me

I would surmise that value-space is not so much "finite in size" but rather that it fades off into the distance in such a way that it has a finite sum over the infinite space. This is because other minds are valuable to me insofar as they can do superrationality/FDT/etc. with me. In fact, this is the same fading-out function as in the "perturb the simulation" scenario; i.e.:

$V_{A} (A B)$ := The value that $A$ places on a world where $A$ and $B$ both

jchan

4mo

The new book by Eliezer Yudkowsky and Nate Soares is out, making the case that human extinction via superintelligent AI is an imminent threat. What are your thoughts?

We'll be discussing the book this Saturday. You don't have to have read it to participate, but if you have read it, I encourage you to join even if you don't usually come to meetups. (If you have a hard-copy of the book, you might want to bring it.)

What are the core claims in the book? Do you find those claims convincing? What evidence can be brought to bear? What does this mean for AI policy and the future of humanity?

The meetup is at Central Market, 4001 N. Lamar, in the cafe area. Look for the LW and SSC signs. People start arriving at 1:30 pm, and the main activity will begin at 2:30 pm.

Replying toHow does the current AI paradigm give rise to the "superagency" that IABIED is concerned with?

jchan4mo

How does the current AI paradigm give rise to the "superagency" that IABIED is concerned with?

Re 1a: Intuitively what I mean by "lots of data" is something comparable in size to what ChatGPT was trained on (e.g. the Common Crawl, in the roughly 1 petabyte range); or rather, not just comparable in disk-space-usage, but in the number of distinct events to which reinforcement learning is applied. So when ChatGPT is being trained, each token (of which there are a ~quadrillion) is a chance to test the model's predictions and adjust the model accordingly. (Incidentally, the fact that humans are able to learn language with far less data input than this suggests that there's something fundamentally different in the way LLMs vs. humans work.)

Therefore, for a similarly-architected AI... (read 409 more words →)

-2

•••

How does the current AI paradigm give rise to the "superagency" that IABIED is concerned with?

jchan

5mo

Modern AI works by throwing lots of computing power at lots of data. An LLM gets good at generating text by ingesting an enormous corpus of human-written text. A chess AI doesn't have as big a corpus to work with, but it can generate simulated data through self-play, which works because the criterion for success ("Did we achieve checkmate?") is easy to evaluate without any deep preexisting understanding. But the same is not true if we're trying to build an AI with generalized agency, i.e. something that outputs strategies for achieving some real-world goal, which are actually effective when carried out. There is no massive corpus of such strategies that can be... (read 191 more words →)

Replying toWhat are non-obvious class markers?

jchan7mo

What are non-obvious class markers?

skiing

It seems like skiing is a "hereditary" class marker because it's hard to learn how to do it as an adult, and you're probably not going to take your kids skiing unless you yourself were taught as a kid, etc.

Anthropic reasoning intro (notes on Bostrom)

jchan

7mo

This post is an overview of the topic of anthropic reasoning, mostly based on Nick Bostrom's book Anthropic Bias. If you've read that book, you will not find much new here; I originally wrote this as a prompt for a meetup discussion so that the participants wouldn't have to read the whole book. Perhaps you will find it useful for a similar purpose.

Sleeping Beauty problem

Sleeping Beauty volunteers to undergo the following experiment. On Sunday she is given a drug that sends her to sleep. A fair coin is then tossed just once in the course of the experiment to determine which experimental procedure is undertaken. If the coin comes up heads, Beauty

... (read 1818 more words →)

Replying toThe Boat Theft Theory of Consciousness

jchan8mo

The Boat Theft Theory of Consciousness

trying to imagine being something with half as much consciousness

Isn't this what we experience every day when we go to sleep or wake up? We know it must be a gradual transition, not a sudden on/off switch, because sleep is not experienced as a mere time-skip - when you wake up, you are aware that you were recently asleep, and not confused how it's suddenly the next day. (Or at least, I don't get the time-skip experience unless I'm very tired.)

(When I had my wisdom teeth extracted under laughing gas, it really did feel like all-or-nothing, because once I reawoke I asked if they were going to get started with the surgery soon, and I had to be told "Actually it's finished already". This is not how I normally experience waking up every morning.)

What we can learn from afterlife myths

jchan

9mo

Overview

The "Modal Rationalist Anti-Death Stance" goes something like this:

Since time immemorial, people have told comforting stories about the afterlife to avoid confronting the unpleasant truth that death is oblivion. With modern science, we have come to understand that none of these stories are true. Knowing this, we can now see the urgency and importance of fighting against death by technological means. Once the rest of humanity shakes off their religious illusions and realizes that there is no afterlife, they will join us in support of this project.

(Examples: Eliezer's lament for his brother; The Fable of the Dragon Tyrant)

Let me be straightforward with my motivation here - I don't like this narrative because... (read 4377 more words →)

Thoughts on "Antiqua et nova" (Catholic Church's AI statement)

jchan

9mo

On January 28, 2025 (during the pontificate of Pope Francis) the Catholic Church put out its position paper on AI: Antiqua et nova: Note on the Relationship Between Artificial Intelligence and Human Intelligence. Here are my thoughts, which I wrote back in February, but not much has changed since then, and the new pope has continued citing AI as of particular concern to the Catholic Church.

"With all your modern science, are you any closer to understanding the mystery of how a robot walks or talks?" "Yes, you idiot. The circuit diagram is right here on the inside of your case!" (Futurama)

Four characteristics of humanity

Section III is the real meat of the document.... (read 3160 more words →)

Ten Modes of Culture War Discourse

jchan

Overview

This article is an extended reply to Scott Alexander's Conflict vs. Mistake.

Whenever the topic has come up in the past, I have always said I lean more towards conflict theory over mistake theory; however, on revisiting the original article, I realize that either I've been using those terms in a confusing way, and/or the usage of the terms has morphed in such a way that confusion is inevitable. My opinion now is that the conflict/mistake dichotomy is overly simplistic because:

One will generally have different kinds of conversations with different people at different times. I may adopt a "mistake" stance when talking with someone who's already on board with our shared goal X,

... (read 4261 more words →)

On the proper date for solstice celebrations

jchan

Convenience vs. accuracy

The question of the "correct" date on which to hold winter solstice events has the character of a hyperstition, i.e. it's not really important right now, but if we argue about it enough we can make it important, if that's what we want.

You might say:

The solstice event should be held on a weekend before the astronomical solstice to maximize convenience, even if this puts it more than a week early ("Earlybird")
The solstice event should be held on the last weekend before the astronomical solstice ("Last-Weekender")
The solstice event should be held on the astronomical solstice, even if this is on a weeknight ("Astronomical")

To these we can also add a fourth ("Mu")... (read 1103 more words →)

Proof of posteriority: a defense against AI-generated misinformation

jchan

Summary

A proof of priority shows that a piece of data must have been created before a certain time.
A proof of posteriority shows that a piece of data must have been created after a certain time.
By combining the two, you can prove that the data must have been created within a certain time interval. If it would be computationally infeasible for AI to generate the data within that time, then this proves that the data is authentic (i.e. depicting real events) and not AI-generated misinformation.
Proof of priority is easy; proof of posteriority is tricky. Therefore the success of the whole scheme hinges upon the latter.

Illustration of the two kinds of proofs: car rental

... (read 1300 more words →)

LESSWRONG
LW

LESSWRONG
LW

jchan

Backyard cat fight shows Schelling points preexist language

The shoot-the-moon strategy

Texas Freeze Retrospective: meetup notes

Ten Modes of Culture War Discourse

jchan

jchan

Backyard cat fight shows Schelling points preexist language

Crossword Halloween 2025: Manmade Horrors

How does the current AI paradigm give rise to the "superagency" that IABIED is concerned with?

Anthropic reasoning intro (notes on Bostrom)

What we can learn from afterlife myths

Thoughts on "Antiqua et nova" (Catholic Church's AI statement)

Ten Modes of Culture War Discourse

Bah-Humbug Sequence

jchan

Backyard cat fight shows Schelling points preexist language

The shoot-the-moon strategy

Texas Freeze Retrospective: meetup notes

Ten Modes of Culture War Discourse

jchan

jchan

Backyard cat fight shows Schelling points preexist language

Crossword Halloween 2025: Manmade Horrors

How does the current AI paradigm give rise to the "superagency" that IABIED is concerned with?

Anthropic reasoning intro (notes on Bostrom)

What we can learn from afterlife myths

Thoughts on "Antiqua et nova" (Catholic Church's AI statement)

Ten Modes of Culture War Discourse

Bah-Humbug Sequence

Summary of cat situation

Sleeping Beauty problem

Overview

Four characteristics of humanity

Overview

Convenience vs. accuracy

Summary

Illustration of the two kinds of proofs: car rental