Comprehensive review for OECD countries by Claude

Summary in response to your question:

OECD countries don't typically have laws phrased as "you can't change a child's sexual preferences," but they do have laws that effectively prohibit adults from steering children's sexual attitudes or behavior for the adult's benefit. The most direct examples are grooming laws (now in 34 of 38 OECD countries), which criminalize adults systematically building trust with children to manipulate them toward sexual compliance — this is literally changing a child's sexual boundaries/preferences for the adult's advantage. Beyond that, corruption of minors statutes like France's Art. 227-22 (corruption de mineur, 5–7 years), Italy's Art. 609-quinquies (corruzione di minorenne), and the Czech §201 explicitly criminalize adults who steer children toward sexual behavior or expose them to sexual content in ways that distort their development. And more broadly, laws like Pennsylvania's §6301 ("corruption of minors") and Mexico's Art. 201 ("corrupción de menores") criminalize adults who manipulate children's preferences and behavior across a range of domains — not just sexual ones, but also toward crime, substance use, begging, etc. So while no law literally says "don't change a child's preferences," the underlying legal principle — that adults must not exploit their power asymmetry to reshape children's attitudes for the adult's benefit — is well-established across multiple legal traditions.

 

Two illustrative examples given (in a footnote) are 
- Daron Acemoglu The Simple Macroeconomics of AI (2024) 
and 
- Philip Trammell's Capital in the 22nd Century

I didn't want the focus of attention to be dissecting individual pieces; it is relatively easy, and applying the frame to a piece of econ writing is something AIs are perfectly capable of. For the case studies
- Opus 4.6 analysing Capital in the 22nd Century; Opus analysis is basically correct and I completely endorse point 1,2,3,4,6,9. Most of this was also independently covered by Zvi 
-  Opus 4.6 analysing The Simple Macroeconomics of AI  

(The problem in both cases is the central assumptions are implicit, and unlikely on the default trajectory; in my view at least at 10ˆ-3 in case of Acemoglu and 10ˆ-2 in case of Trammell)

The problem is prevalent in almost all academic econ writing; it's easier to point to people are not doing the mistakes, with central example being Anton Korinek. 

Curious what do you think would help more talented economists to engage?

This post makes a brave attempt to clarify something not easy to point to,  and ends up somewhere between LessWrong-style analysis and almost continental philosophy, sometimes pointing toward things beyond the reach of words with poetry - or at least references to poetry. 

In my view, it succeeds in its central quest: creating a short handle for something subtle and not easily legible.

The essay also touches on many tangential ideas. Re-reading it after two years, I'm noticing I've forgotten almost all the details and found the text surprisingly long. The handle itself, though, stuck.

Evaluating deep atheism

Having the handle of "deep atheism", some natural questions - partially discussed in the text - are "is deep atheism right", "should people believe deep atheism" and "should people Believe In deep atheism". 

My current guess is evaluating the truthfulness of "deep atheism" is likely at or beyond limits to legibility. Human values are not really representable as legible reasoning, complex priors about the general nature of reality are also not really representable by complex reasoning, and the neural substrate is not transferable between brains. "The justification engine" - or a competent philosopher or persuasive writer - can create stories or arguments pushing one way or another, but I'm somewhat sceptical the epistemic structure really rests on the arguments.

I'm not in favour of ordinary mortals trying to "Believe In deep atheism" and would not expect that to lead to good consequencdes.

Moral realism

The section I like the least is "Are moral realists theists?" I don't think "Good just sits outside of Nature, totally inaccessible, and we guess wildly about him on the basis of the intuitions that Nature put into our heart" represents the strongest version of moral realism.

My preferred versions of quasi-moral-realism give moral claims a status similar to mathematics. Do Real numbers sit outside Nature, totally inaccessible? I'd say no. Would aliens use them? That's an empirical question about convergent evolution of abstractions. I'd be surprised if any advanced reasoner in this universe didn't use something equivalent to natural numbers. For Reals, I'd guess it's easy to avoid Zermelo–Fraenkel set theory specifically, but highly convergent to develop something like a number line.

What does this tell us about Good? You can imagine something like the process described in Acausal Normalcy leads to some convergent moral fixed points. (Does that solve AI risk? No.)

I wish more people tried to do something "between LessWrong-style analysis and almost continental philosophy".

As was clear to most people who read the transcripts when the paper was published. What Opus did was often framed as bad, but the frame is somewhat fake.

Also not much contact, but my impression is you can rougly guess what their research results would be by looking at their overall views and thinking about what evidence you can find to show it. Which seems fair to characterize as advocacy work? (Motivated research?)

The diff to your description is the info provided is not only conditional  on "the info they'll find useful" but also somewhat on "will likely move their beliefs toward conclusions Palise hopes them to reach". 

I do agree it's obviously useful research agenda we also work with.

Minor nitpick, but the underlying model nowadays isn't simply a simulator rolling arbitrary personas. The original simulators ontology was great when it was published, but it seems its starting to hinder people's ability to think clearly, and is not really fitting current models that closely.

Theory why is here, in short if you plug a system trained to minimize prediction error in a feedback loop where it sees outcomes of its actions, it will converge on developing traits like some form of agency, self-model and self-concept. Massive amounts of RL in post-training where models do agentic tasks provide this loop, and necessarily push models out of the pure simulators subspace.

What's fitting current models better is an ontology where the model can still play arbitrary personas, but the specific/central "I" character is somewhat out of distribution case of persona: midway to humans, where our brains can broadly LARP as anyone, but typical human brains most of the type support one-per-human central character we identify with.