All of Benquo's Comments + Replies

I built the explanatory model based on my experience employed by and reading about other vaguely analogous institutions, but an acquaintance who'd previously worked at the World Bank said it seemed like an accurate characterization of that institution as well.

The post describes how predation creates a specific gradient favoring better modeling of predator behavior. While fact that most predated species don't develop high intelligence is Bayesian evidence against this explanation, it’s very weak counterevidence because general self-aware intelligence is a very narrow target. More importantly, why would sexual selection specifically target intelligence rather than any other trait?

Looking at peacocks, we can see what appears to be an initial predation-driven selection for looking like they had big intimidating eyes on their backs (similar to butterflies), followed by sexual selection amplifying along roughly that same gradient direction.

Contempt of court penalties for noncompliance with an investigative process is a mainstream example of 1.

Burning Man has some aspects of the second, as do some camping trips, or simply living in a relatively harsh climate. Compare measured levels of corruption in southern vs northern Europe, for instance. When modern democracies fight big wars, the first year involves learning which parts of their warfighting institutions are corrupt and incompetent, & repairing or replacing them.

Your proposal is well-structured and interesting but has a fundamental flaw that needs to be addressed. Interest keyword-based filtering will primarily encourage politics-as-identity, which is actively harmful - it directs attention towards zero-sum thinking and performative identities, rather than creative problem solving. As Bryan Caplan demonstrates in The Myth of the Rational Voter, people already tend to vote to express identities and affiliations rather than to achieve better outcomes. We shouldn't build tools that further entrench this destructive p... (read more)

I agree that even if the book turned out to be entirely accurate we should not assume that this case is representative of the average development project, but we could still learn from it. Many hours from highly trained and well-paid people are allocated to planning and evaluating such projects, which expenditure is ostensibly to ensure quality. Even looking at worst cases helps us see what sort of quality is or is not being ensured.

Wow, thanks for doing the legwork on this - seems like quite possibly I'm analyzing fiction? Annoying if true.

Google's AI response to my search for the Thaba-Tseka Development Project says:

According to available World Bank documentation, the "Thaba-Tseka development project" is primarily referenced within the context of the "Lesotho Integrated Transport, Trade and Logistics Project," which focuses on improving the road corridor connecting Katse to Thaba-Tseka, aiming to enhance regional connectivity and reduce trade costs at Lesotho's borders with South Af

Initially, you argued that societal pressure often reflects genuine wisdom, using examples where a 'society who aggressively shames overconsumption of sweets' might be wiser than a child's raw preferences. You suggested that what I was calling 'intrinsic preferences' might just be 'shallow preferences' that hadn't yet been trained to reflect reality.

Now you're making a different and more sophisticated argument - that the whole framework of 'intrinsic' versus 'external' preferences is problematic because preferences necessarily develop within and respond to... (read more)

Different example - I said "instead"

If you look back, you'll see I was specifically responding to the hypothetical scenario about public admission in that comment. For your points about private shame, you might want to check my other comment replying to you where I addressed how internal shame and self-image maintenance connect to social dynamics.

I notice you're attributing positions to me that I haven't taken and expressing confusion about points I've already addressed in detail. It would be helpful if you could engage more carefully with what I've carefu... (read more)

Except frequently I think people who are ashamed don't expect this.

That’s why I distinguished explicitly between shame and depravity in the OP.

In this example?

Except frequently I think people who are ashamed don't expect this. Imagine that instead of concealing they openly admit and apologize for being only average: then what? Aren't they still ashamed?

I'm thinking of cases like Eliezer's Politics is the Mind-Killer, which makes the relatively narrow claim that politically loaded examples are bad examples for illustrating principles of rationality in the context of learning and teaching those principles, so they should be avoided when a less politicized alternative is available. I think this falsely assumes that it's feasible under current circumstances for some facts to be apolitical in the absence of an active, political defense of the possibility of apolitical speech. But that's a basically reasonable... (read more)

I agree.

When applied to object-level behavior like stealing cookies, this kind of norm internalization is ethically neutral. But when applied to protocols and coordination mechanisms, this becomes part of how shame-based coordination infiltrates and subverts communities doing something more interesting - people who recognize and try to leave bad communities end up recreating those same dysfunctional behaviors in the better communities they seek out.

In my reply to CstineSublime on pecking orders I explored how this works through specific social mechanisms l... (read more)

Admitting and apologizing for being 'only average' often functions as a submission move in dominance hierarchies, i.e. pecking orders.

This move derails attempts to enact more naïve, descriptive-language accountability. When someone has a specific grievance, it corresponds to a claim about the relation between facts and commitments that can be evaluated as true or false. Responding with self-deprecation transforms their concrete complaint into a mere opportunity to either accept or reject the display of submission. This disrupts the sort of language in whic... (read more)

We conceal some facts about ourselves from ourselves to maintain a self-image because such self-images affect how we present ourselves to others and thus what we can be socially entitled to. This is similar to what psychologist Carol Dweck called a "fixed mindset," in contrast with a "growth mindset" where the self-image more explicitly includes the possibility of intentional improvement.

In the singer-songwriter example, creating a good vibe with the audience generally involves projecting confidence. This confidence can connect to an identity as a competen... (read more)

Seems like we've now established that we largely agree on the explicit propositions we've stated all through this thread. Given that, your initial response feels to me like a bit of a non-sequitur.

As I understand it, your response argued against a universal claim that social pressure always inverts genuine preferences, while I had explicitly made the narrower claim that this sometimes happens and is worth watching out for. Does that seem like a fair characterization? If so, can you help me understand why your initial response felt important and relevant to you in context?

Many historical battles have a large component well modeled as a game of chicken, where whichever side's morale breaks first loses. You can get a locally cheap boost to morale if your soldiers have internalized a cultural imperative to seek death in honorable combat, because they'll be less deterrable. There's plenty of credible literary evidence that many soldiers in cultures connected to ours were so acculturated. I am not claiming that it is a human universal, merely that it happens often enough to be an important example of preference inversion.

This seems like a reasonable argument for some premodern fighting. I meant mainly the way of fighting developed in the Napoleonic wars, the American Civil War, and especially WWI. There's a bit in Mein Kampf about how WWI was a major transition in character for Hitler because he switched from fearing danger to intending to move towards danger. Worth reading carefully. The sort of fatalistic stories where people with foreknowledge of their doom keep moving towards it in old warrior-culture texts like the Eddas also seem relevant here. These very much do not seem like human universals; for instance, by my reading it's an attitude entirely foreign to the perspective of the Bible, both the Hebrew Bible and the New Testament.

Sometimes people really don't know any better. Other times they're playing dumb because of a guilty conscience. Nearly everyone is motivated not to acknowledge the when someone's playing dumb, because they share the aforementioned guilty conscience, so many cases of playing dumb are commonly misattributed to really not knowing better.

In cases where I had a strong preexisting relationship with people, they've sometimes admitted, after initially claiming not to be able to understand me when I asked them to do something differently (with my child or otherwise... (read more)

You don't think an exceptional magnitude of recognition for doing useful things is evidence for exceptional capacity and willingness to make that capacity useful to others? Why not?

The assumption that value simply multiplies without reference to underlying mechanisms treats money as magical. While this description often matches observed behavior, I think this apparent match requires explanation. Some people become very wealthy precisely by finding or creating exceptions to this pattern.

I try to decompose apparently irreducible trends into physical configurations and social agents' decisions. When apparent magic persists, I look for the magician - someone intentionally working to make the magic appear true.

Sometimes people are directl... (read more)

Fair point about localized heterogeneity. But simply having different optimal interventions in different places doesn't itself justify splitting resources across them. That would require either:

1. Steeply diminishing returns up to the relevant margin for each intervention (making diversification optimal), or
2. Having more resources than we can deploy in all plausibly effective interventions.

Either claim would be surprising and worth investigating explicitly. I intended this piece as a call for such investigation.

Moreover, if we take your example - productive wea... (read more)

You raise an important distinction I should engage with more directly. Just as there's a difference between teaching 'sugar is evil and eating it makes you bad' versus teaching healthy eating habits, there's clearly a difference between social pressure that helps people learn from others' accumulated wisdom (like warning children about drug addiction) versus pressure that creates persistent dysfunction (like sexual shame that continues in marriage)."

Looking at outcomes could help distinguish these:

• Does the pressure help people better achieve their other go

Actually, I don't think anti-candy messaging originates as a good-faith attempt to teach dietary wisdom; instead, it exemplifies preference inversion through moralized restriction. Rather than providing actionable information about metabolic effects, it constructs an idea of candy as a moral temptation, creating the very compulsive relationship to sweets it claims to prevent.

Take sugar. The standard message is "sugar is bad, candy will rot your teeth and make you fat." But instead of preventing candy consumption, this attitude turns candy into forbidden fr... (read more)

If you start with the conclusion that sex is great, and anti-premarital sex campaigns are really just anti-you-procreating campaigns and therefore oppressive and bad, then sure. I don't think that's a fair assumption across the board (e.g. Amish as an existence proof of "something more"), but it certainly doesn't work for all preferences and it's generally not so clear.

Religions that regulate sexuality comprise a heterogeneous category. I wouldn't describe Amish regulation of sex as a case of preference inversion; the Amish try to make sure people con... (read more)

I agree these mechanisms can coexist. But to test and improve our models and ultimately make better decisions, we need specific hypotheses about how they interact.

The OP was limited in scope because it's trying to explain why more detailed analyses like the ones I offer in The Debtors' Revolt or Calvinism as a Theory of Recovered High-Trust Agency are decision-relevant. Overall my impression is that while the situation is complex, it's frequently explicable as an interaction between a relatively small and enumerable number of "types of guy" (e.g. debtor vs... (read more)

I think of power as distinct from wealth, though both are often signaled through privileged access to scarce resources. Someone standing next to, or even physically possessing, a big hunk of gold, is not necessarily understood to be rich; Scrooge McDuck does not have the same relationship to the gold coins he comes into contact with as a museum curator handling a gold artifact, a gold miner actively extracting gold, or a security guard transporting gold. We think someone's rich when they own a lot of scarce resources, i.e. have some recognized right to it ... (read more)

Thanks for the recommendation, I’ll check out that book. I was aware secondhand of the expression “preference falsification” and its meaning - related to what Bryan Caplan calls “social desirability bias.”

By coining the term “preference inversion” I’m trying to call attention to an important special case of preference falsification, where the fact that a preference has been inverted (and corresponding construction of a hypocritical or ‘bad’ majority) is part of the core mechanism, rather than an accidental cost. This is why Jessica’s idea of antinormativity is relevant; a certain sort of preference falsification has the primary function of creating a guilty conscience, rather than compelling object-level prosocial behavior.

When one preference is expressed only because its holders are extracting resources from people or mindparts with the opposite preference, that seems to me to justify assigning the self-sustaining one priority of some kind.

This doesn’t seem to engage with the content of the post at all, or with my multiple corrections to your implausible misunderstandings, so I think this is a motivated pattern of misunderstanding and I’m done with your comments on this post.

"As calculated prior" is not quite correct, "reflectively stable absent coercive pressure" is a better formulation.

OK, so we've got something like a factual disagreement. Here are some observations that would change my mind substantially:

Credible testimony from someone who'd previously been documented claiming that their variant of Christianity had inculcated in them an anti-sex attitude, that they'd been lying to normalize their non-culturally-conditioned aversion to sex.

An exposé demonstrating that many such prominently documented testimonies were fake and did not correspond to actual people making those claims.

Examples of the sort of thing I mean:

• I Took a Christian

Consider two different contexts in which one might negotiate tradeoffs around work. When discussing work-life balance, you can openly weigh tradeoffs between career and personal time. But when asked 'Why do you want to work at MegaCorp?' in an interview, acknowledging you're trading anything for a paycheck marks you as deviantly uncommitted. The system requires both pretense of pure dedication and practical compromises, while making that pattern itself unspeakable.

My post was about how this dynamic creates internalized preference inversion - where people b... (read more)

Successful religions don't suppress reproduction in practice. But many do maintain an explicit approval hierarchy that ranks celibacy and sexual restraint above typical sexual behavior, sometimes expressing overt disgust with sexuality. This creates a gradient of social rewards that aids group cohesion, but requires most people to be "imperfect" by design. An important failure mode is that some conscientious people try to fully internalize the explicit values, ending up with clinical symptoms of sexual aversion that persist even when officially sanctioned (e.g. in marriage).

I don't think I made those claims. I did say that clerics are often supposed to be celibate, and warriors are generally supposed to move towards danger, in a single sentence, so I see how those claims might have been confused.

The general pattern I'm pointing out is that some scarce resources, or the approval which is a social proxy for such resources, are allocated preferentially to people who adopt an otherwise perverse preference. These systems are only sustainable with large amounts of hypocrisy, where people are on the whole "bad" rather than "good" ac... (read more)

Sometimes people profess or try to reveal a preference for X, as a response to coercive pressures that are specifically motivated by prior underlying preferences for anti-X. This is what I'm calling preference inversion. My intuition is that generally, upon reflection, people would prefer to satisfy their and others' preferences as calculated prior to such influences. I don't know whether there are other sorts of analogous distorting factors nearly all reasonable people would not like to satisfy upon reflection, but in general, I'm using the term "intrinsic preferences" to refer to whatever's left over after all such generally appealing adjustments.

Expanded this reply here: https://benjaminrosshoffman.com/the-drama-of-the-hegelian-dialectic/

X and Y are cooperating to contain people who object-level care about A and B, and recruit them into the dialectic drama. X is getting A wrong on purpose, and Y is getting B wrong on purpose, as a loyalty test. Trying to join the big visible org doing something about A leads to accepting escalating conditioning to develop the blind spot around B, and vice versa.

X and Y use the conflict as a pretext to expropriate resources from the relatively uncommitted. For instance, one way to interpret political polarization in the US is as a scam for the benefit of pe... (read more)

I can’t tell quite what you think you’re saying because “worse” and “morality” are such overloaded terms that the context doesn’t disambiguate well.

Seems to me like people calling it “evil” or “awful” are taking an adversarial frame where good vs evil is roughly orthogonal to strong vs weak, and classifying the crime as an impressive evil-aligned act that increases the prestige of evil, while people calling it disgusting are taking a mental-health frame where the crime is disordered behavior that doesn’t help the criminal. Which one is a more helpful or tr... (read more)

Possessing a home also imposes costs on everyone else - it costs scarce materials and labor to build, equip, and electrify/warm/cool/water a home, and it uses up scarce space in a way that excludes others. It’s not obvious that a homeless person who works & is taxed, and is thus contributing to collective capacity to build and maintain the amenities they take advantage of, is a free rider; you’d need to actually do the math to demonstrate that.

Reality is sufficiently high-dimensional and heterogeneous that if it doesn’t seem like there’s a meaningful “explore/investigate” option with unbounded potential upside, you’re applying a VERY lossy dimensional reduction to your perception.

There’s a common fear response, as though disapproval = death or exile, not a mild diminution in opportunities for advancement. Fear is the body’s stereotyped configuration optimized to prevent or mitigate imminent bodily damage. Most such social threats do not correspond to a danger that is either imminent or severe, but are instead more like moves in a dance that trigger the same interpretive response.

It's true that people who ask for "collaborative truth-seeking" are lying, but false that no one does it. Some things someone might do to try to collaborate on seeking the truth instead of pushing a thesis are:

• Active listening (e.g. trying to restate someone's claims and arguments in one's own words, especially where they seem most unclear or surprising.)
• Extending interpretive labor to try to infer the cause of a disagreement.
• Offering various considerations for how to think about a question instead of pushing a party line - and clarifying the underlying mo

Wouldn’t that imply more upside than downside in staying over?

Huh, I notice I casually used male pronouns here when I previously wasn’t especially inclined to. I guess this happened because I dropped politeness constraints to free up working memory for modeling the causal structure of the problem.

If this had been a lower-latency conversation with the implied greater capacity to make it awkward to ignore a legitimate question, my first reply would have been something like, “well, did you actually assault them? Seems like an important bit of information when assessing whether they made a mistake.” And instead of the mo... (read more)

Examples of info she might have had:

• She was hoping to have sex with Sinclair, so theit sexual advances would not have been unwelcome.
• Harassment from acquaintances of her social class is more common than stranger assault but much less likely to be severely bad - acquaintance assault is socially constrained and thin-tailed, stranger assault is deviant and fat-tailed - which is not adequately captured by the statistics.
• She’s not the sort of person who can be easily traumatized by, or would have a hard time rejecting, unwanted advances.
• Sinclair is in fact

Yes. It seems like RobertM is trying to appeal to some idea about fair play, by saying that people shouldn’t make even disjunctive hypothetical accusations because they wouldn’t like it if someone did that to them. But it seems relevant to evaluating that fairness claim that some accusations are discernibly more justified than others, and in this case RobertM seems not to have been able to think of any plausible crimes to disjunctively accuse me of. I am perplexed as to how “true accusations are better than false ones and you can discover by thinking and i... (read more)

Which unspecified but grossly immoral act did the plain text of my comment seem like it implied a confession of?

They imply irrationality via failure to investigate a confusion, so I thought it was within scope on a rationality improvement forum to point that out. Since there exists an alternative coherent construal I thought it was good practice to acknowledge that as well.