TFD

TFD2d

For clarity, I'm not trying to make the case for or against consequentialism/virtue ethics, I'm just trying to respond to the narrow point that I quoted. I don't think people should choose an approach to ethical decision making based primarily on this one specific point.

If you care about the consequences of your crypto philanthropy strategy, your performance on those consequences can be contradicted empirically. But so what if the contradiction arrives in the form of your bankruptcy, along with blemishing the reputation of the movement you were trying to support?

I think this is consistent with my point. From what I can tell SBF continues to claim that he was acting with good... (read more)

TFD2d

That said, I take your central point to be that virtue is more direct or local compared to consequences that it is easier to have evidence on your virtues than the consequences of your actions.

My argument above is specifically about the robustness of these to self deception. Being more local and in a sense "personal" is what makes virtue more suscetible to self deception in my view. There... (read more)

Replying toShould we consider Meta to be a criminal enterprise?

TFD2d

Should we consider Meta to be a criminal enterprise?

Saying the optimal amount of fraud is nonzero is a way to avoid the question of what amount of fraud is reasonable.

I think the phrase is meant to suggest the need to look deeper into the tradeoffs and understand what exactly you'd like an actor to do more or less of rather than go off of high-level impressions.

I think the discussions around the numbers you quote illustrates this point. The $16 billion dollars comes from multiplying by a 10% number from Meta's internal reports. This becomes 23% in one of the comments and 25% of net income in yours. I think it is valid and useful to discuss these types of numbers... (read 619 more words →)

TFD2d

It’s far easier to soothe your ego by finding a rationalization for having caused good consequences, than to self-deceive about whether you’re a paragon of courage or honesty.

It seems like the opposite to me, that it is extremely easy to self-deceive about whether you're a virtuous person. In fact this seems like a quintessential example of self-deception that one encounters fairly commonly.

It is also quite easy to self-deceive about the consequences of ones actions, I agree, but in that case there is at least some empirical fact about the world which could drag you back from your self-deception of you were open to it. In contrast, it seems to me like if... (read more)

Replying toShould we consider Meta to be a criminal enterprise?

TFDFeb 10, 2026

Should we consider Meta to be a criminal enterprise?

I think we should have fairly strict standards for calling something a "criminal enterprise", otherwise any sufficiently scaled business will inevitably be one. Verizon surely knows that their cell service is used to carry out crime. Same with Comcast and their internet service. Likewise for Bank of America and their financial services. I agree with Patrick McKenzie that the optimal amount of fraud is non-zero. If you make a service that is generally useful than it is often also going to be useful for commiting crimes. Actions you take to limit crimes inevitably also impact legitmate users as well and disrupt their ability to use the service legally.

I think the general thought... (read more)

Who does a right to compute actually protect?

TFD

Several states appear to be considering so-called "right to compute" laws. The rhetoric around these laws attempts to draw a comparison with other rights such as the right of free speech and property rights. For example, Montana's SB212 has this to say:

Government actions that restrict the ability to privately own or make use of computational resources for lawful purposes, which infringes on citizens' fundamental rights to property and free expression, must be limited to those demonstrably necessary and narrowly tailored to fulfill a compelling government interest.

Draft legislation from The American Legislative Exchange Council which seems to form the basis for much of these laws follows a similar approach:

Section 3. Right to compute.
Government

... (read 1416 more words →)

Disjunctive arguments can be a reverse multiple-stage fallacy

TFD

12d

Assume we want to know the probability that two events co-occur (i.e. of their conjunction). If the two events are independent, the probability of the co-occurrence is the product of the probabilities of the individual events, P(A and B) = P(A) * P(B).

In order to estimate the probability of some event, one method would be to decompose that event into independent sub-events and use this method to estimate the probability. For example, if the target event E = A and B and C, then we can estimate P(E) as P(A and B and C) = P(A) * P(B) * P(C).

Suppose we want to make an event seem unlikely. If we use the... (read 277 more words →)

Replying toBentham’s Bulldog is wrong about AI risk

TFD12d

Bentham’s Bulldog is wrong about AI risk

Glad the comment was helpful. I will register my prediction that BB most likely meant the "relative to priors" meaning rather than the one that you use in the OP. I also think among people who aren't steeped in the background of AI risk, this would be the significantly more common interpretation upon reading what BB wrote.

Replying toBentham’s Bulldog is wrong about AI risk

TFD13d

Bentham’s Bulldog is wrong about AI risk

I can definitely see the benefits of focusing on likelihoods but I think in practice when we are talking about differences that are like 99% vs 5% this difference usually has its roots in something highly relevant to the ideas. So to take the murder example, lets say I talk to someone and they say that their best friend was murdered, and they have had two best friends and use an empirical bayes approach that gives a prior of 50% that they will be murdered. Sure this is phrased as being about a prior but functionally speaking its about a likelihood, how should the observation that their best friend was murdered influence... (read more)

Replying toBentham’s Bulldog is wrong about AI risk

TFD13d

Bentham’s Bulldog is wrong about AI risk

More importantly, that BB has approximately no knowledge of the experiences and priors that led to those pessimistic posteriors. In general I think it’s wise to stick to discussing ideas (using probability as a tool for doing so) and avoid focusing on whether someone has the right posterior probabilities.

I don't understand this idea at all. If someone someone told me they thought the probability that they would be murdered within the next year is 62%, I'd probably point out that the murder rate per capita makes that seem extremely unlikely. I think that would be a reasonable response even if I didn't fully understand the experiences that lead them to have this... (read more)

Replying toBentham’s Bulldog is wrong about AI risk

TFD13d

Bentham’s Bulldog is wrong about AI risk

I think there is a disconnect here related to different usages of the word "confidence". You say in the OP:

he is more extreme in his confidence that things will be ok

Which I would interpret as being 1- P(not-okay), in other words 1 - 0.026 = 97.4% for BB, very confident.

On the other hand, I think many people probably believe that extinction from misaligned AI is very unlikely apriori, and so might use "confidence" in a sense that is relative to priors. To understand why people might do this, lets imagine that I said "there is a 55% chance that irrelevant AI blogger The Floating Droid will win the 2028 US presidential election".... (read more)

My simple argument for AI policy action

TFD

15d

Many arguments over the risks of advanced AI systems are long, complex, and invoke esoteric or contested concepts and ideas. I believe policy action to address the potential risks from AI is desirable and should be a priority for policymakers. This isn't a view that everyone shares, nor is this concern necessarily salient enough in the minds of the public for politicians to have a strong incentive to work on this issue.

I think increasing the saliency of AI and its risks will require more simple and down-to-earth arguments about the risks that AI presents. This is my attempt to give a simple, stratospherically high-level argument for why people should care about Ai... (read 1612 more words →)

Replying toA Look Inside a Frequentist

TFD9mo

A Look Inside a Frequentist

I'm new here and just going through the sequences (though I have a mathematics background), but I have yet to see a good framing of bayesian/frequentist debate as maximum likelihood vs maximum a-posteriori. (I welcome referrals)

I'm definitely not representative of lesswrong in my my views above I don't think. In fact in some sense I think I'm shadowboxing with lesswrong in some of my comments above, so sorry about any confusion that introduced.

I don't think I've ever seen maximum likelihood vs maximum a-posteriori discussed on lesswrong, and I'm kind of just griping about it! I don't have a references off to top of my head but I recall this appearing in debates... (read more)

Ex-OpenAI employee amici leave to file denied in Musk v OpenAI case?

TFD

9mo

Several ex-employees of OpenAI filed an amicus brief in the Musk v OpenAI^[1] case. This proposed brief argues that OpenAI should not be allowed to move from its existing nonprofit structure to a for-profit structure. We now have an order on the motions to dismiss in the case. This order also seems to deny the motion for leave to file the amicus on behalf of the ex-OpenAI employees.

The order in footnote 1 has this to say on the amicus brief:

The Court also received requests to file an amicus brief by former employees of OpenAI. (Dkt. Nos. 152 & 154.) The proposed brief contains factual assertions and does not assist the Court’s understanding of

... (read 514 more words →)

Don't accuse your interlocutor of being insufficiently truth-seeking

TFD

9mo

I argue that you shouldn't accuse your interlocutor of being insufficiently truth-seeking. This doesn't mean you can't internally model their level of truth-seeking and use that for your own decision-making. It just means you shouldn't come out and say "I think you are being insufficiently truth-seeking".

What you should say instead

Before I explain my reasoning, I'll start with what you should say instead:

"You're wrong"

People are wrong a lot. If you think they are wrong just say so. You should have a strong default for going with this option.

"You're being intentional misleading"

For when you basically thinking they are lying but maybe technically aren't by some definitions of "lying".

What about if they are being unintentionally... (read 339 more words →)

The limits of black-box evaluations: two hypotheticals

TFD

10mo

A prominent approach to AI safety goes under the name of "evals" or "evaluations". These are a critical component of plans that various major labs have, such as Anthropic's responsible scaling policy or OpenAI's preparedness framework. At bottom, these approaches propose to monitor increasingly powerful models using evaluations, and then take some conditional action based on the results, such as implementing more stringent safeguards if an evaluation suggests a model presents a particular risk.

The easiest evaluations to implement (and from my perspective by far the most common) are "black-box" evaluations, meaning they simply evaluate the inputs to a model and their corresponding outputs, they don't "look inside" the model as part of... (read 1091 more words →)

A different take on the Musk v OpenAI preliminary injunction order

TFD

The judge in the Musk v OpenAI^[1] case out of the northern district of California has issued an order on Musk's motion for preliminary injunction, which asked for an order temporarily stopping OpenAI from taking further steps towards its possible conversion from nonprofit to for-profit control. As might be expected for such high profile litigants, this has received quite a bit of attention in the media, pointing out that Musk's request for a preliminary injunction was denied. However, several commentators who I perceive to be interested in AI safety have jumped in to say "not so fast!". They argue that while its true that Musk was denied a preliminary injunction, digging deeper reveals... (read 5763 more words →)

LESSWRONG
LW

LESSWRONG
LW

Disjunctive arguments can be a reverse multiple-stage fallacy

Don't accuse your interlocutor of being insufficiently truth-seeking

Who does a right to compute actually protect?

A different take on the Musk v OpenAI preliminary injunction order

TFD

TFD

Who does a right to compute actually protect?

Disjunctive arguments can be a reverse multiple-stage fallacy

My simple argument for AI policy action

Ex-OpenAI employee amici leave to file denied in Musk v OpenAI case?

Don't accuse your interlocutor of being insufficiently truth-seeking

The limits of black-box evaluations: two hypotheticals

A different take on the Musk v OpenAI preliminary injunction order

TFD

Disjunctive arguments can be a reverse multiple-stage fallacy

Don't accuse your interlocutor of being insufficiently truth-seeking

Who does a right to compute actually protect?

A different take on the Musk v OpenAI preliminary injunction order

TFD

TFD

Who does a right to compute actually protect?

Disjunctive arguments can be a reverse multiple-stage fallacy

My simple argument for AI policy action

Ex-OpenAI employee amici leave to file denied in Musk v OpenAI case?

Don't accuse your interlocutor of being insufficiently truth-seeking

The limits of black-box evaluations: two hypotheticals

A different take on the Musk v OpenAI preliminary injunction order

What you should say instead

"You're wrong"

"You're being intentional misleading"