AK1089 — LessWrong

LESSWRONG
LW

AK1089 — LessWrong

I originally saw this on Twitter, and posted mine in response, but I feel in-depth discussion is probably more productive here, so I appreciate you cross-posting this :)

One thing I'm interested in is your position on technical research vs. policy work. At least for me, seeing someone from an organisation focused on technical alignment research claim that "technical research is ~90% likely to be [less] useful" is a little worrying. Is this position mainly driven by timeline worries ("we don't have long, so the most important thing is getting governments to slow capabilities") or by a general pessimism about the field of technical alignment research panning out at all?

Replying toLabs should be explicit about why they are building AGI

AK10892y

Labs should be explicit about why they are building AGI

I'm not sure this effect is as strong as one might think. For one, Dario Amodei (CEO of Anthropic) claimed his P(doom) was around 25% (specifically, "the chance that his technology could end human civilisation"). I remember Sam Altman saying something similar, but can't find an exact figure right now. Meanwhile, Yann LeCun (Chief AI Scientist at Meta) maintains approximately the stance you describe. None of this, as far as I'm aware, has led to significant losses for OpenAI or Anthropic.

Is it really the case that making these claims at an institutional level, on a little corner of one's website, is so much stronger than the CEO of one's company espousing these... (read more)

Replying toUnderspecified Probabilities: A Thought Experiment

AK10892y

Underspecified Probabilities: A Thought Experiment

These are the probabilities of each state:

State	Probability
$A \cap B \cap C$	$x$
$A \cap B \cap \neg C$	$0.3 - x$
$A \cap \neg B \cap C$	$0.15 - x$
$A \cap \neg B \cap \neg C$	$x$
$\neg A \cap B \cap C$	$0.15 - x$
$\neg A \cap B \cap \neg C$	$0.15 + x$
$\neg A \cap \neg B \cap C$	$0.05 + x$
$\neg A \cap \neg B \cap \neg C$	$0.2 - x$

with $x$ being the probability of all three parts of a component being fine. (Obviously, $x \leq 0.15$ , because $P (\neg A \cap B \cap C) \geq 0$ .)

This is not enough information to solve for x, of course, but note that $P (A \cap B \cap C) = P (A \cap \neg B \cap \neg C)$ . Note also that $P (A) P (B) \approx P (A \cap B)$ and $P (A) P (C) \approx P (A \cap C)$ (ie A is not strongly correlated or anti-correlated with $B$ or $C$ ). However, $P (B) P (C) > P (B \cap C)$ by quite a long way: $B$ is fairly strongly anti-correlated with $C$ .

Now here's the estimation bit, I suppose: given that $A$ holds, we'd probably expect a similar distribution of probabilities across values of $B$ and $C$ , given that $A$ is not (strongly) correlated with $B$ or $C$ . So $P (B \cap C | A) \approx P (B \cap C)$ etc. This resolves to $x \approx P (A) \cdot P (B \cap C) = 0.45 \cdot 0.15 = 0.0675$ .

State	Probability
$A \cap B \cap C$	$0.0675$
$A \cap B \cap \neg C$	$0.2325$
$A \cap \neg B \cap C$	$0.0825$
$A \cap \neg B \cap \neg C$	$0.0675$
$\neg A \cap B \cap C$	$0.0825$
$\neg A \cap B \cap \neg C$	$0.2175$
$\neg A \cap \neg B \cap C$	$0.1175$
$\neg A \cap \neg B \cap \neg C$	$0.1325$

This seems... not super unreasonable? At least, it appears slightly better than going for the most basic method, which is $P (A \cap B \cap C) \leq 0.15$ , so split the difference and say it's $0.075$ or thereabouts.

The... (read more)

Replying toHow to decide under low-stakes uncertainty

AK10893y

How to decide under low-stakes uncertainty

The trick I usually use for this is to look at my watch (which is digital and precise to the second), and taking the seconds counter, which is uniformly distributed across ${0 \dots 59}$ . I then take option 0 if this is even and option 1 if it is odd.

(This also extends nicely to decisions with any small number of options: 3, 4, 5, and 6 are also factors of 60, so seconds modulo X is uniformly distributed across ${0 \dots X - 1}$ . Plus, I'm not very good at coming up with words.)

Replying toTwo Percolation Puzzles

AK10893y*

Two Percolation Puzzles

Question 1:

For any given value of $p$ , consider the probability of a queen being able to traverse the board vertically as being $q (p)$ , and the probability of a rook being able to traverse the board horizontally (ie. from left edge to right edge) as being $r (p)$ . This notation is to emphasise that these values are dependent on $p$ .

The key observation, as I see it, is that for any given board, exactly one of the conditions "queen can traverse the board vertically" and "rook can traverse the board horizontally using the missing squares of the board" is true. If the rook has a horizontal path across the missing squares, then this path cuts off the queen's access

... (read 530 more words →)