Jonas Moss — LessWrong

LESSWRONG
LW

We need another "level" here, probably parallel to the others, for when LLMs are used for idea-generation, criticism of outlines, as a discussion partner et cetera. For instance, let's say I think about countries that are below their potential in some tragic way, like Russia and Iran, countries with loads of cultural capital, educated population, that historically have lots going for them. Then I can ask an LLM "any other countries like that?" and it might mention, say, North Korea, Iraq and Syria, maybe Greece or Turkey or South Italy, with some plausible story attached to them. When I do this interaction with an LLM the end product is going to be... (read more)

Replying toStatistical suggestions for mech interp research and beyond

Jonas Moss5mo

Statistical suggestions for mech interp research and beyond

Most statistical tests come with many assumptions. Pearson correlations technically assume: (1) continuous variables, (2) linear relationships, (3) bivariate normality, meaning that the distribution forms an elliptical cloud, (4) homoscedasticity, meaning that the variance of each variable is stable as the other variables change, and (5) no extreme outliers. Evaluation of statistical significance more generally assumes (6) independence among observations.

You should be more precise here! The Pearson correlation between two variables $X$ and $Y$ is defined, and makes sense as a measure of linear association, provided only that the variance of both variables is finite. The most commonly used tests and confidence intervals (Fisher's transform, Pearson t-test) are valid under bivariate normality... (read more)

Replying toOpen problem: how can we quantify player alignment in 2x2 normal-form games?

Jonas Moss5y

Open problem: how can we quantify player alignment in 2x2 normal-form games?

Are you sure zero-sum games are maximally misaligned? Consider the joint payoff matrix

P = [\begin{matrix} (1, 1) & (1, 1) (1, 1) & (1, 1) \end{matrix}] \sim [\begin{matrix} (0, 0) & (0, 0) (0, 0) & (0, 0) \end{matrix}]

This matrix doesn't appear minimally aligned to me; instead, it seems maximally aligned. It might be a trivial case but has to be accounted for in the analysis, as it's simultaneously a constant sum game and a symmetric/common payoff game.

I suppose alignment should be understood in terms of payoff sums. Let $s$ be the (random!) strategy of player 1 and $r$ be the strategy of player 2, and $A$ and $B$ be their individual payoff matrices. (So that the expected payoff of player 1 is $s^{T} A r$ .) Then they are aligned at $s, r$ if the sum of expected payoffs $s^{T} A r + s^{T} B r$ is "large" and misaligned if it is "small", where "large" and "small"... (read more)