Oh yeah. How do I know I'm angry? My back is stiff and starts to hurt.

The second reason that I don't trust the neighbor method is that people just... aren't good at knowing who a majority of their neighbors are voting for. In many cases it's obvious (if over 70% of your neighbors support one candidate or the other, you'll probably know). But if it's 55-45, you probably don't know which direction it's 55-45 in.

My guess is that there's some postprocessing here. E.g. if you assume that the "neighbor" estimate is wrong but without the refusal problem, and you have the same data from the previous election, then you could estimate the shift of opinions and apply that to other pools that ask about your vote. Or you could ask some additional question like "who did your neighbours vote for in the previous election" and compare that to the real data (ideally per county or so). I would be very surprised if they based the bets just on the raw results.

My initial answer to your starting question was "I disagree with this statement because they likely used 20 different ways of calculating the p-value and selected the one that was statistically significant". Also https://en.m.wikipedia.org/wiki/Replication_crisis

I don't think there's a lot of value in distinguishing 3000 and 1,000,000 and probably for any aggregate you'll want to show this will just be "later than 2200" or something like that. But yes this way they can't make a statement that this will be 1,000,000 which is some downside.

I'm not a big fan of looking at the neighbors to decide whether this is a missing answer or high estimate (it's OK to not want to answer this one question). So some N/A or -1 should be ok.

(Just to make it clear, I'm not saying this is an important problem)

I think you can also ask them to put some arbitrarily large number (3000 or 10000 or so) and then just filter out all the numbers above some threshold.

By what year do you think the Singularity will occur? Answer such that you think, conditional on the Singularity occurring, there is an even chance of the Singularity falling before or after this year. If you think a singularity is so unlikely you don't even want to condition on it, leave this question blank

You won't be able to distinguish people who think singularity is super unlikely from people who just didn't want to answer this question for whatever reason (maybe they forgot or thought it's boring or whatever).

Anyway, in such a world some people would probably evolve music that is much more interesting to the public

I wouldn't be so sure.

I think the current diversity of music is largely caused by artists' different lived experiences. You feel something, this is important for you, you try to express that via music. As long as AIs don't have anything like "unique experiences" on the scale of humans, I'm not sure if they'll be able to create music that is that diverse (and thus interesting).

I assume the scenario you described, not a personal AI trained on all your life. With that, it could work.

(Note that I mostly think about small bands, not popular-music-optimised-for-wide-publicity).

Are there other arguments for active skepticism about Multipolar value fragility? I don’t have a ton of great stories

The story looks for me roughly like this:

1. The agents won't have random values - they will be somehow scattered around the current status quo.
2. Therefore the future we'll end up in should not be that far from the status quo.
3. Current world is pretty good.

(I'm not saying I'm sure this is correct, it's just roughly how I think about that)

This is interesting! Although I think it's pretty hard to use that in a benchmark (because you need a set of problems assigned to clearly defined types and I'm not aware of any such dataset).

There are some papers on "do models know what they know", e.g. https://arxiv.org/abs/2401.13275 or https://arxiv.org/pdf/2401.17882.

A shame Sam didn't read this:

But if you are running on corrupted hardware, then the reflective observation that it seems like a righteous and altruistic act to seize power for yourself—this seeming may not be be much evidence for the proposition that seizing power is in fact the action that will most benefit the tribe.