I would like to bring forward from 2017 the paper from Ministry of Foreign Affairs of Finland, done with cooperation of Global Priorities Project and Future of Humanity Institute, University of Oxford

Existential Risk Diplomacy and Governance, 1.1.4 Artificial Intelligence, page 9

1.1.4 Artificial intelligence Currently, artificial intelligence can outperform humans in a number of narrow domains, such as playing chess and searching data. As artificial intelligence researchers continue to make progress, though, these domains are highly likely to grow in number and breadth over time. Many experts now believe there is a significant chance that a machine superintelligence – a system that can outperform humans at all relevant intelligence tasks – will be developed within the next century. In a 2014 survey of artificial intelligence experts, the median expert estimated that there is a 50% chance of human-level artificial intelligence by 2040, and that once human-level artificial intelligence is achieved, there is a 75% chance of superintelligence in the following 30 years.44 Although small sample size, selection bias, and the unreliability of subjective opinions mean that these estimates warrant scepticism, they nevertheless suggest that the possibility of superintelligence ought to be taken seriously. If a superintelligence comes to exist, it will plausibly usher in economic, social, and political changes of a magnitude significantly beyond those wrought by the Industrial Revolution. While it could certainly offer many benefits, such as increased economic productivity and solutions to various technical problems, superintelligence could also be a factor in increasing existential risk. Firstly, it could exacerbate other existential risks by destabilising political equilibria or by enabling the creation and deployment of other dangerous technologies. Secondly, it could cause grave harm through unintended consequences: the technology could be so opaque and powerful as to make it hard to ensure that it behaves in a way conducive to human good. There are a number of difficult technical problems related to the design of accident-free artificial-intelligence systems that have only recently been recognised.45 If superintelligence comes to exist before these problems are solved then it could itself constitute an existential risk.46

Given 2017+30 = 2047, and the passage mentions 50% for AGI before 2040, that would seem to match to "demonstrated past pragmatic power to control the disposition of funding and public promotion of ideas, contrary to “AGI median in 30 years or longer” and “utter ruin at 10% or lower”, before the ChatGPT moment." per my understanding.

Random thought: maybe there could have been disproportional gains got by getting Said to involve more humor in his messaging and branding him the official Fool of Lesswrong.com?

It seems the community indeed gets service out of Said shooting down low quality communication, and limiting that form of communication socially to his specific role maybe would have insulated the wider social implications, so that most value would have been preserved each way, maybe?

I notice I am most confused on the Expansion -> KABOOM 70%.

I have been in a model that Expansion would be limited within Ukraine, annexed territories including Crimea included. Therefore I have (completely subjectively) estimated Expansion -> KABOOM to 1/1000 or lower?

It seems to me that as Russia has moved its nuclear weapon submarines from Crimea to Russian mainland port, that this could be a shared model also in Russia.

MaxTegmark:

1. Why do you see Expansion to include attacks on Russian mainland?
2. If no attacks on Russian mainland, does this alter your previous estimation of 70% of Expansion -> KABOOM?

The point is that 1) after omicron recedes, there won't be much infection going around for a while, local spread should go near zero; and 2) if you get infected from your own previous illness, it is guaranteed your body has defeated that specific pathogen (variants, mutations) and should – given you still have immunity as you have recovered – be able to defeat it again, better than random other person's variant. Your own previous illness should be safer for you than random other person's random illness.

So if I don't get ill but remain immunonaive aside vaccine, surely then I will just enjoy my infection-free life until the next wave. But if I get infected, and there is not likely new vaccine doses available, I have interest on keeping my immunity against infection high with repeated infections the body has learned to repel. That way the next wave in around 2022-08 should have much lower probability to give me an actual infection.

Thanks! In case more boosters will be allowed, indeed then of course one should not expose oneself to live virus on purpose. So the usage decision is a separate from storage decision, as in potential usage time we will likely have much better information on if or when we will get permission for new vaccine doses.

One notable point is that if you have survived the virus before, the frozen virus is the exactly same virus, variant etc, because it is taken from you. Therefore I would understand your body should have a good immunity against that specific virus given your body beat it already.

A friend offered that page 7 of white paper could maybe be used to deduce that Radvac would prevent Covid with ~40%.

This would mean the decision boundaries would get to p(Covid)*40% > 0.01 <=> p(Covid) > 0.01/0.40 <=> p(Covid) > 0.025 so then you would need your chance to get Covid to be over 2.5% for the use to be net beneficial.

If we also presume a 80+ year old person who has 25% probability of death given Covid, then it becomes

p(RV works)*p(get Covid)*p(Covid harm) > p(RV harm) <=> p(Covid)*40%25% > 1/10000 <=> p(Covid) > 0.0001/(0.40.25) = 0.001

so for them the chance to get Covid before official vaccination would need to be over 0.001 for it to be net beneficial with these boundary conditions.

I made a little different, simplified take on the matter:

For Radvac to be net useful, it needs that following is true: p(RV prevents Covid)*p(user gets Covid [is exposed to Covid such that it would lead to infection])*p(Covid causes long term harm) > p(RV causes long term harm)

p(RV harm) is currently from the RV paper likely less than 1/10000, cited example is Pandemrix that caused long term harm of narcolepsy with 1/16000 if you had Swedish or Finnish genome. p(Covid harm) is high in old people, where you can die with up to 25% probability, but for most of young people around here long Covid would seem to dominate and that seems to be maybe 1%. Long Covid probability seems to be not well found, and this seems a likely direction for improving decision with better data.

with these presets we get: p(RV prevents Covid)*p(user gets Covid) > p(RV harm)/p(Covid harm) <=> p(RV prevents Covid)*p(get Covid) > 0,0001/0,01 = 0,01

from this, we get 3 inequalities as boundary conditions: (presume scenario where getting Covid is max, that is 100% => prevention needs to be > 0,01; vice versa)

• p(RV prevents Covid) > 0,01
• p(get Covid) > 0,01
• p(RV prevents Covid)*p(get Covid) > 0,01

so with current boundary conditions the key thing to find out with Radvac is how likely it is to cure Covid. This needs to be shown likely to be over 1% or it should not be used unless other boundary conditions can be shown to differ.

An aside: this same calculation applies to all other vaccines, which is why the effort has been put into making sure p(harm from vaccine) is ascertained to be much less than 1/10000. This making sure the vaccine harms the least is about necessary condition for mass vaccinations to be net useful for the participants themselves. This is why we have used 1 year+ for safety testing, which gives us way better and lower prior for vaccine harm than 1/10000. If you get no long term harm from N trial persons, then per succession rule your naive prior is that p(harm) < 1/(N+2).

Thanks for the info. This conflicts with the specification of

A new instance of your class will be initialized in each round.

which I interpreted to mean that there exists exactly 1 instance of the class per round.

The model you propose makes sense though, I guess my mental model of the thing was mistaken.

You are not peeking at the game engine, you can just message this the same as you can message cooperation (2, 0, 2 code etc).

You also do not need to save any information over instances - all of your bots on a round are running on the same instance. If any of your ForetellerBots observes SignalBot, then SignalBot has not dropped. SignalBot's existence is in itself the flag.

Does this open a security hole of future prediction like Spectre etc?

Some bots could have a thing where they remember if they have met certain another bot (SignalBot) in the game. If they haven't, they decide that the game setup carries certain preset information.

If the bots find out later in the game that the preset condition would be true, then they coordinate so that SignalBot causes infinite loop and gets disqualified and the game restarted. Now the game will miss SignalBot, causing the others to use this information to deduce the signalled information.

Tl;dr:

Givens: In some cases Left is best strategy. Otherwise Right is best strategy.

1. ForetellerBot observes SignalBot in the game, decides to play Right.
2. Bots observe Left is better strategy.
3. SignalBot nonhalts and gets thrown out, game is reset.
4. ForetellerBot observes no SignalBot in the game, decides to play Left.