I wonder if it would be a good idea to put editor's notes after likely typos, like:

but we want [editor's note: Elon likely meant “won’t” here] do any contract or agree to “evangelize”.

Yes, it sounds that he put too much stock into Andrej's paper-counting argument, and then even left the board because he didn't want to be associated with a failing company?

Do you not have a kitchen scale? You could have measured the amount of water you put in much more precisely by putting a cup on the scale, zeroing it and then measuring out a certain weight of water.

various distributions from statistical mechanics turn out to be empirically useful even though our universe isn't in thermodynamic equilibrium yet, and so there's some hope that these "idealized" or "convergently instrumentally useful" concepts degrade cleanly into practical real-world concepts like "trees" and "windows". Which are hopefully so convergently instrumentally useful that the AIs also use them.

I don't quite understand the turn of phrase ‘degrade cleanly into practical real-world concepts like "trees" and "windows"’ here. Per my understanding of the analogy to statistical mechanics, I would expect there to be an idealized concept of ‘window’ that assumes idealized circumstances that are never present in real life, but which is nevertheless a useful concept... and that the human concept is that idealized concept, and not a degraded version. Because – isn't the point that the actual real-world concept that doesn't assume idealized circumstances is too computationally complex so that all intelligent agents have to fall back to the idealized version?

Maybe that's what you mean and I'm just misreading what you wrote.

Great article! It clarified the concepts a lot for me.

therefore by eq. 1 we get $\Delta H\left(X|C\right)\ge -H\left(C\right)=+1$.

I think you're missing a $\Delta$ in front of the $H\left(C\right)$ here. (Entropy cannot be negative.)

The oral polio vaccine is administered without the use of needles, and therefore could serve as a testbed for this hypothesis. Unfortunately, I didn't find much literature addressing the question directly of how much more people are willing to take an oral vaccine compared to a needle-based one.

There might not be a clean RCT for this but just looking at the history of the Polio vaccine, I seem to find confirmation for this. In the West, the Salk vaccine (which had to be injected) was available since the 1950s but uptake was very slow. Then the Sabin vaccine was developed around 1960, mostly in the USSR with the initial idea from Sabin (a naturalized US citizen), which was an oral vaccine and a much bigger success. This is confounded by the fact that the Sabin vaccine was considered more effective, but on the other hand it also had a slight chance of giving others a real infection because it contained live viruses. So, I don't know how that affected its perceived desirability. Still, I'd consider this slight evidence in favor of oral vaccines being more popular.

If someone told me to come up with an AGI design and that I already knew the parts, then I would strongly suspect that person was trying to make me do a Dantzig to find the solution. (Me thinking that would of course make it not really work.)

Although we’ve been focusing heavily on the US in our search, we’re also still interested in country suggestions

One thing I as a non-US citizen am interested in is whether alternate countries are easier to immigrate into. Some light research just now seems to show that Canada has a more liberal immigration policy. I tried finding a list of countries by ease of immigration, but couldn't immediately find anything like that.

I'll see whether I can make a more concrete alternative suggestion, but I just wanted to mention the question of immigration in case you've already thought about it.

Personally, I have been thinking about moving to the bay area for a while (and I visited once), but things like the housing market (and the hassle of immigration) kept me from following through. The two cities you presented here sound absolutely lovely though, so I would quite like to join you there (or in a similar place).

I think it's worth mentioning that if you accept the arguments about AI that have been made on this forum since its inception, then the time horizon on which these companies need to function is more like 100 years than 1,000 years. (Because either we'll have an unaligned superintelligence in which case we're all dead, or we'll have an aligned superintelligence which can take over the cryonics operations and improve them (and start on reviving).)

I found "Word Replacer II" for Chrome works perfectly. You can limit it to only be active on specific sites. And then just specify that you want to replace "ſ" by "s".