All of atlas's Comments + Replies

It would be great if you could post a google calendar link (maybe for the next event). That would make it a lot easier to figure time zone issues (I almost messed up due to our switch to summer time on March 28).

3Ben Pace
Noted.

Regarding the suspension of the AstraZeneca vaccine: a crucial point I haven't seen mentioned is that the suspension is temporary: the AstraZeneca vaccinations were only suspended for four days. This is mentioned in the german FAQ that you linked (though presumably it was updated since you posted). In particular, EMA (the European Medicines Agency) recommended that vaccination be resumed on March 18, and the German Bund decided on the same day to resume vaccination on March 19th.

Also, not all EU countries halted vaccinations; my home country Austria for ex... (read more)

It might well be that 1) people who already know RL shouldn't be much surprised by this result and 2) people who don't know much RL are justified in updating on this info (towards mesa-optimizers arising more easily).

This would be the case if RL intuition correctly implies that proto-mesa-optimizers (like the one in the paper) arise naturally, and that intuition wasn't widely shared outside of RL. Not sure if this is actually the way things are, but it seems plausible to me.

2Rohin Shah
I agree. It seems pretty bad if the participants of a forum about AI alignment don't know RL.

Ah, gotcha, that makes more sense. And thanks for the awesome antigen test table you linked there!

Yeah, this has been confusing me as well. There's an antigen test that Roche claims has 96.52% sensitivity (so <4% false negatives), which seems both surprisingly high (since even PCR tests seem to have far lower sensitivity, as per the study you linked) and suspiciously precise.

2danohu
With antigen tests, I believe the figures are  "P(positive test | positive gold standard test)" Look at the documentation for another antigen test -- section 14 gives sensitivity compared to PCR. The figure they promote (and is e.g. used for the German government approval) is 97.56% sensitivity. This is both based on PCR, and based on filtering down to specimens with a high viral load (Ct 20-30). BTW, that German site is a good collection of documentation links for all the antigen tests currently approved.
5Ben Pace
The event video is online here!

Yeah. My plan is to put it on Youtube iff it looks like a good enough intro resource for Cartesian frames in its own right.

You might argue that each individual service must be dangerous, since it is superintelligent at its particular task. However, since the service is optimizing for some bounded task, it is not going to run a long-term planning process [...]

Does this assume that we'll be able to build generally intelligent systems (e.g. the service-creating-service) that optimize for a bounded task?

2Rohin Shah
Depends what you mean by "generally intelligent". Any individual service could certainly have deep and broad knowledge about the world (as with eg. a language translation service), but no service will be able to do all tasks (eg. the service-creating-service is not going to be able to edit genomes, except by creating a new service that learns how to edit genomes). With that caveat, yes, this assumes that we'll be able to build services that optimize for bounded tasks. But this is meant more as a description of how existing AI systems already work. Current RL agents are best modeled as optimizing for maximizing reward obtained for the current episode. (This isn't exactly right, because the value function is trying to capture the reward that can be obtained in the future, but in practice this doesn't make much of a difference.)

Is there a more recent writeup on the history of AI safety anywhere?