All of MichaelLatowicki's Comments + Replies

Thanks for the pointer. I'll hopefully read the linked article in a couple of days.

I start from a point of "no AI for anyone" and then ask "what can we safely allow". I made a couple of suggestions, where "safely" is understood to mean "safe when treated with great care". You are correct that this definition of "safe" is incompatible with unfettered AI development. But what approach to powerful AI isn't incompatible with unfettered AI development? Every AI capability we build can be combined with other capabilities, making the whole more powerful and there... (read more)

• immensely useful things these AI can do:
  • drive basic science and technology forward at an accelerated pace
  • devise excellent macroeconomic, geopolitical and public health policy
• these things are indeed risk-adjacent, I grant.

Isn't this akin to a protocol for securely monitoring private industry's experiments in thermonuclear weapons? It's better than nothing, but when something is dangerous enough, industrial regulation is never strict enough.

Some things are too dangerous to allow private competition in. The only sensible thing to do is nationalize them and have them run exclusively by extremely security-minded government agencies, if at all. And even that might not be good enough for AI, because we've never had tech whose base case scenario was "kill everyone".

Is this a plausible take?

• some types of AI can be made non-catastrophic with a modest effort:
  • AI trained only to prove math theorems
  • AI trained only to produce predictive causal models of the world, by observation alone (an observer and learner, not an active agent)
  • AIs trained only to optimize w.r.t a clearly specified objective and a formal world model (not actually acting in the world and getting feedback - only being rewarded on solving formal optimization problems well)
    • the last two kinds (world-learners and formal-model-optimizers) should be kept separate