LESSWRONG
LW

All of Tom DAVID's Comments + Replies

Investing in Robust Safety Mechanisms is critical for reducing Systemic Risks

Your first two bullet points are very accurate; it would indeed be relevant to continue by addressing these points further.
Finally, regarding your last bullet point, I agree. Currently, we do not know if it is possible to develop such safeguards, and even if it were, it would require time and further research. I fully agree that this should be made more explicit!!

A list of core AI safety problems and how I hope to solve them

"Instead of building in a shutdown button, build in a shutdown timer."

-> Isn't that a form of corrigibility with an added constraint? I'm not sure what would prevent you from convincing humans that it's a bad thing to respect the timer, for example. Is it because we'll formally verify we avoid deception instance? It's not clear to me but maybe I've misunderstood.

1Jordan Taylor2y

I guess the shutdown timer would be most important in the training stage, so that it (hopefully) learns only to care about the short term.

6davidad2y

A system with a shutdown timer, in my sense, has no terms in its reward function which depend on what happens after the timer expires. (This is discussed in more detail in my previous post.) So there is no reason to persuade humans or do anything else to circumvent the timer, unless there is an inner alignment failure (maybe that’s what you mean by “deception instance”). Indeed, it is the formal verification that prevents inner alignment failures.