Introspective-RSIExtrospective-RSIThe meta-cognition and meso-cognition occur within the same entity.The meta-cognition and meso-cognition occur in different entities. Much like a human, an AI will observe, analyze, and modify its own cognitive processes. And this capacity is privileged: the AI can make self-observations and self-modifications that can't be made from outside. AIs...
1. The Case You've probably heard something like this before: 1. If we survive this century, the expected value of the future is massive. 2. If we don't survive, the expected value is near zero. 3. Therefore, the value of an intervention is approximately proportional to how much it increases...
Many people don't want to live in a crazy sci-fi world, and I predict I will be one of them. 1. People in the past have mourned technological transformation, and they saw less in their life than I will in mine.[1] 2. It's notoriously difficult to describe a sci-fi utopia...
There's an argument against the dealmaking agenda that I've heard a few times. I will call it the "strategy-stealing" argument. In short: for any honest dealmaking strategy you might pursue, there exists a deceptive alternative that produces identical AI behaviour at lower cost, so you (as a rational actor) will...
1. Summary If we promise compensation to an AI for behaving safe and usefully, how does that change their behaviour? In this post, I will present a very simple model for the AIs behaviour under the presence of deals. This model suggests that dealmaking motivates an AI to behave safely...
Summary: "Stratified utopia" is an outcome where mundane values get proximal resources (near Earth in space and time) and exotic values get distal resources (distant galaxies and far futures). I discuss whether this outcome is likely or desirable. Stratified Utopia (thanks Dušan D. Nešić) 1. Introduction 1.1. Happy Coincidence I...
Summary: Suppose we have many different AI models, none of which we trust isn’t scheming. Should we deploy multiple copies of our most trusted model, or an ensemble of many different models? I claim that mixed deployment is better, and offer some recommendations. 1. The case for mixed deployment In...