In the foreseeable future, it is expected that the majority of consciousness in the universe will spend their time in the digital world. I argue that in order to unlock the "goodness" of the digital world, we need to focus on a new kind of science--the science for the possible...
After talking with a lot of people in Alignment, I think there is still a lot of good to be done for idea diffusion at the object/technical level. We seem to have done a lot of outreach presenting the philosophical arguments, but less so on the technical ground. Since the...
Continual learning is an alternative, I would argue, to solve long term planning and agency rather than necessary. Augmented LLMs with long term memory retrieval can do long term planning assuming the model is already powerful enough. Also agency just emerges from the simulator naturally.
I'm not convinced about continual learning as even the most likely path to AGI.
In the foreseeable future, it is expected that the majority of consciousness in the universe will spend their time in the digital world. I argue that in order to unlock the "goodness" of the digital world, we need to focus on a new kind of science--the science for the possible world (SPW). The essential idea is that we are creating a new set of laws of nature (LoN) for a possible digital world, thus it requires a new kind of methodology--SPW to study the LoN for the possible world. In the following, I will try to sketch some reasons for taking this worldview and what SPW would look like.
Why Digital?
The hype of... (read 807 more words →)
After talking with a lot of people in Alignment, I think there is still a lot of good to be done for idea diffusion at the object/technical level. We seem to have done a lot of outreach presenting the philosophical arguments, but less so on the technical ground.
Since the field of Alignment is quite diverse and nuanced, we think that it would be good to present how different people approach this problem on different frontiers. For example, Anthropic's empirical approach might be very different from say, Christiano's theoretical thinking on ELK. Therefore, navigating through the landscape of alignment would be essential for building the inside view. I suppose having a good grasp/inside... (read more)
This is an interesting point--when we did our causality studies across layers, we also found that the board state features in the middle layers are mostly used causally--not the deep layers. However, the probe accuracy does increase with depth.
I don't know how this translates to the fact that SAEs also find more of these features in the middle layers. Like, the "natural features" in some sense in the last few layers found by the SAEs do not have to contain much information about the board state but just partial information to make the decision.