This post is inspired by a series of comments by Bogdan: initial comment follow-up 1 follow-up 2 the goal of this post is to summarize the relevant literature and expand on these ideas. Comment 1: “There will likely still be incentives to make architectures more parallelizable (for training efficiency) and...
Thank you to everyone in AI Safety Camp group 22 for the discussions and suggestions. In particular, thank you to: Bogdan Ionut Cirstea, Vassil Tashev and Jaeson Booker Introduction The goal of AI Safety Camp Team #22 is to assess how promising automating alignment research is (see source 23). We...
This post is inspired by a series of comments by Bogdan: initial comment follow-up 1 follow-up 2 the goal of this post is to summarize the relevant literature and expand on these ideas.
Comment 1: “There will likely still be incentives to make architectures more parallelizable (for training efficiency) and parallelizable architectures will probably be not-that-expressive in a single forward pass (see The Parallelism Tradeoff: Limitations of Log-Precision Transformers). CoT is known to increase the expressivity of Transformers, and the longer the CoT, the greater the gains (see The Expressive Power of Transformers with Chain of Thought). In principle, even a linear auto-regressive next-token predictor is Turing-complete, if you have fine-grained enough CoT data to train it on, and... (read 4969 more words →)
Thank you to everyone in AI Safety Camp group 22 for the discussions and suggestions. In particular, thank you to: Bogdan Ionut Cirstea, Vassil Tashev and Jaeson Booker
The goal of AI Safety Camp Team #22 is to assess how promising automating alignment research is (see source 23). We have decomposed the problem into various sub-problems. One of my teammates (Vassil Tashev) and I have focused a couple weeks of reading and discussion on weak to strong generalization. This is a research direction that the OpenAI super alignment team explored in their first paper, which they published in December of 2023. Here we present a comprehensive review and reading list for weak to... (read 2559 more words →)
looks like theyre trying to double to 6 though: https://x.com/TheNormanMu/status/1948546711074603354