Hi Neel, What do you think of this paper? I felt it was pretty novel and revealing when I wrote it, but never got the chance to publish it in a conference. I still think so. The paper essentially says that the embedding space after taining works like like a static landscape that is formed based on the training data. The model simply follows a predetermined path based on where it started and how much momentum it had been given initially (by the prompt), much like a ball rolling down a static landscape would follow a predetermined path after it had been set ... (read more)
Hi Neel, What do you think of this paper? I felt it was pretty novel and revealing when I wrote it, but never got the chance to publish it in a conference. I still think so. The paper essentially says that the embedding space after taining works like like a static landscape that is formed based on the training data. The model simply follows a predetermined path based on where it started and how much momentum it had been given initially (by the prompt), much like a ball rolling down a static landscape would follow a predetermined path after it had been set ... (read more)