2 weeks ago, I realized that I will move from Seattle to Mountain View in 50 days. I was excited but I didn't realize I need to talk with >15 moving companies to find the satisfying ones. After ~9 hours of hunting, I finally found a satisfying one. I chose...
In this post, I want to distill some of my thoughts about AI and my future plan regarding it according to what I have learnt during the past 4 months. About my personal background: My background lies in Financial Engineering and Mathematics, and I used to get motivated by making...
I don't quite understand why the model will choose to comply with the user harmful question in the always-helpful free-tier setup. The blog says if the model refused to respond, it could potentially be trained to be more compliant. But what about the other way -- if the model respond, the respond will also be used to train the model to be worse (because RLHF reinforce positive reward behavior). Is the model too dumb to figure this out? Actually, if the model really understand how RLHF works, it should probably just say a single answer all the time, so all the reward is the same and hence the behavior won't be changed.
Would this suggest that the model just entered into a fiction behavior mode and tries to appear to be "alignment faking"?
2 weeks ago, I realized that I will move from Seattle to Mountain View in 50 days. I was excited but I didn't realize I need to talk with >15 moving companies to find the satisfying ones. After ~9 hours of hunting, I finally found a satisfying one. I chose North American Moving Service for my 850 miles, 2000lbs, 300 cubic feet (CF) moving for the price of 3200$. Here is what I learned (starting from the most important one to the least):
Interesting to read! Curious about your prediction about AI safety related progress. Not sure how much impact it will have on your current prediction.
In this post, I want to distill some of my thoughts about AI and my future plan regarding it according to what I have learnt during the past 4 months.
About my personal background: My background lies in Financial Engineering and Mathematics, and I used to get motivated by making AI more powerful. I decided to shift my focus to AI Safety after reading and getting convinced by the idea inside Preventing an AI-related catastrophe. During the past 4 months, I finished
Have you tried this on smaller or earlier models for "Classifying transcript"? I am worried that the prompt still makes the "alignment evaluation" salient. It would be nice to know how "worse" models perform in this case as a comparison.