This research was performed by Luke Marks, Amirali Abdullah, nothoughtsheadempty and Rauno Arike. Special thanks to Fazl Barez from Apart Research for overseeing the project and contributing greatly to direction and oversight throughout. We'd also like to thank Logan Riggs for feedback and suggestions regarding autoencoder architecture and experiment design....
This research was performed by Luke Marks, Amirali Abdullah, nothoughtsheadempty and Rauno Arike. Special thanks to Fazl Barez from Apart Research for overseeing the project and contributing greatly to direction and oversight throughout. We'd also like to thank Logan Riggs for feedback and suggestions regarding autoencoder architecture and experiment design.
Sparse Autoencoders Find Highly Interpretable Directions in Language Models showed that sparse coding achieves SOTA performance in making features interpretable using OpenAI's method of automated interpretability. We briefly tried to extend these results to reward models learned during RLHF in Pythia-70m/410m. Our method can be summarized as follows:
1. Identify layers in an language model fine-tuned through likely involved in reward modeling. We do so by... (read 1422 more words →)
Hello, I'm working on mechanistic interpretability alongside technical alignment. I'm interested in AI governance and how technical work can inform policy.
I think another regulatory target, particularly around the distribution of individual GPUs, would be limiting enthusiast grade hardware (as opposed to Enterprise) to something like xGB where x is obligately readjusted every year based on risk assessments
Question: how is the criminal justice system not racist? As I understand it. There are no laws banning slavery in prison, so there were financial incentives to move enslaved people to prison and there are financial incentives today to put people in jail which results in the perpetuation of racial biases in law enforcement
What is the difference between studying Politics and studying History?
Some people with certain identities are actively politicized throughout history, not understanding politics and thus not understanding the memetics of the social world which has the power to enforce rules upon you is not advisable if you're a person with one such identity.
Should I attack the anti-trans legislature attacking many in my community today, in the year 2023? Should I defend the rights and freedoms of undocumented workers given their productivity per capita when adjusted for wages? Should I have participated in the 2014 protests against the Venezuelan government?
There are questions of values, the responses to these last two questions depend on ought statements,... (read more)
Not sure why I'm getting downvoted for posting a quote from a communist in a post about collective identity.
I can only reason they are coming from:
If you are not one of these, please let me know so I can adjust my thinking. If you identify with one of the above I think it's better to use the disagreement karma since I didn't say anything factually incorrect afaik
For beings like humans who should have individual identity, healthy collective identity is when that doesn't get overwritten
"The free development of each is the condition for the free development of all." - Karl Marx & Friedrich Engels, 1848
The times when humans get closest to not having their own values, and acting robustly on behalf of another agency’s[7] goals are when they adopt a sort of collective identity, for example as a part of a military, cult, or clan.
Or corporation, many humans act in ways that maximize the profits of a corporation even when they believe that the work they do is a little unethical.
I'm very glad that there's people thinking outside of the individualist bounds that we usually set for ourselves.