I'm a rising sophomore at the University of Chicago where I co-run the EA group, founded the rationality group, and am studying philosophy and economics/cog sci. I'm largely interested in formal epistemology, metaethics, formal ethics, decision theory, and I have minor interests in a few other areas--I think LessWrong ideas are heavily underrated in philosophy academia, though I have some contentions. I also have a blog where I post about philosophy (and other stuff sometimes) here: https://substack.com/@irrationalitycommunity?utm_source=user-menu.
US AISI will be 'gutted,' Axios reports: https://t.co/blQY9fGL1v. This should have been expected, I think, but it still seems worth sharing,
Downvoted because 1) I don't think people are too hesitant to downvote, and 2) I think explaining one's reasoning is a good epistemic hygiene (downvoting and not explaining is like booing when you hear an idea that you don't like).
This is very interesting research!
One potential critique that I have to this (recognizing that I'm not nearly an expert on the subject, and that this may be a very stupid critique), on the other hand, is that being able to stimulate the environment as if it is the real one to test if they're faking alignment seems like a pretty robust way to see if some model is actually aligned or not. Alignment faking seems much bad of an issue if we can see how much it's faking by (I think).
There is what to be said here about how that may be quite bad when the model is "loose," though.
This is a really good debate on AI doom -- I thought the optimistic side was a good model that I (and maybe others) should spend more time thinking about (mostly about the mechanistic explanation vs extrapolation of trends and induction vs empiricist framings), even though I think I disagreed with a lot of it on an object level:
Saying that we should donate there as opposed to AMF, for example, I would argue is trolleying. You're making tradeoffs and implicitly saying this is worth as much as that. Perhaps you're giving lower trade offs than the pain pleasure stuff, but you didn't really mention these, and they seem important to the end claim "and for these reasons, you should donate to shrimp welfare."
I really don't like when people downvote so heavily without giving reasons - think this is nicely argued!
One issue I do have is that Bob Fischer, the conductor of the Rethink study, warned about exactly what you are sorta doing here in being like ah now we can use x amount of shrimp and saying we can trolly problem a human for that many. This is just one contention, but I think the point is important and people willing to take weird/ controversial ideas seriously (especially here!) should take it more seriously!
https://marginalrevolution.com/marginalrevolution/2024/11/austrian-economics-and-ai-scaling.html
A good short post by Tyler Cowen on anti-AI Doomerism.
I recommend taking a minute to steelman the position before you decide to upvote or downvote this. Even if you disagree with the position object level, there is still value to knowing the models where you may be most mistaken.
I don’t know why you’re getting so many downvotes (okay fine I do it’s because of your tone). Nevertheless, this is an awesome post.
Caveat: A very relevant point to consider is how long you can take a leave of absence, since some universities allow you to do this indefinitely. Being able to pursue what you want/ need while maintaining optionality seems Pareto better.