My name is Mikhail Samin (diminutive Misha, @Mihonarium on Twitter, @misha on Telegram).
Humanity's future can be enormous and awesome; losing it would mean our lightcone (and maybe the universe) losing most of its potential value.
My research is currently focused on AI governance and improving the understanding of AI and AI risks among stakeholders. I also have takes on what seems to me to be the very obvious shallow stuff about the technical AI notkilleveryoneism; but many AI Safety researchers told me our conversations improved their understanding of the alignment problem.
I believe a capacity for global regulation is necessary to mitigate the risks posed by future general AI systems. I'm happy to talk to policymakers and researchers about ensuring AI benefits society.
I took the Giving What We Can pledge to donate at least 10% of my income for the rest of my life or until the day I retire (why?).
In the past, I've launched the most funded crowdfunding campaign in the history of Russia (it was to print HPMOR! we printed 21 000 copies =63k books) and founded audd.io, which allowed me to donate >$100k to EA causes, including >$60k to MIRI.
[Less important: I've also started a project to translate 80,000 Hours, a career guide that helps to find a fulfilling career that does good, into Russian. The impact and the effectiveness aside, for a year, I was the head of the Russian Pastafarian Church: a movement claiming to be a parody religion, with 200 000 members in Russia at the time, trying to increase separation between religious organisations and the state. I was a political activist and a human rights advocate. I studied relevant Russian and international law and wrote appeals that won cases against the Russian government in courts; I was able to protect people from unlawful police action. I co-founded the Moscow branch of the "Vesna" democratic movement, coordinated election observers in a Moscow district, wrote dissenting opinions for members of electoral commissions, helped Navalny's Anti-Corruption Foundation, helped Telegram with internet censorship circumvention, and participated in and organized protests and campaigns. The large-scale goal was to build a civil society and turn Russia into a democracy through nonviolent resistance. This goal wasn't achieved, but some of the more local campaigns were successful. That felt important and was also mostly fun- except for being detained by the police. I think it's likely the Russian authorities would imprison me if I ever visit Russia.]
Posts
Wikitag Contributions
Fallacymania and this game's cards:
Fallacymania is a thing, is freely available (you can print it), and has a very similar gameplay at the core
Locally: can you give an example of when it’s okay to kill someone who didn’t lose deontological protection, where you want to kill them because of the causal impact of their death?
I think the general policy of obeying such deontological rules leads to better outcomes; this is the reason for having deontology in the first place. (I agree with that old post on what to do when it feels like there's a good reason to believe that following a different policy would lead to better outcomes.)
My deontology prescribes not to join a Nazi army regardless of how much fear you're in. It's impossible to demand of people to be HPMOR!Hermione, but I think this standard works fine for real-world situations.
(While I do not wish any Nazi soldiers death, regardless of their views or reasons for their actions. There's a sense in which Nazi soldiers are innocent regardless of what they've done; none of them are grown up enough to be truly responsible for their actions. Every single death is very sad, and I'm not sure there has ever been even a single non-innocent human. At the same time, I think it's okay to kill Nazi soldiers (unless they're in a process of surrenderring, etc.) or lie to them, and they don't have deontological protection.)
You're arguing it's okay to defend yourself against innocent people forced to do terrible things. I agree with that, and my deontology agrees with that.
At the same time, killing everyone because otherwise someone else could've killed them with a higher chance = killing many people who aren't ever going to contribute to any terrible things. I think, and my deontology thinks, that this is not okay. Random civilians are not innocent Nazi soldiers; they're simply random innocent people. I ask of Anthropic to please stop working towards killing them.
it was just for Allied soldiers to kill Nazi soldiers in World War II
Killing anyone who hasn't done anything to lose deontological protection is wrong and clearly violates deontology.
As a Nazi soldier, you lose deontological protection.
There are many humans who are not even customers of any of the AI labs; they clearly have not lost deontological protection, and it's not okay to risk killing them without their consent.
I actually agree with Neel that, in principle, an AI lab could race for AGI while acting responsibly and IMO not violating deontology.
Releasing models exactly at the level of their top competitor, immediately after the competitor's release and a bit cheaper; talking to the governments and lobbying for regulation; having an actually robust governance structure and not doing a thing that increases the chance of everyone dying.
This doesn't describe any of the existing labs, though.
1A bill passed two chambers of New York State legislature. It incorporated a lot of feedback from this community. This bill’s author actually talked about it as a keynote speaker at an event organized by FAR at the end of May.
There’s no good theory of change for Anthropic compatible with them opposing and misrepresenting this bill. If you work at Anthropic on AI capabilities, you should stop.
We’ve given some feedback to this bill, like we do with many bills both at federal and state level. Despite improvements, we continue to have some concerns
(Many such cases!)
- RAISE is overly broad/unclear in some of its key definitions which makes it difficult to know how to comply
- If the state believes there is a compliance deficiency in a lab’s safety plan, it’s not clear you’d get an opportunity to correct it before enforcement kicks in
- Definition of ‘safety incident’ is extremely broad/unclear and the turnaround time is v short (72 hours!). This could make for lots of unnecessary over-reporting that distracts you from actual big issues
- It also appears multi-million dollar fines could be imposed for minor, technical violations - this represents a real risk to smaller companies
As we’ve been saying for some time (last year: anthropic.com/news/the-case-…) and recently (nytimes.com/2025/06/05/opi…) we think it’d be great if there could be a federal transparency standard
If there isn't anything at the federal level, we'll continue to engage on bills at the state level - but as this thread highlights, this stuff is complicated.
Any state proposals should be narrowly focused on transparency and not overly prescriptive. Ideally there would be a single rule for the country.
Here’s what the bill’s author says in response:
Jack, Anthropic has repeatedly stressed the urgency and importance of the public safety threats it’s addressing, but those issues seem surprisingly absent here.
Unfortunately, there’s a fair amount in this thread that is misleading and/or inflammatory, especially “multi-million dollar fines could be imposed for minor, technical violations - this represents a real risk to smaller companies.”
An army of lobbyists are painting RAISE as a burden for startups, and this language perpetuates that falsehood. RAISE only applies to companies that are spending over $100M on compute for the final training runs of frontier models, which is a very small, highly-resourced group.
In addition, maximum fines are typically only applied by courts for severe violations, and it’s scaremongering to suggest that the largest penalties will apply to minor infractions.
The 72 hour incident reporting timeline is the same as the cyber incident reporting timeline in the financial services industry, and only a short initial report is required.
AG enforcement + right to cure is effectively toothless, could lead to uneven enforcement, and seems like a bad idea given the high stakes of the issue.
1Yeah, sorry, I think it’s just very tricky for me to pass Anthropic’s ITT, because to imitate Anthropic, I would need to be concurrently saying stuff like “by being a frontier lab, we can push for impactful regulation”, typing stuff like “this bill will impose multi-million dollar fines for minor, technical violations, representing a risk to smaller companies” about a NY bill with requirements only for $100m+ training runs that would not impose multi-million dollar fine for minor violations, and misleading a part of me about Dario’s role (he is the Anthropic’s politics and policy lead and was a lot more involved in SB 1047 than many at Anthropic think).
It’s generally harder to pass ITT of an entity that lies to itself and others than to point out why it is incoherent and ridiculous.
In my mind, a good predictor of Anthropic’s actions is something in the direction of “a bunch of Sam Altmans stuck with potentially unaligned employees (who care about x-risk), going hard on trying to win the race”.
Fallacymania and this game's cards: