I was a grant evaluator for Lightcone Commons last round. Funders disbursed about $45mil through the program, and this included funding roughly my first $6.8 million worth of recommendations. There are other projects I would've preferred to fund over what I did fund, but nobody was doing them and so they weren't on offer. Here's a series of projects that I'd fund in the next round, in rough order of priority. The next round is starting now, and is scheduled to end mid-December. I expect rounds to continue at similar speed. I expect similar amounts of funding will route through me in future rounds.
Disclaimer: I am NOT looking to manage anyone in doing any of these projects. I am not looking to help someone get started. Initiative and leadership and management capacity (and people that can just figure out how the heck to get something done, and that have the connections to do it) are the critical bottlenecks here. I can throw money at promising pitches, but not really help them all that much otherwise.
1. Leaders for a broad public movement against superintelligent machines. In the wake of Coxon Day, a number of random people around the country have reached out to me asking if there's any way to coordinate and organize a grassroots movement. I wish there was a place to point them, that would help support them in organizing people locally and ultimately working towards some big march. My belief is that it's important for there to exist such a movement that avoids the "omnicause attractor" (of being attended only by people who also pick some particular side about environmental issues or middle eastern diplomacy issues or whatever). Further, I think it's important for it to avoid taking a stance on other AI issues: There's a lot of people who hate AI's effect on artists, and people who hate AI's effects on the education system, and people who hate AI's effects on the environment; and there's also people who like current AI as a technology but still don't want to go extinct from it; and we should all be able to team up when it comes to averting extinction. There should exist an org that can ally with the folks who hate datacenters on account of their water usage, when it comes to lobbying for an end to dangerous AI training runs, but that does not require this belief among its members. A group that focuses narrowly on averting extinction, and that serves as a warm and friendly ally to all those who will oppose that particular segment of AI, as part of a broader coalition that not all members will agree with. As far as I can tell, nobody's trying to build this. If there was a good and compelling application for such a thing from someone who could convince me they had a chance of pulling it off, I'd attempt to route them $2mil in the next round (and $6mil/yr).
2. An org dedicated to informing elites. In the wake of the Coxon affair, lots more people are wondering what’s up with the “AI might kill us all” stuff. This probably includes a bunch of people who have disproportionate cultural sway, such as artists, bankers (who probably have a lot of sway over the Trump administration, which cares a lot about the stock market), union leaders (who probably have a lot of sway over what stances politicians are willing to take), etc. I’d love to see an effort that just networks to those folks and offers access to knowledge on this issue. Perhaps by like, literally arranging dinners with folks like me or folks like various whistleblowers. I think it’d be good if they could also offer access to the sorts of folks who say that everything’s going to be fine! Mostly I want powerful folks to be able to hear all the arguments. I think my arguments just straight-up win, so it doesn’t need to be a group focused on propagating one particular message. It needs to be a group that has the powerful connections, and that is able to offer personalized access to people with the knowledge and the arguments. If there was a good and compelling application for such a thing from someone who could convince me they had a chance of pulling it off, I'd attempt to route them $1mil in the next round (and could be convinced to fund a lot more but I’m not convinced this is the sort of thing that scales).
3. An org dedicated to coordinating AI company employees. I think that AI company employees have a lot more power than they realize, to sway the behavior of their host company. The companies didn’t come out and talk about “pacing” until after the employees made a lot of noise. (I think this is true both about the ‘pacing’ letter in July, and about the ‘pacing’ statements in September.) Employees would have a lot more power if they organized. Employees will have a lot less power once their jobs are automated away, and less still once they’re slain along with the rest of us. So they should get moving on the whole ‘organization’ business. This is a sort of project that most people probably couldn’t attempt (for lack of connections and trust). But if some team had the connections and trust and wanted to put in a really solid effort for helping minimize the effort and coordination costs it’d take for AI company employees to organize, I’d attempt to route them $2mil in the next round (and could be convinced to fund a lot more but I’m not sure this sort of thing scales).
Other projects I’m excited about but that seem more niche include:
Based comms: AI notkilleveryoneist communicators who are unashamed of all the work people are doing to avoid the end of the world. It’s ridiculous that White House advising billionaires were able to discredit Jacob Coxon by being like “after he had decided to quit, he asked some folks he know in the AI safety space whether they had any extra advice”. Screw acting ashamed and chided. We should’ve had a real doom(stop)er group that was supporting Coxon even harder even earlier, and we should be proud of any support we give to people who are trying to avert doom. Where were the unashamed memes and messages? Where’s the high-energy poasters who are having a good time trying to avert the extinction of humanity? Can I exchange $ for you having more time to shitpost?
AI company watchdog: The AI company communications are weasely softpedaling, and the public is right to view them as slimy and untrustworthy. Maybe if someone built up an account of calling out their bullshit, we could shame them into being a little less craven. It’s hard to do right, I think, but would be pretty cool if it was done.
Outreach to conservative or religious communities: I think religious communities and conservative communities around the U.S. are pissed about a lot of the AI stuff (as, e.g., they can tell that AI would wreck their way of life and they didn’t sign up for that). I'm not embedded in those communities, but we have common cause here, and we should be able to be friendly allies who join together as part of a coalition. It feels like there should be ways to reach out more hands in friendship (and not in creepy “here’s an issue that I don’t care about but that I think you care about and so I’ll pretend to care about; if I use that as a gateway then will you care about my issue?” sort of way). If anyone has a bunch of good inroads here in the non-pushy non-annoying ways, I’d love to fund them! (I do already have some friends and allies working on this that I am trying to support where I can, but I have appetite for a lot more.)
Technical people figuring out hardware-enabled monitoring and verification for AI chips, that would make it easier to verify and enforce a ban on large training runs very quickly after the world decides to do that.
A big part of the issue with these is gonna be convincing me that you’ve got the experience and connections and competence to maybe pull the project off, rather than just raising your hand and saying “let me try,” alas. But also there’s a decent chance that, in the next round, I’m gonna fund the closest versions I can even if they’re far from optimal. (Though I might not, if the options on offer seem like they’re more likely to poison the wells or render it harder for the next folks. I don’t quite know how to make these tradeoffs when time is possibly-but-not-clearly short.)
I'm not a leader and wouldn't know what to do with money, but I've been trying to talk to people and push them in a related direction for awhile. But it's a bit different. My angle is not about preventing extinction due to AI, it's about being anti-AI in general.
My crisp public-facing statement today is something like this: "Economically, AI will make wages fall faster than cost of living. Politically, AI will make it easier for governments to control people. So we should slow down the AI race." It resonates with me and I'm sure it resonates with more people than the extinction angle. Grassroots is grassroots; we don't get to decide what grass grows best. If we want a broad movement, I think the simple anti-AI banner is the best one.
If anyone has a bunch of good inroads here in the non-pushy non-annoying ways, I’d love to fund them!
+1. AI Safety Awareness Project is good, Project Imago Dei, Humans First, Irreplaceable, but we need 100x more. I myself would be ardently excited to bring the pauseaiist message to the Church, but I am an ordinary lay Christian of unusually low executive function and doubt I'd be good.
Technical people figuring out hardware-enabled monitoring and verification for AI chips, that would make it easier to verify and enforce a ban on large training runs very quickly after the world decides to do that.
It is baffling–––baffling!!! to me that this gets so little attention and funding, relative to its importance.
Yeah, I believe there's a small team of like 8 people in the UK working on building such devices. And their method requires something like 15-20% additional power draw. Ideally you would have hundreds of top engineers working on this, with the goal of getting additional power draw down to a few percent.
I was a grant evaluator for Lightcone Commons last round. Funders disbursed about $45mil through the program, and this included funding roughly my first $6.8 million worth of recommendations. There are other projects I would've preferred to fund over what I did fund, but nobody was doing them and so they weren't on offer. Here's a series of projects that I'd fund in the next round, in rough order of priority. The next round is starting now, and is scheduled to end mid-December. I expect rounds to continue at similar speed. I expect similar amounts of funding will route through me in future rounds.
Disclaimer: I am NOT looking to manage anyone in doing any of these projects. I am not looking to help someone get started. Initiative and leadership and management capacity (and people that can just figure out how the heck to get something done, and that have the connections to do it) are the critical bottlenecks here. I can throw money at promising pitches, but not really help them all that much otherwise.
1. Leaders for a broad public movement against superintelligent machines. In the wake of Coxon Day, a number of random people around the country have reached out to me asking if there's any way to coordinate and organize a grassroots movement. I wish there was a place to point them, that would help support them in organizing people locally and ultimately working towards some big march. My belief is that it's important for there to exist such a movement that avoids the "omnicause attractor" (of being attended only by people who also pick some particular side about environmental issues or middle eastern diplomacy issues or whatever). Further, I think it's important for it to avoid taking a stance on other AI issues: There's a lot of people who hate AI's effect on artists, and people who hate AI's effects on the education system, and people who hate AI's effects on the environment; and there's also people who like current AI as a technology but still don't want to go extinct from it; and we should all be able to team up when it comes to averting extinction. There should exist an org that can ally with the folks who hate datacenters on account of their water usage, when it comes to lobbying for an end to dangerous AI training runs, but that does not require this belief among its members. A group that focuses narrowly on averting extinction, and that serves as a warm and friendly ally to all those who will oppose that particular segment of AI, as part of a broader coalition that not all members will agree with. As far as I can tell, nobody's trying to build this. If there was a good and compelling application for such a thing from someone who could convince me they had a chance of pulling it off, I'd attempt to route them $2mil in the next round (and $6mil/yr).
2. An org dedicated to informing elites. In the wake of the Coxon affair, lots more people are wondering what’s up with the “AI might kill us all” stuff. This probably includes a bunch of people who have disproportionate cultural sway, such as artists, bankers (who probably have a lot of sway over the Trump administration, which cares a lot about the stock market), union leaders (who probably have a lot of sway over what stances politicians are willing to take), etc. I’d love to see an effort that just networks to those folks and offers access to knowledge on this issue. Perhaps by like, literally arranging dinners with folks like me or folks like various whistleblowers. I think it’d be good if they could also offer access to the sorts of folks who say that everything’s going to be fine! Mostly I want powerful folks to be able to hear all the arguments. I think my arguments just straight-up win, so it doesn’t need to be a group focused on propagating one particular message. It needs to be a group that has the powerful connections, and that is able to offer personalized access to people with the knowledge and the arguments. If there was a good and compelling application for such a thing from someone who could convince me they had a chance of pulling it off, I'd attempt to route them $1mil in the next round (and could be convinced to fund a lot more but I’m not convinced this is the sort of thing that scales).
3. An org dedicated to coordinating AI company employees. I think that AI company employees have a lot more power than they realize, to sway the behavior of their host company. The companies didn’t come out and talk about “pacing” until after the employees made a lot of noise. (I think this is true both about the ‘pacing’ letter in July, and about the ‘pacing’ statements in September.) Employees would have a lot more power if they organized. Employees will have a lot less power once their jobs are automated away, and less still once they’re slain along with the rest of us. So they should get moving on the whole ‘organization’ business. This is a sort of project that most people probably couldn’t attempt (for lack of connections and trust). But if some team had the connections and trust and wanted to put in a really solid effort for helping minimize the effort and coordination costs it’d take for AI company employees to organize, I’d attempt to route them $2mil in the next round (and could be convinced to fund a lot more but I’m not sure this sort of thing scales).
Other projects I’m excited about but that seem more niche include:
A big part of the issue with these is gonna be convincing me that you’ve got the experience and connections and competence to maybe pull the project off, rather than just raising your hand and saying “let me try,” alas. But also there’s a decent chance that, in the next round, I’m gonna fund the closest versions I can even if they’re far from optimal. (Though I might not, if the options on offer seem like they’re more likely to poison the wells or render it harder for the next folks. I don’t quite know how to make these tradeoffs when time is possibly-but-not-clearly short.)