First, I'll question some of the assumptions.
I do think there aren't enough leaders. (I'm happy to discuss whether Ender Wiggins was a good leader in comments.) Part of what makes someone a good leader is being able to recruit and work with other people, not just rationalists. In fact, I think for most rationalist projects, you need a rationalist leader, but not necessarily rationalist helpers. For example, Arbital didn't fail because we didn't have enough rationalists (we had 3/3), but because neither of us knew how to run that kind of a startup. However, Arbital very likely wouldn't have even happened unless I tried. And I'm pretty sure of that because after Eliezer approach me with the idea and I told him "no", six months later nobody was doing it. So I picked it up.
So, I think there are a lot of valuable projects out there that have nobody to do them. And if you look at most failed projects, rationalist or not, they don't fail because there weren't enough helpers, they fail because the person who was supposed to be the leader didn't do a good job.
Over the years I have collected a large pile of various failed or aborted projects in my wake. Things that just didn't work out or didn't go the way I had planned or hoped.
Yeah, welcome to the club. I think most people, even those who have been pretty successful have a long list of failed projects as well. If I understand you correctly, it seems like all of yours failed. I think that's still fine. Most (if not all, depending on how you count) of my projects failed too. I don't think that alone is a good enough reason to stop trying, though obviously you should adjust your strategy. (More on that in the next parts.)
Second, I'll give what I think is a helpful context to approach this from.
To me this question reads loosely as "Hmm, I'm trying things, but they aren't working out. May be the root problem is this one particular underlying assumption." And the umbrella question is "What should I do with my life?" As such, I think any answer has to feel right for you at multiple levels. (Jordan Peterson covers this idea very well.) So I'd caution anyone against hyper-focusing on one assumption / problem when it comes to answering a big question like that.
That said, if you ask yourself "What should I do with my life that feels right for me at all levels?", you often don't get an answer. So it's much better to go try something than not do anything at all. I guess, overall, I'd encourage people to approach this question from a more S1 / holistic feeling rather than S2 / systemic debugging.
I'm not sure this applies in your context, but it might apply to some readers, and it's an important enough point that I'll mention it anyway: when you're young, you're just not that capable. It takes a lot of learning and a lot of mistakes to get to the sweet spot. Some of the most helpful advice I've gotten was: "No, we don't want you here helping on this relatively unimportant project. Go out and learn and become stronger and better." It's a bit hard to swallow because it means going out there and failing. A bunch. But if you're the kind of person who can slog through that and learn, you're also likely the kind of person who can grow a tremendous amount because of that. And that seems a lot more valuable than being stuck helping on relatively unimportant projects, where in the end the project has moved forward, but you have been left behind.
Third, I'll share some of my relevant experience.
I actually have had a very similar question some years ago. "Hmm, Eliezer sure seems to produce a lot of good stuff. I bet if I could help him be even 20% more productive that would be a hell of a lot better than anything I could do." So I did. Around 2015, for about 3-4 months I helped Eliezer with a variety of things. I think it was marginally useful, may be even 20% if I stretch it. But it also became clear to me that it's not what I wanted to do. Even if that was the best I could ever hope to accomplish, that just wasn't a sustainable path. I'm italicizing that point because I think it's a very important realization and even at the risk of the typical mind fallacy, I do think it applies to basically everyone. For a path to be sustainable, you need for it to click on all levels. (Of course I didn't learn this lesson back then, it took another two "failures".)
When I worked on Arbital, I was the CEO. I didn't like most of the CEO-specific tasks like raising money. For my current project, I'm the CTO. And that feels much better for me. One could argue that the CEO is Daimyo and the CTO is the Samurai, but I think that really oversimplifies the relationship and much depends on the context. There're opportunities to lead and to follow, to teach and to listen. I'd say it's much closer to collaboration / interdependence than a hierarchy. The best thing is that I get to do the things I love (coding) and someone else gets to do the things they love (talking to investors).
Fourth, I'll answer the questions directly.
1) I think the question is not extremely useful. It's better to search for the intersection between what you enjoy, what you're good at, and what is valuable to do. Then do it in the best way you can given the situation. If that means leading because nobody is doing it, then do that. If it means joining an existing project, then do that. If it means convincing someone to do the project and then helping them or moving on, then do that.
2) I'd start by talking to people who know you and whom you trust. See in which directions they'd invite you to explore. Check with yourself to see what resonates. Give yourself a month or two to explore, but once exploration starts to slow down, just commit to the best task you've found.
3) I'm guessing that since you've asked these questions, that you are indeed very capable of making "a positive difference." And yes, I think you can improve it. Continue to aim high, learn, strive, accept failures, and do the best you can. I think that's all anyone can ask of you, including yourself. And with a bit of luck, in a decade or two, you're going to be a 1000 year old vampire capable of extraordinary feats.
1) Yes, it absolutely is a useful dynamic- In a less compelling and less flattering phrasing, the PC needs a lot of NPCs to do their bidding.
2) The current top figures are pretty much full-up on the number of Samurai that they can support, and when they find that they need more there is practically a waitlist and minor league. You can either try to work your way up the minor league to an all-star, or you can find a Daimyo that isn't already at the top of the ranking. Those people will typically not get along well with existing Daimyos, because the hierarchical mindset tends to only be stable if there is a single person at the top. One way to find such people is to look at who is having conflicts with the major players. Because the Rat community has evolved a bit and there are still only a small number of major players, it has the characteristic that new potential Daimyos are extinguished rather than nurtured; by finding and swearing to such a Daimyo, you are likely to be a very important and counterfactually valuable contributor, but you will also be swearing to a cause which past history suggests will be extinguished.
3) Any Daimyo worth pledging to should be able to recognize when your presence is harmful to their goals, and dismiss you from service at that time. If you do not trust you Daimyo to do so, it is the responsibility of a Samurai to evaluate that for themselves, and to step aside when they are no longer providing benefit.
I think everyone in the comments, and you yourself, are making this distinction seem sharper and more important than it is. I'd prefer to think about it this way:
A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyse a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.
-Robert Heinlein, emphasis my own.
The way I see it, specialization into different roles and areas of expertise is good and proper, but ultimately we all retain (and should cultivate) a basic level of generality, and that's what makes us human.
For my part I've found great pleasure in taking orders. I'm used to doing my own thing as a grad student, so it's really nice to feel like a part of a well-functioning team. But this doesn't make me feel like an NPC or a 'follower' or anything negative like that. I like the "Samurai/Daimyo" framing best.
I believe the military has a similar ethos: Everyone has special roles, but everyone has to do basic training, wear the same uniform, and be ready to grab a rifle if the situation gets desperate.
3) Am I, as a person, actually capable of making a positive difference in general or is my presence generally going to prove useless or detrimental?
To be blunt, I don't think you are making much of a positive difference in terms of changing the exploitative nature of the world, which you seem to be passionate about in your writing. I know it sounds terribly rude but couldn't find another way to put it lest I treat it as a rhetorical question.
I'm not saying you should stop doing what you're doing or that your work isn't valuable in general, any more than I'm saying athletes and theoretical physicists are morons because it's difficult to become a millionaire that way. It's just that in a world overflowing with competing memes, playing politics (in the broader sense of recruiting more people for your tribe) is not a low-hanging fruit in general. I would say the rationalist community isn't so much an army of generals with no soldiers to command, as it is an army of recruiters with no jobs to offer (that is if you conceive rationality as a project rather than just an interest).
Is this something I can improve and if so, how?
Again, not saying you should prioritize changing the world (over doing what you like and enjoy), but in case you want to, I'd say pick a EA cause (you probably know the details better than me) and make an actionable plan. For example, if your preferred cause is AI alignment, enroll in a MOOC on AI. Less meta-level pondering, more object-level work.
Relying on a mentor/boss/leader/daimyo is an important element of your set of decision-making options, but isn't complete enough to be the only answer.
You really can't fully delegate your goal-selection or strategic decisions, as the final responsibility for your impact always and irrevocably falls on you. Also, the difficulty in finding and choosing someone to work under, and the continual evaluation of whether they're still making the right decisions, is _LARGER_ than the difficulty in directly deciding what to work on.
But you CAN get a lot of benefit by having a mentor or boss (or peer, for a lot of things) who you feel responsible to, and who can help you prioritize and stay focused in actually doing things, and then you only have to spot-check the higher-level alignment.
My name for this Einsteins and Eddingtons.** Besides the vital testing and extension of the big ideas, the Eddington can also handle popularisation and, most important of all, the identification and nurturing of new Einsteins. This is one reason I think teaching in academia could be high-impact, despite all the notorious inefficiencies and moral mazes.
** Not totally fair to Eddington, since he was a pretty strong theorist himself.
If you're not good at discernment, how will you choose whom to follow? There are many lousy leaders to be followed.
Not sure if I'm stating the obvious here, but whether person X would do more good following person Y, or trying to improve themselves to acquire all missing skills, that depends on the combination of [X, Y]; it cannot be answered by looking at X alone. You need to find Y who requires someone with skills you have, and doesn't mind the skills you don't have. It even depends on the project Y is currently working on.
Making the potential X's and Y's meet each other, that in itself is a project someone would need to choose as their priority, otherwise it won't happen. I could imagine something like a job agency, except for rationalist volunteers. (Actually, it could also include paid jobs.) Where some people could specify they are looking for someone with some skills, and others could apply. The agency itself needs to be visible enough so that people know about it. It would probably need some checks against bad actors, because it would be tempting to extract work from volunteers. As I said, this is a problem someone might choose to solve.
Without systematic solution, I suppose you could simply post an offer here, explaining your skills and the amount of time you can donate, and see what happens. (Possibly nothing. Possibly something even more disappointing than nothing.) Or you could do the same in person at your local meetup.
The post Slack Club by The Last Rationalist describes a problem in the rationalist community where there is an overabundance of Ender Wiggins: people trying to be clever as a way of avoiding having to do work. HPMOR and other rationalist media lionizes this sort of behavior and encourages people to try and fit into this archetype while looking down on people who don't. Why Our Kind Can't Cooperate from the Craft and the Community sequence also points to the same general cluster of behavioral trends. I'm not saying this is universal, but there does seem to be something to it.
I've probably been influenced by this myself. Over the years I have collected a large pile of various failed or aborted projects in my wake. Things that just didn't work out or didn't go the way I had planned or hoped. As a result, it's become increasingly clear to me that I just don't make a very good Ender Wiggin. I'm at least getting decent at noticing that I'm not very good at this, but noticing my inability to discern good ideas from bad ones doesn't translate out into having better discernment. I have improved my discernment somewhat as I’ve learned and aged, but I’m still noticeably below average in this area.
For problems where I can just throw lots of low-cost ideas at the wall and see what sticks, this is fine, but where there’s a higher risk or cost associated with each attempt, it’s much less so. I have a lot of strengths, but discernment will probably never be one of them and it seems wise to admit that and leave the discernment to people who are better equipped to do it.
I'm not a very good Ender Wiggin, that just isn't where my strengths lie. But Ender couldn’t have won the Formic War on his own, Ender needed Bean, and Petra, and all the other members of his Jeesh. I might not be a very good Ender, but I could be a good Bean. I’m motivated, intelligent, value-aligned, and hardworking. With how many Enders there are in this community it seems like I should be able to find someone to work under.
A friend of mine questions whether this is actually describing a useful dynamic, or if my time would be better spent more time on personal improvement before trying to take actions that improve the world. I don't know if I agree with this and it seems like if I can make useful contributions now, it would be better for the world than if I waited.
So this question is really a few questions:
1) Is this a useful dynamic? If I can find someone to be the Samurai to their Daimyo, is that something which would be of benefit to the project of rationality?
2) If it is a useful dynamic, how might I go about finding someone to work under?
3) Am I, as a person, actually capable of making a positive difference in general or is my presence generally going to prove useless or detrimental? Is this something I can improve and if so, how?