I do not see how this has any chance at scaling. Who sits at the root of the delegation tree? The CEO? And they are spending all their time doing things they do not know how to do (as your rule does not allow them to delegate those tasks, and presumably there are enough to take all their time)? That does not sound to me like how competent delegation should look like. And being able to do X vs being able to evaluate someone else doing X are of course related, but still quite different skills.

> Yudkowsky, being the best strategic thinker on the topic of existential risk from AGI This seems strange to say, given that he: 1. decided to aim for "technological victory", without acknowledging or being sufficiently concerned that it would inspire others to do the same 2. decided it's feasible to win the AI race with a small team and while burdened by Friendliness/alignment/x-safety concerns 3. overestimated likely pace of progress relative to difficulty of problems, even on narrow problems that he personally focused on like decision theory (still far from solved today, ~16 years later. Edit: see UDT shows that decision theory is more puzzling than ever) 4. had large responsibility for others being overly deferential to him by writing/talking in a highly confident style, and not explicitly pushing back on the over-deference 5. is still overly focused on one particular AI x-risk (takeover due to misalignment) and underemphasizing or ignoring many other disjunctive risks These seemed like obvious mistakes even at the time (I wrote posts/comments arguing against them), so I feel like the over-deference to Eliezer is a completely different phenomenon from "But you can’t become a simultaneous expert on most of the questions that you care about." or has very different causes. In other words, if you were going to spend your career on AI x-safety, of course you could have become an expert on these questions first.

Thank you for writing this! I think a lot of people miss this point, and keep talking about UBI in the AI future without being clear which power bloc will ensure UBI will continue existing, and why. However, I'd like to make a big correction to this. Your point exactly matches my thinking until a few months ago. Then I realized something that changes it a lot, and is also I think crucial to understand. Namely, elites have always needed the labor of the masses. The labor of serfs was needed, the labor of slaves was needed. That circumstance kept serfs and slaves alive, but not in an especially good position. The masses were exploited by elites throughout most of history. And it doesn't depend on economic productivity either: a slave in a diamond mine can have very high productivity by the numbers, but still be enslaved. The circumstance that changed things, and made the masses in Western countries enjoy (temporarily) a better position than serfs in the past, was the military relevance of the masses. It started with the invention of firearms. A peasant with a gun can be taught to shoot a knight dead, and knights correctly saw even at the time that this would erode their position. I'm not talking about rebellion here (rebellions by the masses against the elites have always been very hard), but rather on whether the masses are needed militarily for large scale conflicts. And given military relevance, economic productivity isn't actually that important. It's possible to have a leisure class that doesn't do much work except for being militarily relevant; knights are a good example. It's actually pretty hard to find historical examples of classes that were militarily relevant but treated badly. Even warhorses were treated much better than peasant horses. Being useful keeps you alive, but exploited; being dangerous is what keeps you alive and treated well. If we by some miracle end up with a world where the masses of people remain militarily relevant, but not needed for productive work, then I can imagine the entire masses becoming such a leisure class. That'd be a nice future if we could get it. However, as you point out, the future will have not just AI labor, but AI armies as well. Ensuring the military relevance of the masses seems just as difficult as ensuring their economic relevance. So my comment, unfortunately, isn't replacing the problem with an easier one; just with a different one.