Oh nice! I love that book, but had forgotten it also took a stance on this. Thank you for digging up the quote.

I mean, I think it's true. Nuclear winter was the only plausible story for even a full-out nuclear war causing something close to human extinction, and I think extreme nuclear winter is very unlikely. 

Similarly, it is very hard to make a pathogen that could kill literally everyone. You just have too many isolated population, and the human immune system is too good. It might become feasible soon, but it was not very feasible historically!

What actually happens is the plumber says "it's probably X but I need to check out the sink." You can try your luck on X but more likely they make a house call and you're billed for a house call even if you insist on turning the wrench yourself. The next time you have an issue, you can try the same fix again, but if it has a different cause you're calling the plumber again.

As I clarified above, don't do remote instruction! This is definitely not what I meant. I meant "have the plumber come by in person, show you how to solve the problem in person, then have him get paid for his time and be happy and leave".

If you don't have a person on your staff who already knows, and if the plumber isn't on call, I hope you are paying the plumber for calling him and asking him to explain, while knowing in advance that your plans won't include hiring him.

Sorry, the thing you do is hire the plumber and ask "hey, can you help me fix an issue with my sink/toilet/whatever" (de-facto Lightcone has full-time staff with this skillset, so this exact thing hasn't come up in a long while, but it's how we got started). 

Then, when they show up, you explain what you've observed the problem to be and ask "what is the first thing you would do yourself to figure out what's up?". Then they would either tell you, or start doing the thing. Then you follow them and watch with them, or try it yourself (don't be dumb and open up random valves, ask him to watch you and stop you before you do anything dumb). 

In my experience most contractors I have called love to talk about their work and are happy to get paid to explain what they do. Also, the vast majority of problems really are very easy to understand. The usual way this kind of stuff works tends to be: 

Me: "Hey, so we have this toilet seat which has some water coming up below the toilet when you flush."
Them: "Oh, yeah, that's a super common problem. These things have a seal below that often comes loose."
Me: "How would you fix it?" 
Them: "Oh, yeah, I would just open up these bolts, then lift up the toilet, then look at the seal. There are like 3 kinds of seals different toilets use. One always tends to cause problems." 
Me: "Mind if I do it?" 
Them: "No, not at all. Here let me help you hold the seat"
Me: "Oh, cool, so this is the part below the toilet. I can see some standing water here around this notch, is that where the seal is?" 
Them: "No, that's just the lowest point of the toiler, the seal is actually over here". 
Me: "Ah, I see. This thing?" 
Them: "Yes" 
Me: "How would you tell it's broken? Is it broken?"
Them: "Let me take a look" leans over "Hmm, yeah, I am not sure, but it does look a bit wrong. See here? This corner should usually be flush, but it's elevated. My guess is someone installed it wrong."
Me: "Oh, interesting. Does this kind of installation error happen frequently?" 
Them: "I've seen it a few times, but usually people mess up in different ways"
Me: "Great, so we replace it?"
Them: "Yeah"
Me: "Is it this part?" shows Home depot website with product page open
Them: "Nope, that's the wrong part for this toilet, you want this one" points at a related listing
Me: "Do you have one with you?" 
Them: "Yeah, I happen to have that part with me, let me grab it"

Then you install the part, test it, it works. Everyone is happy, and if you had to you could do the same kind of task next time. If you've done this 4-5 times, you honestly will be able to solve most common plumbing problems yourself. It took you maybe 2 hours in total to learn.

Who's going to do the event invoices when you're fixing the plumbing yourself? You have comparative advantage over the plumber; it doesn't matter that fixing the plumbing yourself instead of having the plumber do it benefits you, because doing the event invoices benefits you more.

I mean, the whole point of this post is to argue that it's better for the organization if you learn how to do it yourself. Next time you have a decent chance of being able to solve the problem yourself in 5 minutes, which will save you much more time than the 30 minutes you spent calling around for good plumbers, and the 10 hours you wasted when you hired a bad plumber and they flooded the whole bathroom and forced us to replace half of the wooden floor because you weren't able to evaluate competence in the space.

Doing the plumbing yourself requires more skill (and probably equipment) than is necessary to understand the plumbing.

Yeah, equipment and things that are dangerous are often two things that are an obstacle to people getting to the point of fully doing it themselves. I do not encourage people to run their own electrical lines, even if they know how it works. Not worth the risk. But it's quite easy to basically understand all the steps involved in doing so and use that to become well-calibrated at evaluating the difficulty of a task, or to speed someone else up with a task.

But still cheaper than learning to renovate a kitchen and doing it.

It's really not hard to learn how to renovate a kitchen! I have done it. Of course, you won't be able to learn how to do it all quickly or to a workman's standard, but I had my contractor show me how to cut drywall, how installing cabinets works, how installing stoves works, how to run basic electrical lines, and how to evaluate the load on an electrical panel. The reports my general contractor was delegating to were also all mostly working on less than 30 hours of instruction for the specific tasks involved here (though they had more experience and were much faster at things like cutting precisely).

My guess is learning how to do this took like 20 hours? A small fraction of what a kitchen renovation took, and a huge boost to my ability to find good contractors. 

This is the kind of mentality I don't understand and want to avoid at Lightcone. Renovating a kitchen is not some magically complicated tasks. If you really had to figure out how to do it fully on your own you could probably just learn it using Youtube tutorials and first-principles reasoning in a month or two. Indeed, you will be able to directly watch the journeys of people who have done exactly that on Youtube, so you can even see what likely goes wrong and not make the same mistakes.

Of course, then don't do it all on your own, but it's really not that hard to get to a point where you could do it on your own, if slowly.

Pretty decent chance it doesn't scale! I am not choosing the principles for Lightcone based on what I expect to scale to hundreds of people. Larger organizations probably need different principles.

That said, in my experience when I look into competent companies, this holds surprisingly true. The best executives are very strong generalists, and the best managers are chosen to be strong performers in the task that they are managing other people to do. Elon is widely known to be a strong engineer, as well as a strong designer, and spends much of his time arguing details of that kind of work with his reports. Mark Zuckerberg is known to do similarly. Executives at their companies are also selected to be strong generalists capable of performing (to an acceptable standard) a large fraction of the work going on in the organization.

And they are spending all their time doing things they do not know how to do

I don't know where you picked this up. In my experience getting to a point where you can perform a task at a basic level takes maybe a few days, and it's rare that you need to add a whole modality of tasks to your domain of management. I do spend a lot of my time learning new skills and figuring out how to do things I've never done before, but it's like 20% of my time, not more than that.

In my experience most people just suck at learning new things, and vastly overestimate the depth of expertise. It doesn't take that long to learn how to do a thing. I have never written a song (without AI assistance) in my life, but I am sure I could learn within a week. I don't know how to draw, but I know I could become adequate for any specific task I am trying to achieve within a week. I have never made a 3D prototype in CAD and then used a 3D printer to print it, but I am sure I could learn within a few days.

And those are three of the more complicated tasks that I am tracking as bigger holes in my skillset! Learning how to clean a room to hotel standards, or learning how to fix most issues with a broken toilet, or how to set up a podcast studio, or learn a new programming language, or put up a wall, are much simpler and can be learned in a few hours. Lightcone as an organization works on a very wide range of projects, and I've had no issues achieving basic competence at all the kinds of things we do. I don't see any particular issue for why not at least my executives should not be able to do the same for things in their purview.

Sure! I definitely agree that's going on a lot as well. But I think that kind of deception is more common in the rest of the world, and the things that set this community apart from others is the ability to do something more intentional here (which then combined with plenty of self-deception can result in quite catastrophic outcomes, as FTX illustrates).

I've been around the community for 10 years. I don't think I've ever seen this?

Also, come on, this seems false. I am pretty sure you've seen Leverage employees do this, and my guess is you've seen transcripts of chats of this happening with quite a lot of agency at FTX with regards to various auditors and creditors.

I do think it's not good! But also, it's an important issue and you have to interface with people who aren't super principled all the time. I just don't want people to think of the AI Safety community as some kind of community of saints. I think it's pretty high variance, and you should have your guard up a good amount. 

Is this not common in politics? I thought this was a lot of what politics was about.

I have been very surprised by how non-agentic politics is! Like, there certainly is a lot of signaling going on, but when reading stuff like Decidingtowin.org it becomes clear how little optimization actually goes into saying things that will get you voters and convince stakeholders. 

I do think a lot of that is going on there, and in the ranking above I would probably put the current political right above AI safety and the current political left below AI safety. Just when I took the average it seemed to me like it would end up below, largely as a result of a severe lack of agency as documented in things like deciding-to-win.

Re corporate campaigns: I think those are really very milquetoast. Yes, you make cool ads, but the optimization pressure here seems relatively minor (barring some intense outliers, like Apple and Disney, which I do think are much more agentic here than others, and have caused pretty great harm in doing so, like Disney being responsible for copyright being far too long in the US because Disney was terribly afraid of anyone re-using their characters and so tainting Disney's image).