LESSWRONG
LW

303
Daniel Kokotajlo
29234Ω392011034220
Message
Dialogue
Subscribe

Was a philosophy PhD student, left to work at AI Impacts, then Center on Long-Term Risk, then OpenAI. Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI. Now executive director of the AI Futures Project. I subscribe to Crocker's Rules and am especially interested to hear unsolicited constructive criticism. http://sl4.org/crocker.html

Some of my favorite memes:


(by Rob Wiblin)

Comic. Megan & Cueball show White Hat a graph of a line going up, not yet at, but heading towards, a threshold labelled "BAD". White Hat: "So things will be bad?" Megan: "Unless someone stops it." White Hat: "Will someone do that?" Megan: "We don't know, that's why we're showing you." White Hat: "Well, let me know if that happens!" Megan: "Based on this conversation, it already has."
(xkcd)

My EA Journey, depicted on the whiteboard at CLR:

(h/t Scott Alexander)


 
Alex Blechman @AlexBlechman Sci-Fi Author: In my book I invented the Torment Nexus as a cautionary tale Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus 5:49 PM Nov 8, 2021. Twitter Web App

Sequences

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
6Daniel Kokotajlo's Shortform
Ω
6y
Ω
867
Daniel Kokotajlo's Shortform
Daniel Kokotajlo3h20

Correct, the video someone elsethread shared had a guy saying 90%ish of the Russian casualties were drones but only a slight majority of Ukrainian casualties were from drones. Still supports my overall point though -- Russia has a very artillery-heavy force, no one can claim that they didn't invest enough in artillery, and yet still they are getting more kills with their hastily-assembled drone force.

I agree drones aren't going to get much faster or quieter. I think their range and EW-resistance will continue to improve (e.g. due to AI) but other than that they'll stay pretty similar to today. Oh, the price might go down too once they are produced at even greater scale.

They won't rely on dodging autoturrets to hit targets; they'll rely on overwhelming. Suppose you have a perfect autoturret that never misses, but N drones are flying at it simultaneously. What is the smallest N such that at least one drone will get through? I'd be interested to see an analysis that takes into account typical drone speeds, turret rotation/targeting times, and typical distance the drones can creep up before being fired upon (depends on terrain).

 

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo1d50

Both problems are severe. Not enough money is being spent on drone procurement, and what money is being spent is being spent inefficiently. I make no claim about which is worse.

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo1d40

Idk, but I have talked to some people in the USAF and the stories I hear are discouraging. Also I hear that the cool US drone startups sent their stuff to Ukraine and were humbled, the FPV kamikaze designs that actually worked best at scale were mostly homebrew by volunteers. Also, in general, I have come to strongly suspect that classified US military R&D programs are wasteful boondoggles just like the nonclassified ones; the tech is probably pretty great in some sense but (a) takes several times longer to develop than SpaceX would take if they ever became a weapons manufacturer and (b) costs orders of magnitude more. Like, Boeing is responsible for some military R&D and also for the atrocious Starliner system. Why should we think they are doing a much better job for the military than they are for NASA?

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo1dΩ220

In ww2 the Americans thought they had bombsights for high-altitude bombing with 23m CEP, but actually they were more like 370m CEP. (according to quick google). So, terribly inaccurate. I wonder if modern computers and sensors could enable significantly more accurate bombing. (Sensors to pinpoint your position relative to the target + to judge wind conditions, computers to simulate bomb trajectories). I wouldn't be surprised if the answer is yes.

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo1d20

Yeah I agree that the physics favors the autoturrets over the drones. I don't think there will be silent drones and even if they are, visual identification will probably work well enough anyway. (There will totally be fixed-wing drones that turn off their propellers and glide silently towards the target btw...)

But even if you have a perfect autoturret, it can probably only take out, like, 10 drones before one gets through and kills it. So your autoturrets can't cost more than 10x the cost of a drone... so, like, $10k. Also, even if you have a $5k autoturret that can reliably take out 10 drones before they close the distance, the drones are way more mobile and so can concentrate force, retreat, etc. and thus will have a huge role to play even if they generally stay away from autoturret-defended areas & even if every vehicle has an autoturret on it.

So I think drones are well worth investing in on the current margins, even if we assume that autoturret tech will advance by leaps and bounds and achieve perfection in the next few years. Which is a very generous assumption.

 

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo2d60

Oh, also, minor thing that isn't particularly relevant: The 40km drone I heard about wasn't a quadcopter, it was fixed-wing. So, it could probably "go quiet" during the terminal dive towards the target. However, I buy your overall claim that it should be possible to after a few years of R&D make autoturrets that cost-efficiently defend against ~all small drones. (Physics based argument: You don't have to fly, so you can rest your weight on the ground, so you can have more weight, so you can have armor, bigger guns, etc. for the same amount of money. You lose mobility but for defense that's probably fine.)

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo2d1510

Also, I would say drones have already revolutionized military affairs. 70-90% casualties caused by drones? Despite the war being a static war of attrition with large amounts of artillery on both sides, the kind of situation that historically would have led to artillery being the star of the show? (As indeed it was in the first year or two of the war)

Just imagine if you could teleport the UAF of today back into summer '22. They'd absolutely wipe the floor with the Russians. Likewise Russia would have achieved their original maximalist objectives if they began the war with the force they have today.

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo2d20

Thanks. I skimmed the transcript and didn't see any mention of drones, though I was only skimming so perhaps I missed it. I'm lazy right now and not particularly motivated to dig deeper but if anyone has a timestamp I'd be grateful.

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo2d40

As I describe in this old post, I do expect anti-drone-drones to be really important. (Analogy: Flak cannons weren't the only line of defense in ww2, or even the primary one; fighter planes were.)

Reply
Daniel Kokotajlo's Shortform
Daniel Kokotajlo2d112

OK, thanks. I'm halfway through the interview so far and am not convinced. He doesn't really address my cruxes, e.g. he doesn't provide (1) or (2) above as far as I can tell, or anything close. To be clear, I agree that anti-drone defenses should be heavily prioritized and mass-produced, and that after some number of years they will 'catch up' to today's drone technology and be a reasonably effective counter to it. I describe what this world looks like in this post btw. But on the current margin, both small drones and defenses against such seem woefully under-invested-in by Western militaries. Like, several orders of magnitude less spending than would be optimal. It's embarrassing that Ukraine is producing millions of drones a year, and the USA is producing... thousands? Tens of thousands? 

That said, I do also think that the main war the US needs to gear up for is a war over Taiwan, and that war to me seems like one that will mostly take place in the sky and seas surrounding the island, and hence won't see much role for small drones, or ground forces in general.

Reply
Load More
Agency: What it is and why it matters
AI Timelines
Takeoff and Takeover in the Past and Future
No wikitag contributions to display.
114Vitalik's Response to AI 2027
2mo
53
177My pitch for the AI Village
3mo
32
99METR's Observations of Reward Hacking in Recent Frontier Models
3mo
9
139Training AGI in Secret would be Unsafe and Unethical
5mo
15
658AI 2027: What Superintelligence Looks Like
Ω
5mo
Ω
222
183OpenAI: Detecting misbehavior in frontier reasoning models
Ω
6mo
Ω
26
88What goals will AIs have? A list of hypotheses
Ω
7mo
Ω
19
36Extended analogy between humans, corporations, and AIs.
Ω
7mo
Ω
2
158Why Don't We Just... Shoggoth+Face+Paraphraser?
Ω
10mo
Ω
58
65Self-Awareness: Taxonomy and eval suite proposal
Ω
2y
Ω
2
Load More