I've thought about the same thing. Automobiles are relatively unsafe compared to other modes of tranportation, but it is amazing to me how commutes in major metro areas seem so free of major accidents on the majority of days.

However, though I'm pretty ignorant of programming, I'm not sure your analogy works.

Driving is hard. But there are lots of mechanisms in play that make it easy enough. Traffic signals and road markings, for instance. Plus, even if there is a crash, damage is limited in scope based on the nature of the situation.

From what I can gather in regard to LW's views of theoretical AI, there are no helpful lanes or traffic lights. And bad code that leads to UFAI doesn't just cause an isolated incident... it spawns a revolution of hyper-powerful agents with ends in direct conflict to humans.

A fender bender can become the end of existence.