Selection bias. We're using driving as an example because it turned out that humans are actually good enough at it. Lots of other things that humans aren't good enough at simply weren't done before automation and computers.
All this really tells us is that we're good at some things that weren't in our ancestral environment. Which we know already (we can do math!).
For things like building AGI, that no human has done, and that for which we don't yet have a coherent theory or roadmap (other than 'copy this hugely complicated black box'), we don't know how easy or difficult they really are. We can get an outside view by comparing with other tasks that we once didn't know how to do and then succeeded on some and failed on others despite a lot of effort. But I think there's a lot of variation between cases and prediction is hard.
We're using driving as an example because it turned out that humans are actually good enough at it.
Plus, our example is specifically driving at the skill level that humans are capable of.
It feels to me like we could drive safely while a little drunk, if we stuck to 20mph and wide roads with shallow turns, and if everyone else did the same. (I haven't driven in years, and never while drunk, so I might be wrong. Even if I'm right, other people do not do the same. Don't do this.) If that was the normal difficulty level to drive at, we might say that humans...
There's been a lot of fuss lately about Google's gadgets. Computers can drive cars - pretty amazing, eh? I guess. But what amazed me as a child was that people can drive cars. I'd sit in the back seat while an adult controlled a machine taking us at insane speeds through a cluttered, seemingly quite unsafe environment. I distinctly remember thinking that something about this just doesn't add up.
It looked to me like there was just no adequate mechanism to keep the car on the road. At the speeds cars travel, a tiny deviation from the correct course would take us flying off the road in just a couple of seconds. Yet the adults seemed pretty nonchalant about it - the adult in the driver's seat could have relaxed conversations with other people in the car. But I knew that people were pretty clumsy. I was an ungainly kid but I knew even the adults would bump into stuff, drop things and generally fumble from time to time. Why didn't that seem to happen in the car? I felt I was missing something. Maybe there were magnets in the road?
Now that I am a driving adult I could more or less explain this to a 12-year-old me:
1. Yes, the course needs to be controlled very exactly and you need to make constant tiny course corrections or you're off to a serious accident in no time.
2. Fortunately, the steering wheel is a really good instrument for making small course corrections. The design is somewhat clumsiness-resistant.
3. Nevertheless, you really are just one misstep away from death and you need to focus intently. You can't take your eyes off the road for even one second. Under good circumstances, you can have light conversations while driving but a big part of your mind is still tied up by the task.
4. People can drive cars - but only just barely. You can't do it safely even while only mildly inebriated. That's not just an arbitrary law - the hit to your reflexes substantially increases the risks. You can do pretty much all other normal tasks after a couple of drinks, but not this.
So my 12-year-old self was not completely mistaken but still ultimately wrong. There are no magnets in the road. The explanation for why driving works out is mostly that people are just somewhat more capable than I'd thought. In my more sunny moments I hope that I'm making similar errors when thinking about artificial intelligence. Maybe creating a safe AGI isn't as impossible as it looks to me. Maybe it isn't beyond human capabilities. Maybe.
Edit: I intended no real analogy between AGI design and driving or car design - just the general observation that people are sometimes more competent than I expect. I find it interesting that multiple commenters note that they have also been puzzled by the relative safety of traffic. I'm not sure what lesson to draw.