Technical skills
There seems to be evidence that programmer productivity varies by at least an order of magnitude. My subjective sense is that I personally can become a lot more productive.
Conventional wisdom says that it's important to build and iterate quickly. Technical skills (amongst other things) are necessary if you want to build and iterate quickly. So then, it seems worthwhile to develop your technical skills before pursuing a startup. To what extent is this true?
Domain expertise
Furthermore, domain expertise seems to be important:
You want to know how to paint a perfect painting? It's easy. Make yourself perfect and then just paint naturally.
I've wondered about that passage since I read it in high school. I'm not sure how useful his advice is for painting specifically, but it fits this situation well. Empirically, the way to have good startup ideas is to become the sort of person who has them.
The second counterintuitive point is that it's not that important to know a lot about startups. The way to succeed in a startup is not to be an expert on startups, but to be an expert on your users and the problem you're solving for them.
So one guaranteed way to turn your mind into the type that has good startup ideas is to get yourself to the leading edge of some technology—to cause yourself, as Paul Buchheit put it, to "live in the future."
So then, if your goal is to start a successful startup, how much time should you spend developing some sort of domain expertise before diving in?
How many startups do you know of that failed this way?
Twitter and reddit both survived despite major performance issues. And social media has quite low profit per HTTP request served relative to other web businesses. So I'd expect scaling to be much less of an issue if your startup is almost any other area. I also suspect that this kind of scaling failure is becoming rarer and rarer as the skills, resources, and technology to scale up a website get commoditized.
Apparently that kind of "foundational" knowledge gets forgotten after people graduate from college because it doesn't really get used: http://blog.triplebyte.com/bootcamps-vs-college I think a lot of this stuff functions partially as a way to signal intelligence. Although I'll grant that CS degrees are not as bad in this regard as most degrees are.
A compromise approach: If you are running in to a tricky issue with your database, give yourself time to study enough about databases so you have a decent mental model of what's going on under the hood so you can fix the problem. ("Just-in-time learning")
If you're worried about not knowing that a particular CS subfield is relevant to a problem your startup is facing, you could try Steve Yegge's breadth-first learning philosophy: http://steve-yegge.blogspot.in/2006/03/math-for-programmers.html Not all CS subfields are worth learning for this reason though. I think this justification doesn't really work for networking, databases, or operating systems. It might work for algorithms or artificial intelligence. However, I think it's a pretty weak justification overall. I still think diving in is a superior path.
If you have an inherent desire to learn more CS for some reason, you could deliberately pick a project that will require you to pick up some CS knowledge along the way. This will also look better on your resume (since it signals intelligence). It also creates a bit of a barrier to entry for competitors.
As for your points on learning by doing, I'm not sure what to think, but I appreciate and value them. I'm someone who tends towards textbooks and classes, but I've been slowly turning towards learning by doing. Both in theory, and in practice (ie. my life). At some point I plan on thinking about this question more thoroughly, and posting on LW about it. As for the question posed in this post, it's on the condition that learning by doing is not the most effective method (which may or may not be a correct premise, at least in my eyes).