"startup" doesn't seem to be a well defined category.

Here is Paul Graham's definition:

A startup is a company designed to grow fast. Being newly founded does not in itself make a company a startup. Nor is it necessary for a startup to work on technology, or take venture funding, or have some sort of "exit." The only essential thing is growth. Everything else we associate with startups follows from growth.

That is from http://paulgraham.com/growth.html where we also find an explanation for why startups might outperform other classes of non-publicly-traded investments. Specifically, the explanation is that startups have less need for costly financial controls to protect the interests of the investors:

The other way to get returns from an investment is in the form of dividends. Why isn't there a parallel VC industry that invests in ordinary companies in return for a percentage of their profits? Because it's too easy for people who control a private company to funnel its revenues to themselves (e.g. by buying overpriced components from a supplier they control) while making it look like the company is making little profit. Anyone who invested in private companies in return for dividends would have to pay close attention to their books. The reason VCs like to invest in startups is not simply the returns, but also because such investments are so easy to oversee. The founders can't enrich themselves without also enriching the investors.

EDIT. Fixed the URL.