See this article, and the links to the NREL for cost data (which Kurzweil does also talk about). Solar energy output per dollar has been improving with a doubling time of about a decade for several decades. If that trend continues, then it will be cheap relative to existing alternatives by the time Kurzweil projects gigantic market share. And prior to that there are markets in areas where competing electricity is expensive (theft on the lines in India, lack of connection to the grid in poor areas, the correlation of solar with peak load for air conditioning, places with carbon taxes) to absorb a lot of growth.

And cost improvements come not only from efficiency in absorbing flux, but from reduced use of materials, more efficient manufacturing processes, and so forth. Balance-of-system costs have also been going down. Distribution costs apply to other power sources (although distributed solar in some places benefits because homeowners can use the solar themselves, and free ride off the utilities for distribution, an implicit subsidy for early growth). Non-arable desert land is not particularly high value, nor are roofs.