I don't worry about it that much either; I don't expect anyone to build a recursively self-improving AI in my lifetime, and, well, the people who actually live in the future will probably be better at solving the problems of the future than I would be. "Friendly AI" falls under the category of problems that are "important, but not urgent".
From Mike Darwn's Chronopause, an essay titled "Would You Like Another Plate of This?", discussing people's attitudes to life:
Conclusion, graphs, and references in article. As usual, I recommend reading Chronopause.com as Darwin has many good articles; to quickly link a few: