One of the reasons I care about friendly AI because I live in a uniquely dysfunctional country, India.
India has a long remembered history, thousands of communities and extreme religiosity. It would have been good if those communities had been allowed to go their own ways, but bolted onto this was a socialist government that has been the largest dog-in-the-manger in the history of humankind. In addition, the country is a one-man one-vote system, where the politicians promise short term goods to people and resources that should be used to build physical and social infrastructure are being frittered away.
In short, governing India is a FAI hard problem, atleast looks to me that way. Any little that I can contribute to that would be worth it. 1.27 billion Indian people would be very grateful if a genuinely benevolent friendly AI emerged.
It's way late in my time zone and I suspect this question isn't technically coherent on the grounds that the right answer to "why care about AI risk?" is going to be complicated and have a bunch of parts that can't be separated from each other. But I'm going to share a thought I had anyway.
It seems to me like probably, the answer to the question of how to make AIs benevolent isn't vastly more complicated than the answer of how to make them smart. What's worrisome about our current situation, however, is that we're currently putting way more effort into making AIs smart than we are into making them benevolent.
Agree? Disagree? Have an orthogonal answers to the title question?