Good points all; these are good reasons to work on AI safety (and of course as a theorist I'm very happy to think about interesting problems even if they don't have immediate impact :-) I'm definitely interested in the short-term issues, and have been spending a lot of my research time lately thinking about fairness/privacy in ML. Inverse-RL/revealed preferences learning is also quite interesting, and I'd love to see some more theory results in the agnostic case.
Hi all,
Thanks for the very thoughtful comments; lots to chew on. As I hope was clear, I'm just an interested outside observer, and have not spent very long thinking about these issues, and don't know much of the literature. (My blog post ended up as a cross post here because I posted it to facebook, and asked if anyone could point me to more serious literature thinking about this problem, and a commenter suggested that I should crosspost here for feedback)
I agree that linear feedback is more plausible if we think of research breakthroughs as producing multiplicative gains, a simple point that I hadn't thought about.
Thanks!
Sure
I've been having fun recently reading about "AI Risk". There is lots of eloquent writing out there about this topic: I especially recommend Scott Alexander's Superintelligence FAQ for those looking for a fun read. The subject has reached the public consciousness, with high profile people like Stephen Hawking and Elon Musk speaking publicly about it. There is also an increasing amount of funding and research effort being devoted to understanding AI risk. See for example the Future of Humanity Institute at Oxford, the Future of Life Institute at MIT, and the Machine Intelligence Research Institute in Berkeley, among others. These groups seem to be doing lots of interesting research, which I am mostly ignorant... (read 2458 more words →)
Thanks for the corrections. I changed the text to "in Berkeley". How should FLI be described? (I was just cribbing from Scott's FAQ when claiming it was at MIT)