= 27d567bc2d48d9f40e9ccc20ea370b15)
BaconServ comments on MIRI strategy - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (94)
It would be far more useful if MIRI provided technical argumentation for its Scary Idea. There are a lot of AGI researchers, myself included, which remain entirely unconvinced. AGI researchers - the people who would actually create an UFAI - are paying attention and not sufficiently convinced to change their behavior. Shouldn't that be of more concern than a non-technical audience?
A decade of effort on EY's part has taken the idea of friendliness mainstream. It is now accepted as fact by most AGI researchers that intelligence and morality are orthonormal concepts, despite contrary intuition, and that even with the best of intentions a powerful, self-modifying AGI could be a dangerous thing. The degree of difference in belief is in the probability assigned to that could.
Has the community responded? Yes. Quite a few mainstream AGI researchers have proposed architectures for friendly AGI, or realistic boxing/oracle setups, or a friendliness analysis of their own AGI design. To my knowledge MIRI has yet to engage with any of these proposals. Why?
I want a believable answer to that before a non-technical pamphlet or video, please.
In other words, all AGI researchers are already well aware of this problem and take precautions according to their best understanding?
s/all/most/ - you will never get them all. But yes, that's an accurate statement. Friendliness is taught in artificial intelligence classes at university, and gets mention in most recent AI books I've seen. Pull up the AGI conference proceedings and search for "friendly" or "safe" - you'll find a couple of invited talks and presented papers each year. Many project roadmaps include significant human oversight of the developing AGI, and/or boxing mechanisms, for the purpose of ensuring friendliness proactive response.