= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on Evaluating the feasibility of SI's plan - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (186)
Yes, it is this layered approach that the OP is asking about -- I don't see that SI is trying it.
In what way would SI be 'trying it'? The point about multiple layers of security being a good idea for any seed AI project has been made at least as far back as Eliezer's CFAI and brought up periodically since with innovations like the suicide button and homomorphic encryption.
I agree: That sort of innovation can be researched as additional layers to supplement FAI theory
Our question was -- to what extent should SI invest in this sort of thing.
My own view is 'not much', unless SI were to launch an actual 'let's write AGI now' project, in which case they should invest as heavily as anyone else would who appreciated the danger.
Many of the layers are standard computer security topics, and the more exotic layers like homomorphic encryption are being handled by academia & industry adequately (and it would be very difficult for SI to find cryptographers who could advance the state of the art); hence, SI's 'comparative advantage', as it were, currently seems to be in the most exotic areas like decision theory & utility functions. So I would agree with the OP summary:
Although I would amend 'heuristic AGIs' to be more general than that.
That's all the more reason to publish some articles on how to apply known computer security techniques to AGI. This is way easier (though far less valuable) than FAI, but not obvious enough to go unsaid.
Yes. But then again, don't forget the 80/20 rule. There may be some low-hanging fruit along other lines than FAI -- and for now, no one else is doing it.