= 27d567bc2d48d9f40e9ccc20ea370b15)
TheAncientGeek comments on An overall schema for the friendly AI problems: self-referential convergence criteria - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (110)
The common thread I am noticing is the assumption of singletonhood.
Technologically, if you have a process that could go wrong, you run several in parallel.
In human society, an ethical innovator can run an idea past the majority to seems it sounds like an improved version of what they believe already.
It's looking, again, like group rationality is better.
.
Groups converge as well. We can't assume AI groups will have the barriers to convergence that human groups currently do (just as we can't assume that AIs have the barriers to convergence that humans do).
I'm not doubting that groups converge, I am arguing that when a group achieves reflective equilibrium, that is much more meaningful than a singleton doing so, at least as long as there is variation within the group.
There are bad ways to achieve group convergence.
In absolute terms, maybe, but that doesn't stop it being relatively better.
What you are trying to do is import positive features from the convergence of human groups (eg the fact that more options are likely to have been considered, the fact that productive discussion is likely to have happened...) into the convergence of AI groups, without spelling them out precisely. Unless we have a clear handle on what, among humans, causes these positive features, we have no real reason to suspect they will happen in AI groups as well.
The two concrete examples you gave weren't what I had in mind. I was addressing the problem of an AI "losing" values during extrapolation,and it looks like a real reason to me. If you want to prevent an AI undergoing value drift during extrapolation, keep an extrapolated one as a reference. Two is a group minimally.
There may well be other advantages to doing rationality and ethics in groups, and yes, that needs research, and no, that isnt a show stopper.