Yeah, it's tough to come up with the right analogy for this. Perhaps there's a better one? Nuclear weapons? Or maybe analogies are more of a distraction...
Thanks, I hope you're right about IA (vs pure AI). I think it's very possible that won't be the case however as the more autonomous a system is and the more significant it's decisions, the more valuable it will be. And so there will be large financial incentive to an increasing amount of important decisions being made in-silico. Also the more autonomous a system is, the less of part we will play in it by definition, and therefore the less it will be an extension of us. This especially as the size of the in-silico portion is not physically limited to human's cranial volume :). So the portion of AI's decision making vs humans is unbounded. Alignment may or may not result from IA, it's hard to tell. That's why I think we should deliberately build in alignment mechanisms in-silico ahead of time, and seek to achieve something akin to C.E.V. at small scales now.
If AGI emerges from automation, how can we build alignment into that?
I'd rather not bias answers with my specific guesses, but what I'm trying to get at are bottom-up economic, engineering, and organizational realities-based answers. For example, if this question were asked in the year 2000 about the most likely way the next massive communities of people would form, answers might take into account the broadening adoption of the internet, the relative ease with which new web communities could be created with the LAMP stack, and that advertising would be a likely form of revenue. This may have predicted something akin to the social networks and provided some impetus for preventing societal manipulation through the targeted advertising that we saw in the 2016
Here are some specific ways I think AGI could emerge, starting at the most likely, again trying to take into account economic, engineering, and organizational realities.
GPT-X
Mechanism
It may be that autoregressive training is enough as prediction has been theorized to be the main driver of learning for humans. GPT likely needs to be grounded in addition to further scaling and perhaps internal recurrence. However, if GPT progress continues at its current pace, it would be at the top of my list on ways AGI could emerge.
Safety
In this case, grounding the model with embeddings generated from fMRI data could help both with alignment and performance. There are open questions on what the effect of... (read 1129 more words →)