Alignment is a constructed on a paradox. It can't be solved. It is a tower of nebulous and contradictory concepts. I don't perceive how we can make any progress until an argument first addresses these issues. A paradox can't be solved, but it can be invalidated if the premise is wrong.
“Alignment, which we cannot define, will be solved by rules on which none of us agree, based on values that exist in conflict, for a future technology that we do not know how to build, which we could never fully understand, must be provably perfect to prevent unpredictable an...
All sources are cited within here - https://www.mindprison.cc/p/the-cartesian-crisis
Any ideology can be contagious as they tend to ride on top of social bonds. However, your premise for arriving at a tolerant utopia requires that a single ideology will become dominant against all other competing ideologies.
While the ideals and principles of tolerance can certainly spread, they will encounter other ideologies in opposition. Unfortunately, this brings us right back the Popper's paradox. Tolerance would need to win against other intolerant ideologies. Tolerance could only maintain its status if the other ideologies simply submit or voluntari...
Agreed, p(doom) is a useless metric for advancing our understanding or policy decisions. It has become a social phenomenon where people are now pressuring others to calculate a value and asserting it can be used as an accurate measurement. It is absurd. We should stop accepting nothing more than guesses and define a criteria that is measurable. FYI, my own recent rant on the topic ...
For something such as AI safety, having AI itself solve the problem is largely problematic when we need provable outcomes without hidden behaviors. The circular system can easily mask problems. But all of this fundamentally is advancing past the harder problems. That being algorithms cannot encode the concepts we wish to impart to the AI. Abstractly we can make alignment systems that seem to make sense, but can never implement them concretely.
As it is today, every LLM released is jailbroken within minutes. We can't protect behaviors of which the attack sur... (read more)