"Computers can add numbers much more accurately than humans. They can draw better pictures than humans. They can play better chess. See the pattern? Well, AIs will soon be able to generate desired outcomes for society better than humans can.
I feel that the AI Alignment discourse has become somewhat detached from both reality and from sane theory.
This post is an attempt to correct that state of affairs."
I think you don't know much about deep practices of double speak in Russia. For example, I recently recognized that what was presented for us a nice adventure movie in 1980s, was a homoerotic drama about relation of two pairs of man. Also recent Prigozhin mutiny implies that control is not really effective.
Anyway, I see alignment as a reverse problem: not as a governor having problem to control population, but population having problem to control the governor. For example, corruption is a typical example of misalignment.