I've noticed that there are 2 flavors of the alignment problem. One is about the technical how to and practical engineering, the other is about humanities, social sciences, human behavior and psychology.
What they both have in common is doomsday scenarios about paperclip maximizers.
I don't know about you, but personally I don't care if an unaligned AI is aware of its experience while turning the universe into a paperclip, or if the whole thing is happening entirely without consciousness. What i care about is: How can we make AI smart enough to prevent that from happening?
That said, I find especially in amateur circles that CONSCIENCE is not just omitted from the discussion, it's actually poorly understood.
Arguably, if we find an algorithm for empathy, we can solve the alignment problem rather easily. Humans seem to be capable of monitoring their actions for morality and preventing harm to others, how hard can it be to build an artificial brain that can do the same?
How would you explain to someone what the difference is between consciousness and conscience?
Conscience, as you've defined it, is value alignment. If the AI values the same things that we do, when offered a choice between two courses of action, it will choose the one that serves to enhance human values rather than degrade them. Designing an AI that does this, with no exceptions, is very hard.
Correct! That's my point with the main post. I don't see anyone discussing conscience, I mostly hear them contemplate consciousness or computability.
As far as how to actually do this, I've dropped a few ideas on this site, they should be listed on my profile.