I mean a moral terminal goal. But I guess we would be a large step closer to a solution of the control problem if we could specify such a goal.
What I had in mind is something like this: Evolution has provided us with a state which everyone prefers who is healthy (who can survive in a typical situation in which humans have evolved with high probability) and who has an accurate mental representation of reality. That state includes being surrounded by other healthy humans, so by induction everyone must reach this state (and also help others to reach it). I haven't carefully thought this through, but I just want to give an idea for what I'm looking for.
Evolution doesn't produce terminal goals.
The most recent post in December's Stupid Questions article is from the 11th.
I suppose as the article's been pushed further down the list of new articles, it's had less exposure, so here's another one for the rest of December.
Plus I have a few questions, so I'll get it kicked off.
It was said in the last one, and it's good advice, I think:
This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.
Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.