From 2023's perspective, people should have been encouraged (not discouraged) from building AI like this.
This is too much of a bare assertion to be a good rationality quote.
"Who wants to live forever when love must die?"
Yes, the average human is dangerously easy to manipulate, but imagine how bad the situation would be if they didn't spend a hundred thousand years evolving to not be easily manipulated.
Beautiful.
That seems like a useful concept to me. What's your argument it isn't?