My question stems from a place of personal experiences where people see a certain solution as the best option and agree to apply the said solution. Only to later fail to follow up. This failure may lead to grave consequences, over and over again but the same mistakes keep getting repeated again and again.
The conclusion so far is that this is caused by some psychological limitations, usually emotional in nature.
An ASI may try to straighten this up for us. But would have to take a support role to us. Is that likely if the ASI develops its own consciousness?
And, it's highly likely that an ASI will have non-reductive emergent properties beyond... (read more)
My main concern is:
Humans can be irrational and illogical, allowing them to let things slide for better or for worse. There are also psychological and reach limitations that put a hard cap on them somewhere.
An AGI will most likely do everything it does rationally and logically. Including emotions. And this may be detrimental to most humans.
it's not at all clear where one would stop
Yes
You know the saying： “I just want him to listen not to solve my problem”
Would an all-reaching AGI accept this?
How would AGI respond to self-sabotaging tendencies?
If it's true that world problems stem from individual preferences which are based on individual perspectives that originate from individual personality traits and experiences, where will an AGI that's hellbent to improve the world stop and accept things as they are and realize that any attempt to improve things may cause more damage to humans than good?
Take social media algorithm for context, keeping everyone in a relatively closed bubble it believes each person should be in and guides each person's decisions by determining what information to show... (read more)
Yes, I do.
Me too.
But I'm adopting the term "AGI-humans" from today.
...