This theory proposes a natural game-theoretic constraint on hostile artificial superintelligence (ASI) behavior based on cosmic risk assessment: 1. If ASIs tend toward hostility, our potential ASI would likely not be the first in the cosmos 2. More advanced ASIs would have evolutionary/technological advantages over newer ones 3. Therefore, any...
As AI technology continues to advance, it is becoming increasingly likely that we will see the emergence of superintelligent AI in the near future. This raises a number of important questions and concerns, as we have no way of predicting just how intelligent this AI will become, and it may...
I think the main idea I was pushing for here is that the probability function is likely to have the gradient described because of the unknowables involved and the infinite loss curve
This theory proposes a natural game-theoretic constraint on hostile artificial superintelligence (ASI) behavior based on cosmic risk assessment:
When combined with Dark Forest theory, this creates an infinite risk for minimal gain. A rational ASI calculating expected utility would recognize that:
This creates strong selective pressure favoring cooperative or non-expansionist ASIs. Even if the probability of more... (read more)
Reddit post submitted at the same time with more comments:
https://www.reddit.com/r/singularity/comments/100soau/alignment_anger_and_love_preparing_for_the
Thank you!
As AI technology continues to advance, it is becoming increasingly likely that we will see the emergence of superintelligent AI in the near future. This raises a number of important questions and concerns, as we have no way of predicting just how intelligent this AI will become, and it may be beyond our ability to control its behavior once it reaches a certain level of intelligence.
Ensuring that the goals and values of artificial intelligence (AI) are aligned with those of humans is a major concern. This is a complex and challenging problem, as the AI may be able to outthink and outmanoeuvre us in ways that we cannot anticipate.
One potential solution would... (read 166 more words →)
Based on my hypothesis above, any such scuffle would likely be categorized as unfriendly action?