First post please be brutal.
For better or worse I learnt about the Roko's Basilisk Problem that developed from this site and I had an idea I wanted to bounce off the community most acquainted with the problem.
What if everyone knew? The AI in this case punishes people for not pursuing its creation and thereby tacitly allowing suffering to continue. Fear of this punishment compels people to act towards its creation such that the threat or even actual punishment of the few (who know) allows for the good of the many in the future. But what if everyone knew about the problem, the AI would then have no utilitarian incentive to punish anyone... (read more)
Brutal and facile are not the same things. I was hoping more for a categorical, complete, and total annihilation of my arguments, that's what I think brutal to mean.
Regarding the blackmail: blackmail only works to the extent that you take a threat to be credible, I don't believe the threat to be credible. An AI would know the integrity of this belief and reason it would be purposeless to blackmail me. For example, I question that there is ever or could ever be enough information to simulate perfectly another being, its thoughts, emotions, experiences. Such that no simulation could be so accurate as to itself be an extension of me.
When it comes to learning there are two ways of going about it, starting in the shallows and familiarizing yourself with swimming or jumping into the deep end and forcing yourself to learn. Both are effective. So I think this post makes for an excellent first one.
The topic doesn't make rationalists and AI look ridiculous, the responses do.
You don't have to kill anyone, you merely have to imply that they will be killed, such that the probability of future utility being equal or higher to past/present is lower than the probability of it lower than past/present utility. 20-30 years is a lot of people, manipulate events such that in the infinite years that follow there is never a higher probability of there being more people than existed and were aware than in those 20-30 years.
An interesting point I'd add is you don't need this probability to be true, you merely have to believe it to be true. You can only be blackmailed if threats are credible believed. If... (read more)
Regarding the first point: You merely have to ensure that the population that knows but doesn't contribute is larger than the combined past populations that have contributed and the expected future populations. An improbable thing to do but still a solution.
Regarding the second point: If the populations requiring punishment are greater than those that would benefit surely such an AI could never reason in a utilitarian manner that it was better to punish the many for the few. Unless as a result of the AI's actions an individual in the future is consistently always able to experience a higher utility than anyone in the past. So high in fact that it outweighed... (read more)