Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: time4 10 June 2016 02:32:40AM 2 points [-]

The AI might eventually conclude that the "JFK" in it's prior didn't refer to President John F. Kennedy but to some other guy with the initials JFK.

More interestingly, if the prior was specified as "The president isn't assassinated on November 22, 1963" it might conclude that John F Kennedy wasn't the president, it might look at the election of 1960, notice that it was one of the closest in American history and that there was some funny business with the election in Chicago and Texas, and thus conclude that the legitimate president was Richard Nixon who in fact wasn't assassinated on November 22, 1963.

From here it could go in any number of directions, e.g., conclude that since the president Kennedy was illegitimate then all the laws he signed as president were illegitimate and that it should therefore set about preventing them from being enforced. Or more generally develop a somewhat eccentric theory of what it means to be a legitimate elected official and act accordingly.

Note, that in the above scenario the AI doesn't have any "material" beliefs different from not having the not-assassinated prior but is still likely to act differently.

Comment author: time4 10 June 2016 02:32:11AM 0 points [-]

I always interpreted the quote as a witty way to tell someone to shut up.