MichaelVassar comments on Pascal's Muggle: Infinitesimal Priors and Strong Evidence - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (404)
It seems to me like the whistler is saying that the probability of saving knuth people for $5 is exactly 1/knuth after updating for the Matrix Lord's claim, not before the claim, which seems surprising. Also, it's not clear that we need to make an FAI resistant to very very unlikely scenarios.
I'm a lot more worried about making an FAI behave correctly if it encounters a scenario which we thought was very very unlikely.
Also, if the AI spreads widely and is around for a long time, it will eventually run into very unlikely scenarios. Not 1/3^^^3 unlikely, but pretty unlikely.