You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

solipsist comments on Anti-Pascaline satisficer - Less Wrong Discussion

3 Post author: Stuart_Armstrong 14 April 2015 06:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (7)

You are viewing a single comment's thread.

Comment author: solipsist 15 April 2015 02:10:47AM 2 points [-]

So it has a current utility of (1-ε)10, and can increase this by reducing ε - hence by building even more paperclips.

I take ε to be the probability that something weird is happening like you're hallucinating your paperclips. Why would building more paperclips reduce ε? If you are dreaming, you're just making more dream paperclips.

I'm sure you'd spend your time with trying to find increasingly elaborate ways to probe for bugs in Descartes' demon's simulation. It is not clear to me why your increasingly paranoid bug probes would involve making paperclips.

Comment author: Mac 15 April 2015 04:05:02PM 2 points [-]

I agree that making more paperclips does not reduce ε, but an unsure AI might build more paperclips nonetheless.

If x is the probability that any one paperclip is hallucinated, the AI will never be certain it has created 10 paperclips (or any for that matter) as long as x > 0. But it can increase the probability it has created 10 paperclips by making 3^^^3 of them.

Bug probes may be a more efficient way to increase the probability, but that isn't certain.

Comment author: Stuart_Armstrong 17 April 2015 03:10:27PM 1 point [-]

It is not clear to me why your increasingly paranoid bug probes would involve making paperclips.

It need not. The problem occurs for any measure that burns resources (and probing the universe for bugs in the Descartes demon would be spectacular at burning resources).