= 27d567bc2d48d9f40e9ccc20ea370b15)
whpearson comments on Can we create a function that provably predicts the optimization power of intelligences? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (17)
Let me see if I understand this. Basically, you're making a hypothetical in which, if you do something right, a quasi-omniscient being intervenes and makes it not work. If you do something wrong, it lets that happen. Please correct me if I'm wrong, but if this is even close to what you're saying, it seems quite pointless as a hypothetical.
Let us say I am an AI, I want to replace myself with a new program, I want to be sure that this new program will perform the tasks I want done better than me (including creating new better copies of myself), I need to be able to predict how good a program is.
I don't want to have to run it and see as I will have to run it for a very long time to see if the program is better than me a long time in the future. So I want a proof that the new program is better than me. Are such proofs possible? My argument is that they are not if you cannot constrain the environment and make it not reference your proof.