amcknight comments on Thoughts and problems with Eliezer's measure of optimization power - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (23)
I think that if you're looking for a useful measure of optimization power, you will not want to use actual achievement rather than potential achievement if you want to have a nicely encapsulated concept that doesn't include properties of the rest of the environment. Clearly the dangerous optimizers are the ones with actual power rather than merely potential power, but I think it's much more clear to just talk about optimization power as potentially dangerous, given an environment conducive to that optimizer.