Comment on Superintelligence 9: The orthogonality of intelligence and goals

Tue, 11 Nov 2014 03:27:21 GMT

Yes, with some technicalities.

If your resources are limited, you cannot follow certain goals. If your goal is to compute at least 1000 digits of Chaitin's constant, sucks to be computable. I think no agent with a polynomial amount of memory can follow a utility function vulnerable to Pascal's Mugging.

Other than those sorts of technicalities, the thesis seems obvious. Actually, it seems so obvious that I worry that I don't understand the counterarguments.

Discuss