Comment on Noisy Reasoners

Fri, 14 Dec 2012 08:20:28 GMT

Repeated self modification is problematic, because it represents a product of series (though possibly a convergent one, if the AI gets better at maintaining its utility function / rationality with each modification) -- naively, because no projection about the future can have a confidence of 1, there is some chance that each change to the AI will be negative-value.

Discuss