= 27d567bc2d48d9f40e9ccc20ea370b15)
amcknight comments on General purpose intelligence: arguing the Orthogonality thesis - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (156)
AIXI's "reward number" is given directly to it via an input channel, and it's non-trivial to change it so that it's equal to "the number of paperclips in the territory as far as you know". UDT can be seen as a step in this direction.
I don't see how UDT is a step in this direction. Can you explain?
UDT shows how an agent might be able to care about something other than an externally provided reward, namely how a computation, or a set of computations, turn out. It's conjectured that arbitrary goals, such as "maximize the number of paperclips across this distribution of possible worlds" (and our actual goals, whatever they may turn out to be) can be translated into such preferences over computations and then programmed into an AI, which will then take actions that we'd consider reasonable in pursue of such goals.
(Note this is a simplification that ignores issues like preferences over uncomputable worlds, but hopefully gives you an idea what the "step" consists of.)