= 27d567bc2d48d9f40e9ccc20ea370b15)
Will_Sawin comments on Controlling Constant Programs - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (33)
I don't believe you can obtain proper information about the world through observation (unless you are a human and don't know what you want). Preference is unchanging and is defined subjectively, so formal agents can't learn new things about it (apart from resolution of logical uncertainty). Observations play a role in how plans play out (plans are prepared to be conditional on observations), or in priority for preparing said plans as observations come in, but not as criteria for making decisions.
On the other hand, observations could probably be naturally seen as constructing new agents from existing ones (without changing their preference/concept of environment). And some notion of observation needs to be introduced at some point, just as a notion of computational time.
I was using "information" loosely. Define it as "that thing you get from observation" if you want.
The point is, you will make different choices if you get different sensory experiences, because the sensory experiences imply something about how you control the world program. You're right that this could be modeled with different agent functions instead of parameters. Interesting - that seems rather deeply meaningful.
You can see it as unconditionally making a single conditional choice. The choice is itself a program with parameters, and goes different ways depending on observation, but is made without regard for observation. As an option, the choice is a parametrized constructor for new agents, which upon being constructed will make further choices, again without parameters.
There are several different possible formulations here. It seems like they will lead to the same results. The best one is, I suppose, the most elegant one.
Not sure which is.