= 27d567bc2d48d9f40e9ccc20ea370b15)
Perplexed comments on Controlling Constant Programs - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (33)
I don't believe you can obtain proper information about the world through observation (unless you are a human and don't know what you want). Preference is unchanging and is defined subjectively, so formal agents can't learn new things about it (apart from resolution of logical uncertainty). Observations play a role in how plans play out (plans are prepared to be conditional on observations), or in priority for preparing said plans as observations come in, but not as criteria for making decisions.
On the other hand, observations could probably be naturally seen as constructing new agents from existing ones (without changing their preference/concept of environment). And some notion of observation needs to be introduced at some point, just as a notion of computational time.
So it sounds like you are saying that the agent() program only represents the decision-theory portion of the agent. The Bayesian cognitive portion of the agent and the part of the agent that prefers some things over others are both modeled in world(). Communication with other agents, logic, all these features of agency are in world(), not in agent().
May I suggest that "agent()" has been misnamed? Shouldn't it be called something more like choice() or decision()? And shouldn't "world()" be named "expected-value-of-decision-by-agent()", or something like that?
Note that world() is part of the agent(). Certainly, world() will become preference in the next post (and cease to be necessarily a program, while agent() must remain a program), but historically this part was always "world program", and preference replaces (subsumes) that, which is not an obvious step.
I want to see it!