= 27d567bc2d48d9f40e9ccc20ea370b15)
SilasBarta comments on Metaphilosophical Mysteries - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (255)
Also, correct me if I'm wrong, but doesn't the Solomonoff prior bypass the issue of explicit hypotheses? That is, it puts a (non-zero) prior on every (prefix) bitstream of sensory data.
So, it doesn't even seem necessary to talk about what such an agent's "priors on hypotheses" are -- everything it believes is encoded as an expectation of sensory data, and nothing more. It does not explicitly represent concepts like, "this thing is a halting oracle".
Instead, when it encounters a halting oracle, it increases the weight it assigns to expectations of observations of things that are consistent with having been produced by a halting oracle, not the existence of a halting oracle as such.
No matter how uncomputable, or lengthy-to-specify, a function might be, you can always finitely specify your expectation weight on a finite observation prefix stream (i.e. the first n things you observe from the oracle).
So, I don't see how an agent with a Solomonoff prior chokes on an encounter with a halting oracle.
Normal agents won't. Genuinely intelligent agents won't.
I think those who are arguing that it will are imagining an agent with the Solomonoff prior totally wired into them in a manner that they can't possibly unlearn.
But still, even if you have the Occamian prior (which I think is what's meant by the Solomonoff prior), there is no need to unlearn it. You retain a prior on all hypotheses that decreases in weight exponentially with length, and it persists on top of any observations you've updated on. Those new observations, combined with the Occamian prior give you the optimal weights on (prefix) sensory bitstreams, discounting the ruled-out ones and favoring those closer to what you've actually observed.
Even then, it keeps updating in favor of the observations that match what an oracle gives (without having to explicitly represent that they're from an oracle). No penalty from failure to unlearn.
The thing is, there is no one true razor. Different sources have different associated reference machines - some are more like Turing Machines, others are more like CA. If what you are looking at is barcodes, then short ones are pretty rare - and if you go into simulated worlds, sources can have practically any distribution you care to mention.
Yes, you can model these as "complier overhead" constants - which represent the "cost" of simulating one reference machine in another - but that is just another way of saying you have to unlearn the Solomonoff prior and use another one - which is more appropriate for your source.
You can still do that, whatever your reference machine is - provided it is computationally universal - and doesn't have too much "faith".