Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

ThrustVectoring comments on Probability is in the Mind - Less Wrong

60 Post author: Eliezer_Yudkowsky 12 March 2008 04:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (190)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: ThrustVectoring 24 May 2013 03:17:56PM 3 points [-]

There are only so many distinct states of experience, so yes, causal models are countable. The set of all causal models is a set of functions that map K n-valued past experiential states into L n-valued future experiential states.

This is a monstrously huge number of functions in the set, but still countable, so long as K and L are at most countably infinite.

Note that this assumes that states of experience with zero discernible difference between them are the same thing - eg, if you come up with the same predictions using the first million digits of sqrt(2) and the irrational number sqrt(2), then they're the same model.

Comment author: Watercressed 24 May 2013 03:46:38PM 1 point [-]

But the set of causal models is not the set of experience mappings. The model where things disappear after they cross the cosmological horizon is a different model than standard physics, even though they predict the same experiences. We can differentiate between them because Occam's Razor favors one over the other, and our experiences give us ample cause to trust Occam's Razor.

At first glance, it seems this gives us enough to diagonalize models--1 meter outside the horizon is different from model one, two meters is different from model two...

There might be a way to constrain this based on the models we can assign different probabilities to, given our knowledge and experience, which might get it down to countable numbers, but how to do it is not obvious to me.

Comment author: Watercressed 24 May 2013 04:15:32PM 0 points [-]

Er, now I see that Eliezer's post is discussing finite sets of physical laws, which rules out the cosmological horizon diagonalization. But, I think this causal models as function mapping fails in another way: we can't predict the n in n-valued future experiential states. Before the camera was switched, B9 would assign low probability to these high n-valued experiences. If B9 can get a camera that allows it to perceive color, it could also get an attachment that allows it to calculate the permittivity constant to arbitrary precision. Since it can't put a bound on the number of values in the L states, the set is uncountable and so is the set of functions.

Comment author: ThrustVectoring 24 May 2013 09:26:40PM *  0 points [-]

we can't predict the n in n-valued future experiential states.

What? Of course we can - it's much simpler with a computer program, of course. Suppose you have M bits of state data. There are 2^M possible states of experience. What I mean by n-valued is that there are a certain discrete set of possible experiences.

If B9 can get a camera that allows it to perceive color, it could also get an attachment that allows it to calculate the permittivity constant to arbitrary precision.

Arbitrary, yes. Unbounded, no. It's still bounded by the amount of physical memory it can use to represent state.

Comment author: Watercressed 24 May 2013 09:41:53PM 0 points [-]

In order to bound the states at a number n, it would need to assign probability zero to ever getting an upgrade allowing it to access log n bytes of memory. I don't know how this zero-probability assignment would be justified for any n--there's a non-zero probability that one's model of physics is completely wrong, and once that's gone, there's not much left to make something impossible.