Eugine_Nier comments on Understanding and justifying Solomonoff induction - Less Wrong

1 Post author: gedymin 15 January 2014 01:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (75)

Comments (75)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eugine_Nier 16 January 2014 02:53:57AM -1 points [-]

Yep, you're confused. You can't possibly assign equal probability to every program in an infinite space, because then the sum of all probabilities will diverge (go to infinity), and you need it to sum up to 1. What you do is assign probabilities in a way that the infinite sum will converge to 1, for example by lining all programs up in an infinite list: M1, M2, M3... and assigning probabilities 1/2, 1/4, 1/8, 1/16, 1/32 and so on to infinity. That's essentially what the universal prior does, using a length-sorted infinite list of all possible programs (I'm skipping a few small technical difficulties).

Only if you insist on assigning non-zero probability to each individual program.

Comment author: ThisSpaceAvailable 20 January 2014 11:05:44PM 0 points [-]

Is that responding to the "You can't possibly assign equal probability to every program in an infinite space" part of the parent comment? If you assign equal probability to every program in an infinite space, then (assuming "probability" refers to a real number, and not some other mathematical object such as an infinitesimal) either that equal probability is non-zero, in which case the sum is infinite, or the equal probability is zero, in which case the sum is zero. Remember, if all the probabilities are equal, then either all of them are equal to zero, or none are, and it's generally agreed that an infinite number of zeros add up to zero.

Comment author: Eugine_Nier 21 January 2014 03:26:12AM *  -1 points [-]

and it's generally agreed that an infinite number of zeros add up to zero.

Actually 0·∞ is considered an indeterminate form. For example the interval from 0 to 1 has length 1 even though it's composed of infinitely many points each of which has length 0.

See my reply to asr for links to more technical explanations.

Comment author: asr 16 January 2014 05:00:09AM 0 points [-]

Yep, you're confused. You can't possibly assign equal probability to every program in an infinite space, because then the sum of all probabilities will diverge (go to infinity), and you need it to sum up to 1.

Only if you insist on assigning non-zero probability to each individual program.

If you assign an equal probability to every program, and that probability is zero, then the sum, the total probability, is zero and. If you assign an equal nonzero probability, the probability is infinite.

If you want to have a valid probability distribution, you can't assign equal probabilities.

Comment author: Eugine_Nier 16 January 2014 06:31:07AM 1 point [-]

Well, if you're willing to relax the third axiom of probability to finite additivity you can. Alternatively, use an uncountable state space, e.g., the space of all ultra-filters on Turing machines. Also, while we're on the subject, there are numerous types of quasi-probability distributions worth considering.

Comment author: asr 16 January 2014 06:35:30AM 1 point [-]

Ah, fair enough. I had been assuming only real probabilities.

Powered by Reddit Powered by Reddit