Comment author: 05 March 2008 01:00:48AM 1 point [-]

Disallowing a symbol for "all events" breaks the definition of a probability space. It's probably easier to allow extended reals and break some field axioms than figure out do rigorous probability without a sigma-algebra.

In response to Trust in Bayes
Comment author: 31 January 2008 05:35:53PM 0 points [-]

Gray Area -- good question, thanks for bring my attention to reservoir sampling. I found a compact description of it in Devroye's "Non-Uniform ..." and for sampling just 1 integer x it looks as follows

1. At step 1, let x=1 2. At step k, let x=k with probability 1/k

sum_i 1/i diverges, this means x will never stop growing

pdf23ds -- I think you can use reservoir sampling for sampling from infinite streams, an interesting question is when it works. For instance, consider an infinite stream of IID bits, 1-element reservoir sampling converges after 1 step. An interesting question is when exactly it works -- my intuition is that it works whenever the stream has finite entropy, and a stationary Markov property

In response to Trust in Bayes
Comment author: 30 January 2008 09:06:48AM 0 points [-]

Interesting post, thanks for the Jaynes link. Related book which is a great read is Szekely's Paradoxes in Probability Theory and Statistics

I think the most intriguing paradoxes are the ones that experts can not agree how to resolve. For instance, take the two envelope paradox: you are presented with two envelopes, one has twice as much money as the other. You are told that first envelope contains x dollars, which envelope should you choose? From expected value calculations, the other envelope has \$1.25x which is larger regardless of x. Turns out that the paradoxical "always pick the other one" solution comes out even if we introduce a proper prior on the amounts in envelopes