fryolysis — LessWrong

LESSWRONG
LW

A comment on Bayesian vs. frequentist statistical practice

7mo

Both schools use the same mathematical modeling tool, the probability measure, but they use it to model different things in the real world. Frequentists use it as a model for repeatable events like tossing a coin, while Bayesians use it to represent the degree of subjective belief to any statement.

I am more interested in the main causes of why Bayesian and frequentist statistical analysis gives different results on the same problem and the same data. Both camps, while trying to solve a given statistical problem, often make certain assumptions about the mechanism that generates the data. The core distinction and the source of fierce debate between them seems to be deciding when... (read 227 more words →)

Blunt Razor

fryolysis

In this post, I would like to share a simple observation of mine which is about deriving some crude form of Occam's razor from a small set of assumptions.

Assumptions:

Subjective Bayesian probability is used to model an agent's degree of belief in hypotheses.
The set of hypotheses $H$ that we assign beliefs is countably infinite.
Complexity of a hypothesis is represented with a positive integer where bigger integers represent more complex hypotheses.
For any $h \in H$ there exists finitely many hypotheses in $H$ that have the same complexity as $h$ .

When we accept these assumptions it is easy to prove the following statement.

For any $h \in H$ there are only finitely many hypotheses in $H$ that are more complex and more probable or less complex and less probable than $h$ .

Proof.

The set of... (read 402 more words →)

Infinite tower of meta-probability

fryolysis

Suppose that I have a coin with probability of heads $p$ . I certainly know that $p$ is fixed and does not change as I toss the coin. I would like to express my degree of belief in $p$ and then update it as I toss the coin.

Using a constant pdf to model my initial belief, the problem becomes a classic one and it turns out that my belief in $p$ should be expressed with the pdf $f (x) = (\frac{n}{h}) x^{h} (1 - x)^{n - h}$ after observing $h$ heads out of $n$ tosses. That's fine.

But let's say I'm a super-skeptic guy that avoids accepting any statement with certainty, and I am aware of the issue of parametrization dependence too. So I dislike this solution and instead choose to attach beliefs to statements of... (read 603 more words →)

An attempt to break circularity in science

fryolysis

We do science using the data we collected through our senses and we use science to understand how our senses work. Although I lack any rigorous formulation of the problem, the following plan seems interesting and I want to share it with you.

$p =$ "Human senses collect accurate data about reality."

$q =$ "Reality is governed by laws of physics."

Now we want to know $Pr (p)$ and $Pr (q)$ . I don't know any way to directly calculate these values. It seems to me that $Pr (p | q)$ and $Pr (q | p)$ are easier to get our hands dirty. The former seems to be a scientific question and I heard that there are research done already, and the latter could be addressed by something like Solomonoff's Induction. At this point, we... (read 214 more words →)