All of Eyal Roth's Comments + Replies

I'm going to take the role of the "undergrad" here and try to interpret this in the following way:

Given that a hypothesis is true -- but it is unknown to be true -- it is far more likely to come by a "statistically significant" result indicating it is wrong, than it is likely to come by a result indicating that another hypothesis is significantly more likely.

In simpler words - it is far easier to "prove" a true hypothesis is wrong by accident, than it is to "prove" that an alternative hypothesis is superior (a better estimator of reality) by accident.

Would you consider this interpretation accurate?

I feel like this paragraph might be a little necessary for someone who haven't read the bayes rule intro, but on the other hand is a bit off-topic in this context and quite distracting, as it raises questions which are not part of this "discussion"; mainly, questions regarding how to approach "one-off" events.

Say, what if I can't quantify the outcome of my decision so nicely like in the case of a bet? What if I need to decide whether to send Miss Scarlet to prison or not based on these likelihood probabilities?

This "argument" by the "scientist" doesn't IMO represent how a true experimentalist would approach the issue; they would not necessarily be so opposed to trying new ways of improving their methods, as long as it is done step by step without replacing the entire system over night (just like the "bayesian" explains in the next paragraph).

This is also a bit side-tracking as it opens up the topic of how much more "experience" computer scientists have given the simpler and much more reproducible systems they're dealing with -- especially in the modern commercial world -- in contrast with natural sciences (I'm a programmer myself, so I'm a bit biased on this).

Not quite. The private state of mind of the researcher changes nothing. It's only the issue of which question is asked.

In this case, the two questions are (a) what is the probability of such an event occurring after tossing a fair coin 6 times and (b) what is the probability of such an event occurring if a fair coin is tossed until it lands tails.

The meaning of the questions does not change, nor do the answers to them. It is only a matter of what question is being asked -- which is obviously important when conducting a study, but is not so counter-intuitive (and much less confusing) when presented in such way (IMO).

Broken link :(

I really love this example; it is one of the few I've managed to find online which actually helped me understand the differences between the approaches.

Why the capital letters? Is this suppose to refer or to link to something?

I believe it is essential to explain why it is independent in the case of the bathtub example and not in the other examples.

In the bathtub example, the evidence presents an event which is directly described by the assessed trait; i.e, the fairness of a coin is directly concerned with the appearance of either heads or tails. In contrast, the definition of the degree of "spamness" in an email is not directly concerned with the appearance of a word in the email, but is rather concerned with the abstract concept of the meaning a person assigns to the email.

The... (read more)

I find the entire explanation described below very misleading and perhaps even largely incorrect. The workshop participants had it wrong mostly for two reasons:

1. They did not consider what is the likelihood of visiting a museum / workplace given any other alternative (mutually exclusive) relationship - not strangers but also not romantically involved; i.e, friends. Being acquaintances is not a relevant type of a relationship as it is not mutually exclusive with a romantic relationship (a pair can be both dating and working together).

2. They did not know th

Easier to grasp perhaps, but dangerously misleading. Increasing the likelihood of an event from 10^-100 to 10^-99 is very different and much less significant than increasing it from 10^-2 (1%) to 10^-1 (10%). I hope this is covered later in this guide.

It is really confusing to apply one of the initial steps of a study as evidence to a prior which is the result (last step) of the same study.

But that really gives a different magnitude to the evidence. Why not be consistent with the log base?

For example, if we were to use log base 2, the prior would be ~16.6 magnitudes strong and the evidence ~8. This means that the evidence would alter the prior by (slightly) less than half the order of magnitudes, where's in the case of log base 10 the alteration is (slightly) more than half the order of magnitudes (5 vs 2.7).

Also, imagine the absurd choice of log base 100k. The prior would remain practically intact in terms of this kind of order of magnitudes.

Wrong, they are exactly the same distances. I read the next paragraph so I get where you were going with this, but I find it confusing to start off with a blatantly wrong claim, especially when the next line compares 0.11 to 0.1 (11% to 10%) -- not to 0.100001 -- in order to describe how the significance of 0.00001 gets "lost in translation" when speaking in probabilities and not in bits.

I really have a hard time understanding the point of this section.

What difference is there between calculating the posterior given an evidence -- thus updating the future prior -- and questioning the prior "in the first place"? Isn't this the whole point of the process? to examine the prior and question it in case of an extraordinary evidence?