MadRocketSci comments on Science Isn't Strict Enough - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (60)
Contrast this to the notion we have in probability theory, of an exact quantitative rational judgment. If 1% of women presenting for a routine screening have breast cancer, and 80% of women with breast cancer get positive mammographies, and 10% of women without breast cancer get false positives, what is the probability that a routinely screened woman with a positive mammography has breast cancer? 7.5%. You cannot say, "I believe she doesn't have breast cancer, because the experiment isn't definite enough." You cannot say, "I believe she has breast cancer, because it is wise to be pessimistic and that is what the only experiment so far seems to indicate." 7.5% is the rational estimate given this evidence, not 7.4% or 7.6%. The laws of probability are laws.
I'm having trouble understanding what is meant by this essay. Where do you think these probabilities in your likelihood estimation problem came from in the first place? What on Earth does it mean to be precisely certain of a probability?
In my mind, these are summaries of prior experience - things that came out of the dynamic state machine we know of as the world. Let's start at the beginning: You have a certain incidence of cancer in the population. What is it? You don't know yet, you have to test - this is a science problem. Hey, what is cancer in the first place? Science problem. You have this test - how are you going to figure out what the probabilities of false positives and false negatives are? You have to try the test, or you have no information on how effective it is, or what to plug into the squares in your probability problem.
Bayesian reasoning can be made to converge on something like a reasonable distribution of confidence if you keep feeding it information. Where does the information you feed it come from? If not experience, the process is eating it's own tail, and cannot be giving you information about the world! Prior probabilities, absent any experience of the problem domain, are arbitrary, are they not?
Also, for more complicated problems such as following a distribution around in dynamic system: You also have to have a model of what the system is doing - that is also an assumption, not a certainty! If your theory about the world is wrong or inapplicable, your probability distribution is going to propagate from it's initial value to a final value, and that final value will not accord with the statistics of the data coming from the external world. You'll have to try different models, until you find a better one that starts spitting out the right output statistics for the right input statistics. You have no way of knowing that a model is right or wrong a-priori. Following bayesian statistics around in a deterministic state machine is a straightforward generalization of following single states around in a deterministic state machine but your idea of the dynamics is distinct (usually far simpler for one thing) from what the world is actually doing.