Let me see if I understand what you are saying. You seem to be analogising the problem to detecting whether a coin or die is biased? That is, a biased die has some frequency of coming up 6 which is not one-sixth; and you are proposing beta(0.5, 0.5) as my prior distribution for that frequency. I roll the die 100 times, getting some number of sixes; for each point in the [0, 1] space of possible frequencies, I do Bayes. This presumably concentrates my probability around some particular frequency, and if that frequency is different from 0.16667 I say the die is biased.
Now, the analogy is that commenting out the line is equivalent to removing a piece of gum from the die. So I perform the test twice, once with and once without the gum; and if the results differ then I say that the gum changed the bias, and if the un-gummed die is fair then the gum caused the bias. Or, going back to the program, the mean of my probability distribution for the crash frequency in the presence of the line may be high relative to the mean without the line, and that's what we mean by "We think this line is causing the crash". Right?
So, if I understand correctly, you are proposing the same math as gjm did below, but suggesting a specific prior
P(p, q) = beta(p; 0.5, 0.5) beta(q; 0.5, 0.5).
Is that correct?
No. My way was assuming that it either crashes exclusively because of that line or exclusively because of something else. Furthermore, I only gave priors for if it's given which it's doing.
Let A = that line is the cause of the crash.
Let q!=x mean that this is true for any value of q besides x.
P(p,0|A) = beta(0.5,0.5)(p)
P(p,q!=0|A) = 0
P(p,p|!A) = beta(0.5,0.5)(p)
P(p,q!=p|A) = 0
I have successfully confused myself about probability again.
I am debugging an intermittent crash; it doesn't happen every time I run the program. After much confusion I believe I have traced the problem to a specific line (activating my debug logger, as it happens; irony...) I have tested my program with and without this line commented out. I find that, when the line is active, I get two crashes on seven runs. Without the line, I get no crashes on ten runs. Intuitively this seems like evidence in favour of the hypothesis that the line is causing the crash. But I'm confused on how to set up the equations. Do I need a probability distribution over crash frequencies? That was the solution the last time I was confused over Bayes, but I don't understand what it means to say "The probability of having the line, given crash frequency f", which it seems I need to know to calculate a new probability distribution.
I'm going to go with my intuition and code on the assumption that the debug logger should be activated much later in the program to avoid a race condition, but I'd like to understand this math.