You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

ChristianKl comments on What are the most common and important trade-offs that decision makers face? - Less Wrong Discussion

10 Post author: Andy_McKenzie 03 November 2014 05:03AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (30)

You are viewing a single comment's thread. Show more comments above.

Comment author: ChristianKl 03 November 2014 10:26:48AM 3 points [-]

I think that's Sensitivity vs Specificity

Comment author: Froolow 03 November 2014 12:26:17PM *  5 points [-]

They are slightly different, but in practical terms they describe the same error; sensitivity and specificity are properties of a test while Type I and II errors are properties of a system, but both errors are basically saying, "Our test is not perfectly accurate so if we want to catch more people with a disease we need to misdiagnose more people"

To illustrate the distinction, consider a test which is 90% sensitive and 90% specific in a population of 100 where a disease has a 50% prevelance. This means 50 people have the disease, of which the test will identify 45 as having the problem (90% sensitive). 50 people are free of the disease, of which the test will correctly identify 45 (90% specific). So if diagnosed your probability of the diagnosis being a Type I error is 5/50 = 10% (if given the all clear the same logic applies for a Type II error). You derive this from the number of people in the population who were told they have a disease who were incorrectly diagnosed divided by the total population who were told they have a disease (rightly or wrongly)

But if the disease prevelance changes due to demographic pressue to 10% then 10 people have the disease of whom 9 are diagnosed, and 90 people are disease-free of whom 81 are given the all-clear. This means the probabilities of the different 'Type' errors change dramatically; now 9/18 = 50% for a Type I error and 1/82 ~ 1.2% for a Type II error. But the sensitivity and specificity of the test are completely unchanged.