= 27d567bc2d48d9f40e9ccc20ea370b15)
Elo comments on Open Thread, Jul. 27 - Aug 02, 2015 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (220)
How to tell if a process is out of control-- that weirdness might be random, but you should check in case it isn't.
I received feedback from some friends to suggest that this is not applicable to large datasets - i.e. big data. I play with my own quantified self data sets of 100,000+ lines from time to time. (think minutised data at 1440 minutes a day for a year and counting). Can you discuss this more (maybe in the next open thread?)
It shouldn't be too challenging to apply Nelson rules to 100k lines, but the point of statistical process control is continuous monitoring--if you weigh yourself every day, you would look at the two-week trend every day, for example. Writing a script that checks if any of these rules are violated and emails you the graph if that's true seems simple and potentially useful.
I think what Elo's friends mean is that the constants hard-coded into Nelson's rules reflect some assumption on sample size. With a big sample, you'll violate them all the time and it won't mean anything. But they are a good starting point for tuning the thresholds.
If you have many parallel sensors, then yes, a flag that occurs 5% of the time due to noise will flag on at least one twentieth of your sensors. Elo's point, as I understood it, was that they have a long history--which is not relevant to the applicability of SPC.
The long history is not relevant, but the frequency. Most of Nelson's rules are 1/1000 events. If you don't expect trends to change more often than 1000 measurements, that's too sensitive. I don't know what Elo is measuring every minute, but that probably is too sensitive and most of the hits will be false positives. (Actually, many things will have daily cycles. If Nelson notices them, that's interesting, but after removing such patterns, it will probably be too sensitive.)
I see what you mean--yes, if you're oversampling, you need to downsample / average / whatever to get to the reasonable frequency, otherwise you'll just be doing SPC on noise.
All I know about it is that the link looked like it was worth mentioning here. If you're interested in further discussion, you should bring it up yourself..