AlexSchell comments on How do I make sample size matter? - Less Wrong

1 [deleted] 22 March 2015 05:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (9)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 22 March 2015 06:46:31PM *  3 points [-]

How did they come up with the likelihood distribution? Maybe they sampled 100 products from each machine and for each sample counted the number of faulty products. Maybe they sampled 1.000.000 products from each machine...

We don't know which sample size is used: the likelihood distribution doesn't reveal this.

Implicitly, they have an infinite sample size, because the distribution on P(B|A_1) is infinitely precise. Suppose we also wanted to learn P(B|A_1) from the history of the factory: then we might model it as having a fixed rate of defective outputs, and the probability we assign to particular defect rates is a beta distribution. We might start off with a Jeffreys prior and then update as we see the tool produce defective or normal products, eventually ending up with, say, a beta(5.5,95.5) for tool A_1.

Exercise for the reader: given that hyperparameter distribution for P(B|A_1) (and similar ones for A_2 and A_3), do we need the full hyperparameter distribution for all three tools to determine the probability that a known defective output came off of A_1, or can we get the same answer using only a handful of moments from each distribution?

Comment author: AlexSchell 22 March 2015 08:15:34PM *  1 point [-]

Hayrff guvf vf gevpxvre guna vg frrzf, whfg gur svefg zbzrag bs rnpu qvfgevohgvba fubhyq qb. (Sbe guvf ernfba V qvfnterr gung gur Jvxv negvpyr vzcyvpvgyl nffhzrf vasvavgr fnzcyr fvmr. Gur pbaqvgvbany cebonovyvgvrf hfrq va gur pnyphyngvba ner gur svefg zbzragf (= pbafgnagf) bs gur erfcrpgvir cnenzrgre qvfgevohgvbaf, abg gur cnenzrgref gurzfryirf (= enaqbz inevnoyrf).)

Comment author: Vaniver 22 March 2015 09:58:59PM 0 points [-]

whfg gur svefg zbzrag bs rnpu qvfgevohgvba fubhyq qb.

Yep.

(Sbe guvf ernfba V qvfnterr gung gur Jvxv negvpyr vzcyvpvgyl nffhzrf vasvavgr fnzcyr fvmr. Gur pbaqvgvbany cebonovyvgvrf hfrq va gur pnyphyngvba ner gur svefg zbzragf (= pbafgnagf) bs gur erfcrpgvir cnenzrgre qvfgevohgvbaf, abg gur cnenzrgref gurzfryirf (= enaqbz inevnoyrf).)

V zbfgyl nterr jvgu guvf. V nterr gung lbh bayl arrq gur svefg zbzrag bs lbhe cbfgrevbe gb pnyphyngr jung gurl nfx sbe, ohg V guvax gung gurz cebivqvat gur ulcrecnenzrgre nf n fvatyr qngncbvag vf vzcyvpvgyl pynvzvat na vasvavgr cerpvfvba naq guhf na vasvavgr fnzcyr fvmr (be n pregnva haqreylvat zbqry), va gur fnzr jnl gung zl orgn qvfgevohgvba zbqry gung qbrfa'g vapyhqr gvzr vf vzcyvpvgyl pynvzvat gung gur znpuvar vf rdhnyyl yvxryl gb cebqhpr qrsrpgvir zngrevny ng nyy gvzrf qhevat vgf bcrengvba. Hayrff nffhzcgvbaf / zbqry fvzcyvsvpngvbaf yvxr gung ner rkcyvpvgyl npxabjyrqtrq, vg znxrf frafr gb pnyy gurz vzcyvpvg.