Zubon: This also feels a bit like the "outside view" Eliezer criticizes Robin for using to make predictions.

Problem is not in using outside view, but in using outside view that doesn't really apply to what it's being applied to, in trying to infer properties from surface similarities that don't indicate that objects have similar causal structure. If you are studying a single object, statistics of arbitrarily surface level provides valid ground for predictions, if this single object doesn't change its causal structure while under study.