All of kylrth's Comments + Replies

kylrth70

In How Not to Network a Nation, Benjamin Peters points out that Soviet computer networking projects failed partially due to this same centralizing idealism. Soviet networking projects all saw the ideal network as a central nervous system, connecting productive outposts with the central "brain" in Moscow. By contrast, the developers of ARPANET envisioned their network as a brain with the individual computers as the neurons. The formulation of ARPANET was as a distribution of homogeneous elements, whereas the formulation of the OGAS (the largest Soviet proje... (read more)

kylrth100

This reminds me a lot of what I would do when I was trying to access the Spirit of God to get divine guidance, back when I was Mormon. The essential distinction is in your sentence:

Of course, the fact that you’ve accurately expressed your brain’s sense of what’s going on doesn’t mean you’ve found the bona-fide truth.

In the process of getting my mind untied from my Mormon upbringing, I became a lot more connected with my rational mind and more untrusting of my subconscious. This is a good reminder that you have to pay attention to that subconscious proc... (read more)

kylrthΩ040

Forget OOD for a minute; ERM can't even learn to avoid spurious correlations that have counterexamples in the training data. Datasets like Waterbirds (used in that previously linked paper) are good toy datasets for figuring this out. I think we need to solve this problem first before trying to figure out OOD generalization.

1RobertKirk
I expect that these kinds of problems could mostly be solved by scaling up data and compute (although I haven't read the paper). However, the argument in the post is that even if we did scale up, we couldn't solve the OOD generalisation problems.