Looking for ideas about Epistemology related topics

Onemorenickname

Notes :

"Epistemy" refers to the second meaning of epistemology : "A particular theory of knowledge".
I'm more interested in ideas to further the thoughts exposed here than exposing them.

Good Experiments

The point of "Priors are useless" is that if you update after enough experiments, you tend to the truth distribution regardless of your initial prior distribution (assuming its codomain doesn't include 0 and 1, or at least that it doesn't assign 1 to a non-truth and 0 to a truth). However, "enough experiments" is magic :

The pure quantitative aspect : you might not have time to do these experiments in your lifetime.
Having independent experiments is not defined. Knowing which experiments are pairwise independent embeds higher-level knowledge that could easily be used to derive truths directly. If we try to prove a mathematical theorem, comparing the pairwise success probability correlations of different approaches would give much more insights and results than trying to prove it as usual.
We don't need pairwise independence. For instance, assuming we assume P=/=NP because we couldn't prove it, we assume so because we expect all used techniques not to be all correlated together. However, this expectation is ether wrong (Small list of fairly accepted conjectures that were later disproved), or stems from higher-order knowledge (knowledge about knowledge). Infinite regress.

Good Priors

However, conversely, having a good prior distribution is magic too. You can have a prior distribution affecting 1 to truths, and 0 to non-truths. So you might want the additional requirement that the prior distribution has to be computable. But there are two problems :

There aren't many known computable prior distribution. Occam's razor (in term of Kolmogorov complexity in a given language) is one. But fails miserably in most interesting situations. Think of poker, or a simplified version thereof : A+K+Q. If someone bets, the simplest explanation is that he has good cards. Most interesting situations where we want to apply bayesianism are from human interactions (we managed to do hard sciences before bayesianism, and we still have troubles with social sciences). As such, failing to take into account bluff is a big epistemic fault for a prior distribution.
Evaluating the efficiency of a given prior distribution will be done over the course of several experiments, and hence requires a higher order prior distribution (a prior distribution over prior distributions). Infinite regress.

Epistemies

In real-life, we don't encounter these infinite regresses. We use epistemies. An epistemy is usually a set of axioms, and a methodology to derive truths with these axioms. They form a trusted core, that we can use if we understood the limits of the underlying meta-assumptions and methodology.

Epistemies are good, because instead of thinking about the infinite chain of higher priors every time we want to prove a simple statement, we can rely on an epistemy. But they are regularly not defined, not properly followed or not even understood. Leading to epistemic faults.

Questions

As such, I'm interested in the following :

When and how do we define new epistemies ? Eg, "Should we define an epistemy for evaluating the Utility of actions for EA ?", "How should we define an epistemy to build new models of human psychology ?", etc.
How to account for epistemic changes in Bayesianism ? (This requires self-reference, which Bayesianism lacks.)
How to make sense of of Scott Alexander's yearly predictions ? Is it only a blackbox telling us to bet more on future predictions, or do we have a better analysis ?
What prior distributions are interesting to study human behavior ? (For a given restricted class of situations, of course.)
Are answers to the previous questions useful ? Are the previous questions meaningful ?

I'm looking for ideas and pointers/links.

Even if your thought seems obvious, if I didn't explicitly mention it, it's worth commenting it. I'll add it to this post.

Even if you only have idea for one of the question, or a particular criticism of a point made in the post, go on.

Thank you for reading this far.

Notes :

"Epistemy" refers to the second meaning of epistemology : "A particular theory of knowledge".
I'm more interested in ideas to further the thoughts exposed here than exposing them.

Good Experiments

The pure quantitative aspect : you might not have time to do these experiments in your lifetime.
Having independent experiments is not defined. Knowing which experiments are pairwise independent embeds higher-level knowledge that could easily be used to derive truths directly. If we try to prove a mathematical theorem, comparing the pairwise success probability correlations of different approaches would give much more insights and results than trying to prove it as usual.
We don't need pairwise independence. For instance, assuming we assume P=/=NP because we couldn't prove it, we assume so because we expect all used techniques not to be all correlated together. However, this expectation is ether wrong (Small list of fairly accepted conjectures that were later disproved), or stems from higher-order knowledge (knowledge about knowledge). Infinite regress.

Good Priors

There aren't many known computable prior distribution. Occam's razor (in term of Kolmogorov complexity in a given language) is one. But fails miserably in most interesting situations. Think of poker, or a simplified version thereof : A+K+Q. If someone bets, the simplest explanation is that he has good cards. Most interesting situations where we want to apply bayesianism are from human interactions (we managed to do hard sciences before bayesianism, and we still have troubles with social sciences). As such, failing to take into account bluff is a big epistemic fault for a prior distribution.
Evaluating the efficiency of a given prior distribution will be done over the course of several experiments, and hence requires a higher order prior distribution (a prior distribution over prior distributions). Infinite regress.

Epistemies

Questions

As such, I'm interested in the following :

When and how do we define new epistemies ? Eg, "Should we define an epistemy for evaluating the Utility of actions for EA ?", "How should we define an epistemy to build new models of human psychology ?", etc.
How to account for epistemic changes in Bayesianism ? (This requires self-reference, which Bayesianism lacks.)
How to make sense of of Scott Alexander's yearly predictions ? Is it only a blackbox telling us to bet more on future predictions, or do we have a better analysis ?
What prior distributions are interesting to study human behavior ? (For a given restricted class of situations, of course.)
Are answers to the previous questions useful ? Are the previous questions meaningful ?

I'm looking for ideas and pointers/links.

Even if your thought seems obvious, if I didn't explicitly mention it, it's worth commenting it. I'll add it to this post.

Even if you only have idea for one of the question, or a particular criticism of a point made in the post, go on.

Thank you for reading this far.

I suspect that you are leaping to the idea of "infinite regress" much too quickly, and also failing to look past it or try to simply "patch" the regress in a practical way when you say:

Evaluating the efficiency of a given prior distribution will be done over the course of several experiments, and hence requires a higher order prior distribution (a prior distribution over prior distributions). Infinite regress.

Consider the uses that the Dirichlet distribution is classically put to...

Basically, if you stack your distributions two or three (or heaven forbid four) layers deep, you will get a LOT of expressiveness and yet the number of steps up the abstraction hierarchy still can be counted with the fingers of one hand. Within only a few thousand experiments even the topmost of your distributions will probably start acquiring a bit of shape that usefully informs subsequent experiments.

Probably part of the reason you seem to give up at the first layer of recursion and just assume that it will recurse unproductively forever is that you're thinking in terms of some small number of slogans (axioms?) that can be culturally transmitted in language by relatively normal people engaging in typical speech patterns, perhaps reporting high church Experiments that took weeks or months or years to perform, and get reported in a peer reviewed journal and so on.

Rather than conceptually center this academic practice, perhaps it would make more sense to think of "beliefs" as huge catalogues of microfacts, often subverbal, and "experiments" as being performed by even normal humans on the time scales of milliseconds to minutes?

The remarkable magical thing about humans is not that we can construct epistemies, the remarkable thing is that humans can walk, make eye contact and learn things from it, feed ourselves, and pick up sticks to wave around in a semi-coordinated fashion. This requires enormous amounts of experimentation, and once you start trying to build them from scratch yourself you realize the models involved here are astonishing feats of cognitive engineering.

Formal academic science is hilariously slow by comparison to babies.

The problems formal intellectual processes solve is not the problem of figuring things out quickly and solidly, but rather (among other things) the problem of lots of people independently figuring out many of the same things in different orders with different terminology and ending up with the problem of Babel.

Praise be to Azathoth, for evolution already solved "being able to learn stuff pretty good" on its own and delivered this gift to each of us as a birthright. The thing left to us to to solve something like the "political economy of science". Credit assignment. Re-work. Economies of scale... (In light of social dynamics, Yvain's yearly predictions start to make a lot more sense.)

A useful keyword here is "social epistemology" and a good corpus of material is the early work of Kevin Zollman, including this overview defending the conceptual utility of social epistemology as a field.

I suspect that you are leaping to the idea of "infinite regress" much too quickly, and also failing to look past it or try to simply "patch" the regress in a practical way when you say

No. I mention the practical patch right after : epistemies.

The remarkable magical thing about humans is not that we can construct epistemies, the remarkable thing is that humans can walk, make eye contact and learn things from it, feed ourselves, and pick up sticks to wave around in a semi-coordinated fashion.

Formal academic science is hilariously

... (read more)