For example, given an object that can either have or not have each of Nn properties, there are 22^Nn different descriptions corresponding to the possible objects of that kind (a number exponential in N)n). The number of possible concepts, each of which either includes a given description or doesn't, is one exponential higher: 22^(22^Nn)
In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, because we don't live in a maximum-entropy universe of absolute chaos, we can talk about repeatable higher-level regularities in the world instead: we can distinguish particular configurations of matter as instantiations of object concepts like chair or human, and say that these objects have particular properties, like red or alive.
The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the superexponential conceptspace of the number of different ways to categorize these possible objects is even vaster. If (for purposes
For example, given an object that can either have or not have each of exposition)N properties, there are 2nN different descriptions corresponding to the possible objects of that kind (a number exponential in the world which either are, or are not instantiations of some concept, then theN). The number of possible conceptsconcepts, each of which either includes a given description or doesn't, is 2^one exponential higher: 2(2nN (for)
Without an inductive bias, restricting attention to only a small portion of possible concepts, it's not possible to navigate the mathematics involved,conceptspace: to learn a concept, a "fully general" learner would need to see all the individual examples that define it. Using powersetprobability). Most to mark the extent to which each possibility belongs to a concept is another approach to express prior information and its control over the process of these possible concepts are complicated enough to be ruled out a priori by your prior; you can't expect to encounter enough evidence to cut down such a large space. The work of proper inference is to "carve reality at its joints"; to find simple generalizations and simple concepts that let you make useful inferences with respect to your goals.learning.
In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, because we don't live in a maximum-entropy universe of absolute chaos, we can talk about repeatable higher-level regularities in the world:world instead: we can distinguish particular configurations of matter as instantiations of object concepts like chair or human, and say that these objects have particular properties, like red or alive.
The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the superexponential conceptspace of the number of different ways to categorize these possible objects is even vaster.
In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, because we don't live in a maximum-entropy universe of absolute chaos, we can do this, and talk about repeatable higher-level regularities in the world: we can distinguish particular configurations of matter as instantiations of object concepts like chair or human, and say that these objects have particular properties, like red or alive.
The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the superexponential conceptspace of the number of different ways to categorize these possible objects is even vaster. If (for purposes of exposition) there are n objects in the world which either are, or are not instantiations of some concept, then the number of possible concepts is 2^n (for the mathematics involved, see powerset). Most of these possible concepts are complicated enough to be ruled out a priori by your prior; you can't expect to encounter enough evidence to cut down such a large space. The work of proper inference is to "carve reality at its joints"; to find simple generalizations and simple concepts that let you make useful inferences with respect to your goals.
The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the superexponential conceptspace of the number of different ways to categorize these possible objects is even vaster.
In order to do inference, we constantly need to make use of categories and concepts: it is neither possible nor desirable to deal with every unique arrangement of quarks and leptons on an individual basis. Fortunately, we don't live in a maximum-entropy universe of absolute chaos, we can do this, and talk about repeatable higher-level regularities in the world: we can distinguish particular configurations of matter as instantiations of object concepts like chair or human, and say that these objects have particular properties, like red or alive.
The sheer number of distinct configurations in which matter could be arranged is unimaginably vast, but the superexponential conceptspace of the number of different ways to categorize these possible objects is even vaster.
What are concepts?
The sheer number ofdistinct configurationsin which matter could be arranged isunimaginablyvast, but thesuperexponential conceptspaceof the number of different ways tocategorizethese possible objects is even vaster.Blog postsKey Posts