This is another open problem: given a data set which classifies outcomes in terms of some world model, how can dimensions along which the data set gives little information be identified?

Ambiguity identification may be difficult to do correctly. It is easy to imagine a system which receives value-labeled training data, and then spends the first week querying about wind patterns, and spends the second week querying about elevation differentials, only to query whether brains are necessary long after the programmers lost interest. There is an intuitive sense in which humans “obviously” care about whether the human-shaped-things have brains more than we care about whether the people are on mountains, but it may not be obvious to the system. On the other hand, it could plausibly be the case that there is some compact criterion for finding the most likely or most surprising ambiguities. Further research into ambiguity identifi- cation could prove fruitful.

This seems like an appropriate place to cite my concept learning paper? The quoted paragraphs seems like it's basically asking the question of "how do humans learn their concepts", and ambiguity identification is indeed one of the classic questions within the field. E.g. Tenenbaum 2011 discusses ambiguity identification:

The same principles can explain how people learn from sparse data. In concept learning, the data might correspond to several example objects (Fig. 1) and the hypotheses to possible extensions of the concept. Why, given three examples of different kinds of horses, would a child generalize the word “horse” to all and only horses (h1)? Why not h2, “all horses except Clydesdales”; h3, “all animals”; or any other rule consistent with the data? Likelihoods favor the more specific patterns, h1 and h2; it would be a highly suspicious coincidence to draw three random examples that all fall within the smaller sets h1 or h2 if they were actually drawn from the much larger h3 (18). The prior favors h1 and h3, because as more coherent and distinctive categories, they are more likely to be the referents of common words in language (1). Only h1 scores highly on both terms. Likewise, in causal learning, the data could be co-occurences between events; the hypotheses, possible causal relations linking the events. Likelihoods favor causal links that make the co-occurence more probable, whereas priors favor links that fit with our background knowledge of what kinds of events are likely to cause which others; for example, a disease (e.g., cold) is more likely to cause a symptom (e.g., coughing) than the other way around.

Or see this talk or even e.g. just the first 10 minutes of it, where the concept learning problem is basically defined as being the same thing as the ambiguity identification problem.