This is highly related to Bayesian surprise, the KL-divergence between a posterior and prior distribution. I'm actually working on some research right now to try to see in the setting of PAC-learning whether correlated statistical queries regarding Bayesian surprise buys you something in terms of learning efficiency. If a theoretical result can be found, it would suggest that "salience" truly is a good way to focus attention and so proxies for salience w.r.t. Nature's distribution would be advisable in practice much the same way that proxies for Kolmogorov complexity are advisable in practice when simplicity is of importance (such as gzip which is actually used when studying genetic sequences).

A further result I would like to think about is what role Bayesian surprise queries (or VoI queries) might play in scientific research at large. To me, VoI-type reasoning leads me to believe our current journal / conference system is deeply flawed if the goal is to update some sort of community prior to a better posterior efficiently. It will be interesting to see where this leads.