Baez' series on network theory and information geometry answer a couple of your questions in a very accessible way.
Here Baez and Fong prove a version of Noether's theorem for Markov processes.
http://math.ucr.edu/home/baez/networks/networks_11.html
Earlier Baez talked about conservation of total probability.
http://johncarlosbaez.wordpress.com/2011/09/09/network-theory-part-8/
Here he relates the distribution of existing species to a prior and how Bayes' rule says how the number of each species changes over time.
http://johncarlosbaez.wordpress.com/2011/05/26/information-geometry-part-8/
One of the sharpest and most important tools in the LessWrong cognitive toolkit is the idea of going meta, also called seeking whence or jumping out of the system, all terms crafted by Douglas Hofstadter. Though popularized by Hofstadter and repeatedly emphasized by Eliezer in posts like "Lost Purposes" and "Taboo Your Words", Wikipedia indicates that similar ideas have been around in philosophy since at least Anaximander in the form of the Principle of Sufficient Reason (PSR). I think it'd be only appropriate to seek whence this idea of seeking whence, taking a history of ideas perspective. I'd also like analyses of where the theme shows up and why it's appealing and so on, since again it seems pretty important to LessWrong epistemology. Topics that I'd like to see discussed are: