To clarify what I think, take "me" as a node, and recursively build a causality graph (Pearl's thing) of all the causes that lead into that node. By some theorem somewhere, that graph will be connected. Then label that graph "my map of the universe" and label it's compressing model "physics". That is what "materialism" means to me.

I think you are making a category error with respect to what Pearl's theory actually does.