IAWYC, but let me point out that you are describing your position rather than supporting it.

As a way of supporting it, imagine that there's some other universe with physics that encapsulate ours: the Dark Lords of the Matrix can cheaply run a faithful simulation of anything that happens in this universe on that universe's computers. It's clear to me that, given the setup you two are discussing, the Dark Lords would see that those extra bits aren't doing anything at all; they can be removed without altering anything that they could observe in the simulation.

Now, anything we can observe about our conscious experience is assumed to have an effect on our brains as we think of it, and thus the Dark Lords could observe it as well. (Namely, if there's a distinction between a "physically real" world and another one that's just "mathematically possible" with an identical copy of you, there's no difference from the Dark Lords' perspective between the description of your brain thinking "But I really exist!" and the description of your copy's brain thinking the same thing.) Note that this is an instance of the GAZP in action.

So by Occam's Razor, I don't think there's a justification for adding extra bits to the way the universe is described, when literally nobody within or outside the universe can be pointed to as having justification that those bits are one way or another.