Qualia or not, it seems a straightforward consequence of Tegmark's mathematical universe hypothesis that we derive significant proportions of our measure from simulators.

that if I replaced your brain with something else embodying the same computation that you would feel yourself to be unharmed -- what is your justification for believing this?

I think the quantum physics sequences basically argues this in a really roundabout way.

http://lesswrong.com/lw/qx/timeless_identity/