I do something vaguely similar to this: a lot of my introspective thought takes the form of imagined dialogues with someone who I think would take the other side of the argument, or at the very least need convincing. For example, in a lot of my thinking about metaethics recently, (an imaginary construction of) Eliezer gets cast in the dissenting role. This seems to help me avoid arguments I can't justify, though far from perfectly.

(I may have a metaethics post in the pipeline.)