While this is certainly a nasty pitfall of rationalization, it is necessary to rehearse the evidence from time to time, for those of us without perfect memories. Otherwise, we end up in the situation "I know there was a good reason I believed this but I don't remember what it was"; this occurs to me far too often. Retracing all of the evidence that led to a particular belief is terribly time-consuming and impractical ("I know this was in a neuroscience book I read three years ago..."). Forgetting why you hold a particular belief is almost as bad as having no reason at all, and every rationalist should naturally strive to avoid this.
Of course, the time to rehearse why you hold a particular belief is not when being confronted with opposing arguments.