That's a good hypothesis. Unfortunately this doesn't come from asking people, "How do you know when you're rationalizing?" or any variant thereof. The original problem arose when we could not for the life of us convey to some individuals why the Litany of Tarski might be useful. We gave examples from our own lives and watched these individuals just blink and say, "Huh. Yeah, I guess I just don't relate to that at all."
Anna Salamon and I are confused. Both of us notice ourselves rationalizing on pretty much a daily basis and have to apply techniques like the Litany of Tarski pretty regularly. But in several of our test sessions for teaching rationality, a handful of people report never rationalizing and seem to have little clue what Tarski is for. They don't relate to any examples we give, whether fictitious or actual personal examples from our lives. Some of these people show signs of being rather high-level rationalists overall, although some don't.