people in the ingroup aren't going to think of you as an ally, they're going to think of you as a heretic

The examples that come to mind for me suggest that they're only going to think of you as a heretic if you identify as a member of the ingroup. If you have exactly the same beliefs but place yourself outside of the group, they'll likely ignore you altogether - you're similar enough to not be worth attacking, and small enough to not affect the group's reputation or inner workings.