For my own part, I'd say that I need to do more work brainstorming through possible conversation paths, and especially identifying why this all bothers me so much.

You might find that the sense of bother never quite goes away. In my experience, there are some (bad) arguments which will always feel right, and some great arguments which will always feel wrong. There are many ex-theists, for example, who still fear hell, even though they know it doesn't exist.

I admit that I still don't find some of the counters to the Teleological Argument to be very satisfying. I suspect that this is a leftover from my theist days, and I'm really not sure how to get rid of that nibbling uncertainty. I'm not even sure if I want to, because I can use it to try to understand people like my old self. Yeah, the question "If god is so omnipotent, can he make a rock so big that even he can't lift it" points to an obvious absurdity, but I remember hearing it and thinking that the questioner was just trying to be cute and clever, and laughing at the question. It never bothered me after that, and it never contributed to my deconversion. So why should "If everything has a cause, then what caused god?" bother a theist?

One possible trick to help build your confidence is to notice when you do things that are known to be wrong. Make a game out of catching yourself when you engage in the most dualistic or irrational of behaviors (like seeing faces in wood, or thinking you have control over the upcoming dice roll, or even being scared of ridiculously improbable things because of watching a horror movie in the dark the other day). If you haven't taken the time, study the cognitive biases that make you susceptible to wrongness, and try to catch your biggest offenders.

This worked to help me gain an understanding for why I was uncomfortable with losing my religion, and got me out of the corner, so to speak.