If you enjoyed going to church as a Christian, and considered it enough to make this post, then you should probably just go. There is not much penalty for trying.

I go to a UU church, which looks kind of similar. (They are not all atheist, but they are all different things and agree to disagree about theology.) I don't really enjoy the singing that much, at least not the hymns, and I still enjoy the experience as an atheist. Just don't expect to get the same level of intelligence or rationality you get from here though. If you are looking for good philosophical discussion, that probably isn't the place to get it.