the pre-modern Catholic Church was opposed to the concept of the Earth orbiting the Sun with the deliberate purpose of hindering scientific progress and to keep the world in ignorance.

Are you sure you are not attacking a strawman/nut picking? I mean, there are certainly people who believe that, but is it really a representative position among atheists (*)?

(* Here I assume we are talking about atheists who don't partecipate to a secular/political religion, as these ones lend towards fanaticism, therefore I suppose they are more likely to hold false and inflammatory beliefs as long as they support their ideology and demonize competing ideologies)

Gravity. Why do the objects have weight, and why are they all pulled towards the center of the Earth? Why don't objects fall off the Earth on the other side of the planet? Remember, Newton wasn't even born yet! The geocentric view had a very simple explanation, dating back to Aristotle: it is the nature of all objects that they strive towards the center of the world, and the center of the spherical Earth is the center of the world.

So why don't the Sun and the planets fall on the Earth?

In the Aristotelic model you still needed a distinction between terrestial mechanics, ruling the sublunary sphere where gravity but also friction, drag, decay, and all kinds of irreversible processes occur, and celestial mechanics, ruling the celestial spheres, where everything moves like clockwork rather than "falling down" without any apparent energy source and doesn't show any signs of decay and irreversibility that 17th century people could have observed with instruments of their time.

Before Newton unified terrestial and celestial mechanics, you needed to keep them separate whether you were using a geocentric or a heliocentric model. You still needed a sublunary sphere sphere around the Earth where things slow down and fall and break and decay on their way towards the End of Time, while God and the Angels watch us from their perfect and immutable Heaven.
Neither the geocentric nor the heliocentric model had an advantage in terms of explanatory power here.